Skip to content

Bump ovn-controller CPU request to 50m to prevent CPU starvation - #3051

Open
shreyasbe wants to merge 1 commit into
openshift:masterfrom
shreyasbe:UDN_CORENET_7217_adjust_cpu_priority
Open

Bump ovn-controller CPU request to 50m to prevent CPU starvation#3051
shreyasbe wants to merge 1 commit into
openshift:masterfrom
shreyasbe:UDN_CORENET_7217_adjust_cpu_priority

Conversation

@shreyasbe

@shreyasbe shreyasbe commented Jul 14, 2026

Copy link
Copy Markdown

Why this change is needed:
The default 10m CPU request for ovn-controller causes CPU starvation during bulk User Defined Network (UDN) creation, resulting in "Unreasonably long poll interval" warnings (1000ms-4000ms+) and delayed network provisioning.

What this change does:
Increases the ovn-controller CPU request from 10m to 50m in the ovnkube-node DaemonSet to ensure sufficient CPU resources during heavy control-plane reconciliation operations.

Testing:

  • Manual testing: Created 50 UDNs across multiple namespaces on a 3-node cluster
  • Before: Multiple "Unreasonably long poll interval" warnings (1154ms-4401ms) with 0ms user time indicating CPU starvation
  • After: Zero warnings during the same bulk UDN creation test

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci
openshift-ci Bot requested review from kyrtapz and marty-power July 14, 2026 07:36
@shreyasbe shreyasbe changed the title Increase ovn-controller CPU request from 10m to 100m to prevent CPU s… Increase ovn-controller CPU request from 10m to 100m to prevent CPU starvation Jul 14, 2026
@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Walkthrough

The managed and self-hosted OVN Kubernetes node DaemonSet manifests increase the ovn-controller container CPU request from 10m to 50m.

Changes

OVN controller resources

Layer / File(s) Summary
Update OVN controller CPU request
bindata/network/ovn-kubernetes/managed/ovnkube-node.yaml, bindata/network/ovn-kubernetes/self-hosted/ovnkube-node.yaml
Changes the ovn-controller CPU request from 10m to 50m in both manifests.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Suggested reviewers: marty-power, bpickard22


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (3 errors)

Check name Status Explanation Resolution
Pr Quality ❌ Error Fail: the PR body's Testing is manual only (no CI lanes/platforms), no issue link is provided, and the description says 100m while the diff is 50m. Add CI-based verification details (jobs/platforms), include an issue link, and correct the CPU request description to match the 50m diff; add user-impact/rollback notes if needed.
Commit Message Quality ❌ Error Single commit is non-merge and concise, but it lacks the required component prefix and its body is only a Signed-off-by line, not a rationale. Amend the subject to include the scoped component (e.g. "ovn-kubernetes: ...") and add a body that explains why the CPU request increase is needed.
Unit Tests For Go Changes ❌ Error Two bindata ovnkube-node.yaml templates changed, and the PR diff shows no *_test.go files were added or modified. Add or update a *_test.go covering the rendered ovn-controller CPU request, or document why tests are unnecessary in 'How to verify it' and use @coderabbitai ignore pre-merge checks.
✅ Passed checks (21 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
E2e Tests For Feature Changes ✅ Passed No pkg/cmd Go files or test/e2e files changed; this is a manifest-only update, so the E2E-test requirement is not triggered.
Rbac Least Privilege ✅ Passed Only DaemonSet CPU request lines changed in the two ovnkube-node YAMLs; no Role/ClusterRole rules or RBAC sections were added or modified.
Docs For Feature And Behavior Changes ✅ Passed PASS: This bug-fix resource tuning change touches only two YAML manifests; no docs/ files were modified, and bug fixes are exempt.
Stale Project Docs And Config ✅ Passed No project docs/config mention the ovn-controller CPU request; docs only reference OVN_NODE_MODE and the same manifest paths, so nothing is stale.
Go And Test Code Quality ✅ Passed The PR only changes two YAML manifests; no Go or test files were modified, so none of the listed code-quality issues apply.
Ai-Generated Code Smell ✅ Passed Only two manifest CPU-request literals changed; no new comments, AI prompts, or verbose/auto-generated tests were introduced.
Stable And Deterministic Test Names ✅ Passed The PR only changes two YAML manifests and adds no Ginkgo tests, so there are no test titles to evaluate for stability.
Test Structure And Quality ✅ Passed PR only changes ovnkube-node YAML manifests; no Ginkgo test files were added or modified, so the test-quality criteria are not applicable.
Microshift Test Compatibility ✅ Passed PR only changes two ovnkube-node YAML manifests; no Go test files or Ginkgo test additions were introduced.
Single Node Openshift (Sno) Test Compatibility ✅ Passed No Ginkgo e2e tests were added; the patch only changes two ovnkube-node YAML manifests' CPU requests.
Topology-Aware Scheduling Compatibility ✅ Passed Only ovn-controller CPU request changed (10m→50m); no nodeSelector, anti-affinity, topology spread, or ControlPlaneTopology logic was added.
Ote Binary Stdout Contract ✅ Passed Only two YAML manifests changed; no main/init/suite code or stdout logging behavior was modified.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No new Ginkgo e2e tests were added; the PR only changes two ovnkube-node YAML manifests, so the IPv4/disconnected-network test check is not applicable.
No-Weak-Crypto ✅ Passed The only touched files are OVN YAML manifests changing ovn-controller CPU requests to 50m; no MD5/SHA1/DES/RC4/3DES/Blowfish, ECB, custom crypto, or secret comparisons appear.
Container-Privileges ✅ Passed Only cpu request changed from 10m to 50m in both manifests; no privilege-related fields were added or modified, and no allowPrivilegeEscalation/SYS_ADMIN changes appear.
No-Sensitive-Data-In-Logs ✅ Passed Only CPU request values changed in two YAML manifests; no logging, secrets, or sensitive data exposure was introduced.
Title check ✅ Passed The title is imperative, under the length limit, and accurately describes the CPU request increase.
Description check ✅ Passed The description directly explains the ovn-controller CPU request increase and its motivation and testing.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@tssurya

tssurya commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

@coderabbitai: why aren't you asking for "changes requested" via GUI?

@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

@tssurya CodeRabbit intentionally does not submit formal GitHub "Request Changes" review states by default. Here's why:

  • CodeRabbit posts its review as a set of comments (using GitHub's "Comment" review event) rather than "Approve" or "Request Changes," so it doesn't block merges via branch protection rules that require review approval/resolution.
  • This design choice avoids CodeRabbit becoming a hard gate on PRs — teams often want AI feedback to be advisory, with humans deciding whether findings are blocking.
  • The only exception is explicit approval: when a maintainer or PR author comments @coderabbitai approve, CodeRabbit will submit an "Approve" review via the GUI. There isn't a corresponding @coderabbitai request changes command — CodeRabbit is designed to leave the "Request Changes" decision to human reviewers.

If you'd like CodeRabbit to behave differently (e.g., more strongly flag certain issues, or you want stricter gating), that would need to be handled through your repository's branch protection rules or required status checks rather than CodeRabbit's review state, since CodeRabbit's core behavior here isn't currently configurable via YAML settings.

@tssurya

tssurya commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

@coderabbitai: I want it to be hard gating, how can I achieve that?

@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

@tssurya: Let me pull the current configuration schema to give you the precise setting name and behavior.

resources:
requests:
cpu: 10m
cpu: 100m

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@coderabbitai why did you miss

- When changing a file in one, ALWAYS check if the same change is needed in
the other. This is the #1 source of HyperShift drift bugs. Examples:
masquerade subnet config, interconnect flags, IPv6 URL formatting.
Some settings are HyperShift-only (SOCKS proxy, different cert paths,
`{{.OVNControlPlaneImage}}`), but most feature flags must appear in both.

@shreyasbe you are missing: https://github.com/openshift/cluster-network-operator/blob/master/bindata/network/ovn-kubernetes/managed/ovnkube-node.yaml

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
cat -n .coderabbit.yaml | sed -n '95,125p'
echo "---managed ovnkube-node.yaml ovn-controller resources---"
rg -n "ovn-controller" -A 40 bindata/network/ovn-kubernetes/managed/ovnkube-node.yaml | head -80

Length of output: 2084


🏁 Script executed:

#!/bin/bash
fd -i coderabbit.yaml
find / -maxdepth 2 -iname ".coderabbit.yaml" 2>/dev/null

Length of output: 229


🏁 Script executed:

#!/bin/bash
ls -la | grep -i coderabbit
git show HEAD:.coderabbit.yaml 2>&1 | sed -n '95,125p'

Length of output: 233


🏁 Script executed:

#!/bin/bash
pwd
ls -la
git log --oneline -3
git show e7f4de6204c26e2f8784fc4c7e2b508b08324fb5:.coderabbit.yaml | sed -n '95,125p'

Length of output: 4099


@kyrtapz You're right, and I apologize for the miss — the managed/self-hosted alignment check in .coderabbit.yaml (lines 109-113) should have flagged this. I confirmed the drift directly:

bindata/network/ovn-kubernetes/managed/ovnkube-node.yaml (HyperShift control-plane) still has:

      - name: ovn-controller
        ...
        resources:
          requests:
            cpu: 10m
            memory: 300Mi

while bindata/network/ovn-kubernetes/self-hosted/ovnkube-node.yaml was bumped to cpu: 100m in this PR. Since the same CPU-starvation issue (long poll intervals under bulk UDN creation) would equally affect the HyperShift-managed ovn-controller, the managed/ovnkube-node.yaml file should get the identical change for consistency and to avoid drift between the two topologies.


✏️ Learnings added
Learnt from: kyrtapz
Repo: openshift/cluster-network-operator PR: 3051
File: bindata/network/ovn-kubernetes/self-hosted/ovnkube-node.yaml:189-189
Timestamp: 2026-07-14T16:57:34.955Z
Learning: In openshift/cluster-network-operator, when a resource request/limit (e.g. CPU/memory) is changed for a container in `bindata/network/ovn-kubernetes/self-hosted/ovnkube-node.yaml` (e.g. the `ovn-controller` container), the parallel container in `bindata/network/ovn-kubernetes/managed/ovnkube-node.yaml` (HyperShift control-plane) must be checked and updated to match, per the managed/self-hosted alignment rule in `.coderabbit.yaml`. Missing this is the `#1` source of HyperShift drift bugs.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @kyrtapz for the pointer. I have addressed the code changes.

@shreyasbe
shreyasbe force-pushed the UDN_CORENET_7217_adjust_cpu_priority branch 2 times, most recently from 3388762 to bfa0556 Compare July 15, 2026 04:45
@shreyasbe shreyasbe changed the title Increase ovn-controller CPU request from 10m to 100m to prevent CPU starvation Bump ovn-controller CPU request to 100m to prevent CPU starvation Jul 15, 2026
@shreyasbe

Copy link
Copy Markdown
Author

/test 5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-upgrade

@shreyasbe

Copy link
Copy Markdown
Author

/test 5.0-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade

@shreyasbe

Copy link
Copy Markdown
Author

/retest-required

@kyrtapz

kyrtapz commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jul 16, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-ovn-upgrade-ipsec
/test e2e-gcp-ovn-techpreview

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aws-ovn-fdp-qe
/test e2e-aws-ovn-hypershift-conformance
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-aws-ovn-upgrade
/test e2e-aws-ovn-windows
/test e2e-azure-ovn-upgrade
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-dualstack-bgp
/test e2e-metal-ipi-ovn-dualstack-bgp-local-gw
/test e2e-metal-ipi-ovn-ipv6
/test e2e-metal-ipi-ovn-ipv6-ipsec
/test e2e-ovn-ipsec-step-registry
/test hypershift-e2e-aks

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 16, 2026
@shreyasbe

Copy link
Copy Markdown
Author

/retest-required

@shreyasbe

Copy link
Copy Markdown
Author

/retest

@shreyasbe

Copy link
Copy Markdown
Author

/test e2e-aws-ovn-upgrade-ipsec
/test e2e-gcp-ovn-techpreview
/test e2e-aws-ovn-fdp-qe
/test e2e-aws-ovn-hypershift-conformance
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-aws-ovn-upgrade
/test e2e-aws-ovn-windows
/test e2e-azure-ovn-upgrade
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-dualstack-bgp
/test e2e-metal-ipi-ovn-dualstack-bgp-local-gw
/test e2e-metal-ipi-ovn-ipv6
/test e2e-metal-ipi-ovn-ipv6-ipsec
/test e2e-ovn-ipsec-step-registry
/test hypershift-e2e-aks

@kyrtapz

kyrtapz commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

/retest

3 similar comments
@shreyasbe

Copy link
Copy Markdown
Author

/retest

@kyrtapz

kyrtapz commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

/retest

@shreyasbe

Copy link
Copy Markdown
Author

/retest

@kyrtapz

kyrtapz commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

/test e2e-aws-ovn-hypershift-conformance
/test e2e-gcp-ovn
/test hypershift-e2e-aks

Signed-off-by: Shreyas Be <52690686+shreyasbe@users.noreply.github.com>
@shreyasbe
shreyasbe force-pushed the UDN_CORENET_7217_adjust_cpu_priority branch from bfa0556 to 8e09af9 Compare July 22, 2026 01:01
@openshift-ci openshift-ci Bot removed the lgtm Indicates that a PR is ready to be merged. label Jul 22, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@bindata/network/ovn-kubernetes/managed/ovnkube-node.yaml`:
- Line 153: Add a regression assertion in the existing template test covering
the rendered ovn-controller CPU request of 50m for both managed and self-hosted
manifests, using the relevant template test symbols. Keep the test focused on
verifying this bindata resource request and document any exception only if one
applies.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 6eb9b2e7-695f-4ea8-99cc-2629773954a9

📥 Commits

Reviewing files that changed from the base of the PR and between bfa0556 and 8e09af9.

📒 Files selected for processing (2)
  • bindata/network/ovn-kubernetes/managed/ovnkube-node.yaml
  • bindata/network/ovn-kubernetes/self-hosted/ovnkube-node.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
  • bindata/network/ovn-kubernetes/self-hosted/ovnkube-node.yaml

Comment thread bindata/network/ovn-kubernetes/managed/ovnkube-node.yaml
@shreyasbe shreyasbe changed the title Bump ovn-controller CPU request to 100m to prevent CPU starvation Bump ovn-controller CPU request to 50m to prevent CPU starvation Jul 22, 2026
@kyrtapz

kyrtapz commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jul 22, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-ovn-upgrade-ipsec
/test e2e-gcp-ovn-techpreview

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aws-ovn-fdp-qe
/test e2e-aws-ovn-hypershift-conformance
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-aws-ovn-upgrade
/test e2e-aws-ovn-windows
/test e2e-azure-ovn-upgrade
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-dualstack-bgp
/test e2e-metal-ipi-ovn-dualstack-bgp-local-gw
/test e2e-metal-ipi-ovn-ipv6
/test e2e-metal-ipi-ovn-ipv6-ipsec
/test e2e-ovn-ipsec-step-registry
/test hypershift-e2e-aks

@openshift-ci

openshift-ci Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kyrtapz, shreyasbe

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kyrtapz

kyrtapz commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

/retest-required
@shreyasbe please analyze the CI failures and link the appropriate bugs(if they exists).
It looks like we are going to need to override some jobs but we need to know the failures are not related to this change and have bugs tracking them.

@shreyasbe

Copy link
Copy Markdown
Author

/retest

@shreyasbe

Copy link
Copy Markdown
Author

/test e2e-metal-ipi-ovn-ipv6-ipsec
/test e2e-aws-ovn-fdp-qe
/test 5.0-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade
/test e2e-aws-ovn-upgrade-ipsec
/test 5.0-upgrade-from-stable-4.22-e2e-azure-ovn-upgrade
/test e2e-ovn-ipsec-step-registry
/test e2e-aws-ovn-upgrade

@shreyasbe

shreyasbe commented Jul 23, 2026

Copy link
Copy Markdown
Author

/retest-required @shreyasbe please analyze the CI failures and link the appropriate bugs(if they exists). It looks like we are going to need to override some jobs but we need to know the failures are not related to this change and have bugs tracking them.

Hi @kyrtapz, I've analyzed the failing jobs, Which are not related to this CPU request change. Similar failure also seen other PRs as well.

I am rerunning the failed job again: #3051 (comment)

@shreyasbe

Copy link
Copy Markdown
Author

/test e2e-metal-ipi-ovn-ipv6-ipsec
/test e2e-aws-ovn-fdp-qe
/test 5.0-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade
/test e2e-aws-ovn-upgrade-ipsec
/test 5.0-upgrade-from-stable-4.22-e2e-azure-ovn-upgrade
/test e2e-ovn-ipsec-step-registry

1 similar comment
@shreyasbe

Copy link
Copy Markdown
Author

/test e2e-metal-ipi-ovn-ipv6-ipsec
/test e2e-aws-ovn-fdp-qe
/test 5.0-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade
/test e2e-aws-ovn-upgrade-ipsec
/test 5.0-upgrade-from-stable-4.22-e2e-azure-ovn-upgrade
/test e2e-ovn-ipsec-step-registry

@shreyasbe

Copy link
Copy Markdown
Author

/test e2e-metal-ipi-ovn-ipv6-ipsec
/test e2e-aws-ovn-fdp-qe
/test e2e-aws-ovn-upgrade-ipsec
/test e2e-ovn-ipsec-step-registry

1 similar comment
@shreyasbe

Copy link
Copy Markdown
Author

/test e2e-metal-ipi-ovn-ipv6-ipsec
/test e2e-aws-ovn-fdp-qe
/test e2e-aws-ovn-upgrade-ipsec
/test e2e-ovn-ipsec-step-registry

@openshift-ci

openshift-ci Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

@shreyasbe: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-upgrade-ipsec 8e09af9 link true /test e2e-aws-ovn-upgrade-ipsec
ci/prow/e2e-metal-ipi-ovn-ipv6-ipsec 8e09af9 link true /test e2e-metal-ipi-ovn-ipv6-ipsec
ci/prow/e2e-ovn-ipsec-step-registry 8e09af9 link true /test e2e-ovn-ipsec-step-registry
ci/prow/e2e-aws-ovn-fdp-qe 8e09af9 link true /test e2e-aws-ovn-fdp-qe

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants