Skip to content

fix(inplaceupdate): preserve injected resources during resize#537

Open
silver-chard wants to merge 6 commits into
openkruise:masterfrom
silver-chard:fix/preserve-injected-resources-on-resize
Open

fix(inplaceupdate): preserve injected resources during resize#537
silver-chard wants to merge 6 commits into
openkruise:masterfrom
silver-chard:fix/preserve-injected-resources-on-resize

Conversation

@silver-chard

Copy link
Copy Markdown

Ⅰ. Describe what this PR does

This PR preserves live Pod resource entries that are not managed by the Sandbox
template when generating a resize subresource body.

Previously, the resize body could omit request/limit entries that exist on the
live Pod. If an admission plugin or runtime integration injects an extra
resource, the resize request may remove that key and be rejected by the API
server.

This change:

  • builds resize resources from a deep copy of the live Pod resources;
  • merges desired template resources into the live resources;
  • preserves injected request/limit entries;
  • adds regression coverage for injected resources.

Ⅱ. Does this pull request fix one issue?

NONE

Ⅲ. Describe how to verify it

go test -count=1 ./pkg/utils/inplaceupdate

Ⅳ. Special notes for reviews

No API, CRD, generated client, proto, or manifest changes.

@kruise-bot
kruise-bot requested review from furykerry and zmberg June 13, 2026 13:39
@kruise-bot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign furykerry for approval by writing /assign @furykerry in a comment. For more information see:The Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kruise-bot

Copy link
Copy Markdown

Welcome @silver-chard! It looks like this is your first PR to openkruise/agents 🎉

@codecov

codecov Bot commented Jun 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.05%. Comparing base (c1c04fb) to head (cc7272d).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #537   +/-   ##
=======================================
  Coverage   80.04%   80.05%           
=======================================
  Files         229      229           
  Lines       17861    17841   -20     
=======================================
- Hits        14297    14282   -15     
+ Misses       2979     2974    -5     
  Partials      585      585           
Flag Coverage Δ
unittests 80.05% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

resizeBody.Spec.Containers[i] = corev1.Container{
Name: pod.Spec.Containers[i].Name,
Resources: pod.Spec.Containers[i].Resources,
Resources: *pod.Spec.Containers[i].Resources.DeepCopy(),

@PersistentJZH PersistentJZH Jun 14, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @silver-chard for bringing this up. This is indeed a problem but I'm wondering if we could use Patch at:

https://github.com/openkruise/agents/blob/master/pkg/utils/inplaceupdate/inplace_update.go#L505

to solve this at the root.

The underlying issue: SubResource("resize").Update() is a PUT, so the body must include every resource key present on the pod — including ones injected by admission webhooks (e.g., nvidia.com/gpu, ephemeral-storage) or set outside the template (e.g., resizePolicy). That's what forces the deep-copy + merge dance. A Patch would avoid all of this since unmentioned keys are preserved automatically.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion. Updated the resize path to use a strategic merge patch on the resize subresource instead of Update.

The resize body now only contains desired resource changes for normal containers. Injected resource keys and ephemeral-storage are no longer copied into the request body; they are preserved by patch semantics instead.

I also updated the tests to cover the subresource patch path and verify injected resources are preserved.

Tested with:

go test -count=1 ./pkg/utils/inplaceupdate

@silver-chard
silver-chard force-pushed the fix/preserve-injected-resources-on-resize branch from 6f366d8 to a10a15c Compare June 25, 2026 14:35
@kruise-bot kruise-bot added size/L and removed size/M labels Jun 25, 2026
@@ -256,37 +257,24 @@ func DefaultGenerateResizeSubresourceBody(opts InPlaceUpdateOptions) *corev1.Pod
ResourceVersion: pod.ResourceVersion,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the ObjectMeta fields in DefaultGenerateResizeSubresourceBody are unused now that we PATCH — buildResourcePatch only reads Spec.Containers. A few small cleanups this unlocks:

  • Drop the ObjectMeta allocation entirely.
  • Return []corev1.Container instead of *corev1.Pod — the Pod wrapper is PUT-era baggage.
  • Rename to something like DefaultBuildResizeContainers to match what it actually does ("SubresourceBody" implies a full PUT payload).
  • Change buildResourcePatch's param from (*corev1.Pod) to ([]corev1.Container) as well.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the silence, and thanks for the review. I updated the patch based on the comments:

  1. DefaultGeneratePatchBodyFunc now uses ResourcesEqual instead of reflect.DeepEqual, so injected resources such as ephemeral-storage and semantically equivalent quantities like 1000m vs 1 no longer cause a false UpdateResources state.

  2. The resize helper now only builds resize containers from regular containers. initContainers are intentionally not included in the resize payload, and the comment has been clarified.

  3. Added unit coverage for:

    • extra injected resources should not trigger UpdateResources
    • equivalent Quantity representations should not trigger UpdateResources
    • image-only updates with injected resources should still only set UpdateImages
  4. Strengthened the resize-container minimal-field test for the initContainer case.

  5. Kept the e2e test portable by covering initContainer preservation during claim resize. Webhook-injected resource behavior is covered by unit tests because it is hard to force consistently across e2e clusters.

Validation:

  • go test -count=1 ./pkg/utils/inplaceupdate
  • go test -count=1 ./pkg/controller/sandbox/core
  • go test -c ./test/e2e -o /private/tmp/openkruise-agents-e2e.test
  • go vet ./pkg/utils/inplaceupdate
  • git diff --check HEAD~1..HEAD

@PersistentJZH

Copy link
Copy Markdown
Member

@silver-chard, Could you add a couple of SandboxClaim e2e scenarios that exercise the resize path? Specifically:

  • A claim that triggers a CPU/memory resize with system-injected resources (e.g., ephemeral-storage) or init containers present on the pod, to verify they survive the resize on both the subresource and fallback code paths.
  • The unit test coverage is solid, but an e2e smoke test would give us confidence that the PATCH-on-subresource path actually works against a real API server.

@PersistentJZH

Copy link
Copy Markdown
Member

@silver-chard Hi, are you still work on this?

@silver-chard

Copy link
Copy Markdown
Author

@silver-chard Hi, are you still work on this?

sorry for the silence. I’m still working on it.
I should have it done later today.

Signed-off-by: xiezhida <xie.zhida@icloud.com>
@silver-chard
silver-chard force-pushed the fix/preserve-injected-resources-on-resize branch from 8ab487c to 58c08b4 Compare July 7, 2026 08:58
Use the same semantic resource comparison when generating the
in-place update patch and resize containers. This avoids marking
UpdateResources when the live pod only has system-injected resources
or equivalent Quantity values such as 1000m and 1.

Also keep resize payloads limited to regular containers, preserve init
containers, and add unit/e2e coverage for these cases.

Signed-off-by: xiezhida <xie.zhida@icloud.com>
@silver-chard
silver-chard force-pushed the fix/preserve-injected-resources-on-resize branch from 58c08b4 to 91da268 Compare July 7, 2026 10:02
Revision: revision,
UpdateTimestamp: metav1.Now(),
LastContainerStatuses: map[string]InPlaceUpdateContainerStatus{},
UpdateResources: opts.ResourceUpdateRequired,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@silver-chard there are now two places in DefaultGeneratePatchBodyFunc that set state.UpdateResources:

  1. At initialization via opts.ResourceUpdateRequired
  2. Inside the container loop via if resourceChanged { state.UpdateResources = true }

While both only ever set the flag to true (no conflicting false writes), the intent is ambiguous — which code path owns the decision?

Suggestion: may be can drop the if resourceChanged { state.UpdateResources = true } block from the loop and rely solely on opts.ResourceUpdateRequired. This keeps a single source of truth and avoids confusion about which path is authoritative.

@PersistentJZH

Copy link
Copy Markdown
Member

PTAL @furykerry @zmberg

Signed-off-by: xiezhida <xie.zhida@icloud.com>
@silver-chard
silver-chard force-pushed the fix/preserve-injected-resources-on-resize branch from 7ae251f to a94d722 Compare July 14, 2026 14:11
@silver-chard
silver-chard force-pushed the fix/preserve-injected-resources-on-resize branch from a94d722 to cc7272d Compare July 15, 2026 10:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants