fix(inplaceupdate): preserve injected resources during resize#537
fix(inplaceupdate): preserve injected resources during resize#537silver-chard wants to merge 6 commits into
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Welcome @silver-chard! It looks like this is your first PR to openkruise/agents 🎉 |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #537 +/- ##
=======================================
Coverage 80.04% 80.05%
=======================================
Files 229 229
Lines 17861 17841 -20
=======================================
- Hits 14297 14282 -15
+ Misses 2979 2974 -5
Partials 585 585
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| resizeBody.Spec.Containers[i] = corev1.Container{ | ||
| Name: pod.Spec.Containers[i].Name, | ||
| Resources: pod.Spec.Containers[i].Resources, | ||
| Resources: *pod.Spec.Containers[i].Resources.DeepCopy(), |
There was a problem hiding this comment.
Thanks @silver-chard for bringing this up. This is indeed a problem but I'm wondering if we could use Patch at:
https://github.com/openkruise/agents/blob/master/pkg/utils/inplaceupdate/inplace_update.go#L505
to solve this at the root.
The underlying issue: SubResource("resize").Update() is a PUT, so the body must include every resource key present on the pod — including ones injected by admission webhooks (e.g., nvidia.com/gpu, ephemeral-storage) or set outside the template (e.g., resizePolicy). That's what forces the deep-copy + merge dance. A Patch would avoid all of this since unmentioned keys are preserved automatically.
There was a problem hiding this comment.
Thanks for the suggestion. Updated the resize path to use a strategic merge patch on the resize subresource instead of Update.
The resize body now only contains desired resource changes for normal containers. Injected resource keys and ephemeral-storage are no longer copied into the request body; they are preserved by patch semantics instead.
I also updated the tests to cover the subresource patch path and verify injected resources are preserved.
Tested with:
go test -count=1 ./pkg/utils/inplaceupdate
6f366d8 to
a10a15c
Compare
| @@ -256,37 +257,24 @@ func DefaultGenerateResizeSubresourceBody(opts InPlaceUpdateOptions) *corev1.Pod | |||
| ResourceVersion: pod.ResourceVersion, | |||
There was a problem hiding this comment.
the ObjectMeta fields in DefaultGenerateResizeSubresourceBody are unused now that we PATCH — buildResourcePatch only reads Spec.Containers. A few small cleanups this unlocks:
- Drop the ObjectMeta allocation entirely.
- Return []corev1.Container instead of *corev1.Pod — the Pod wrapper is PUT-era baggage.
- Rename to something like DefaultBuildResizeContainers to match what it actually does ("SubresourceBody" implies a full PUT payload).
- Change buildResourcePatch's param from (*corev1.Pod) to ([]corev1.Container) as well.
There was a problem hiding this comment.
Sorry for the silence, and thanks for the review. I updated the patch based on the comments:
-
DefaultGeneratePatchBodyFunc now uses ResourcesEqual instead of reflect.DeepEqual, so injected resources such as ephemeral-storage and semantically equivalent quantities like 1000m vs 1 no longer cause a false UpdateResources state.
-
The resize helper now only builds resize containers from regular containers. initContainers are intentionally not included in the resize payload, and the comment has been clarified.
-
Added unit coverage for:
- extra injected resources should not trigger UpdateResources
- equivalent Quantity representations should not trigger UpdateResources
- image-only updates with injected resources should still only set UpdateImages
-
Strengthened the resize-container minimal-field test for the initContainer case.
-
Kept the e2e test portable by covering initContainer preservation during claim resize. Webhook-injected resource behavior is covered by unit tests because it is hard to force consistently across e2e clusters.
Validation:
- go test -count=1 ./pkg/utils/inplaceupdate
- go test -count=1 ./pkg/controller/sandbox/core
- go test -c ./test/e2e -o /private/tmp/openkruise-agents-e2e.test
- go vet ./pkg/utils/inplaceupdate
- git diff --check HEAD~1..HEAD
|
@silver-chard, Could you add a couple of SandboxClaim e2e scenarios that exercise the resize path? Specifically:
|
|
@silver-chard Hi, are you still work on this? |
sorry for the silence. I’m still working on it. |
Signed-off-by: xiezhida <xie.zhida@icloud.com>
8ab487c to
58c08b4
Compare
Use the same semantic resource comparison when generating the in-place update patch and resize containers. This avoids marking UpdateResources when the live pod only has system-injected resources or equivalent Quantity values such as 1000m and 1. Also keep resize payloads limited to regular containers, preserve init containers, and add unit/e2e coverage for these cases. Signed-off-by: xiezhida <xie.zhida@icloud.com>
58c08b4 to
91da268
Compare
| Revision: revision, | ||
| UpdateTimestamp: metav1.Now(), | ||
| LastContainerStatuses: map[string]InPlaceUpdateContainerStatus{}, | ||
| UpdateResources: opts.ResourceUpdateRequired, |
There was a problem hiding this comment.
@silver-chard there are now two places in DefaultGeneratePatchBodyFunc that set state.UpdateResources:
- At initialization via
opts.ResourceUpdateRequired - Inside the container loop via
if resourceChanged { state.UpdateResources = true }
While both only ever set the flag to true (no conflicting false writes), the intent is ambiguous — which code path owns the decision?
Suggestion: may be can drop the if resourceChanged { state.UpdateResources = true } block from the loop and rely solely on opts.ResourceUpdateRequired. This keeps a single source of truth and avoids confusion about which path is authoritative.
|
PTAL @furykerry @zmberg |
Signed-off-by: xiezhida <xie.zhida@icloud.com>
7ae251f to
a94d722
Compare
a94d722 to
cc7272d
Compare
Ⅰ. Describe what this PR does
This PR preserves live Pod resource entries that are not managed by the Sandbox
template when generating a resize subresource body.
Previously, the resize body could omit request/limit entries that exist on the
live Pod. If an admission plugin or runtime integration injects an extra
resource, the resize request may remove that key and be rejected by the API
server.
This change:
Ⅱ. Does this pull request fix one issue?
NONE
Ⅲ. Describe how to verify it
go test -count=1 ./pkg/utils/inplaceupdateⅣ. Special notes for reviews
No API, CRD, generated client, proto, or manifest changes.