Skip to content

[PR #7896/d57f7275 backport][3.105] Narrow distribution locks with safe base_path protection - #7955

Merged
mdellweg merged 1 commit into
3.105from
patchback/backports/3.105/d57f7275e4fb599f5488fe207fb1c8ae4472f4f5/pr-7896
Aug 11, 2026
Merged

[PR #7896/d57f7275 backport][3.105] Narrow distribution locks with safe base_path protection#7955
mdellweg merged 1 commit into
3.105from
patchback/backports/3.105/d57f7275e4fb599f5488fe207fb1c8ae4472f4f5/pr-7896

Conversation

@patchback

@patchback patchback Bot commented Aug 11, 2026

Copy link
Copy Markdown

This is a backport of PR #7896 as merged into main (d57f727).

Problem

Async distribution CUD operations currently reserve the domain-wide
pdrn:<domain>:distributions resource too broadly. For ordinary updates
that leave base_path unchanged, this serializes unrelated
pulpcore.app.tasks.base.ageneral_update tasks behind a lock they do not
need.

This showed up as a bottleneck during capsule sync, where many
RefreshDistribution updates can run at once but end up waiting on the
same reservation.

What this changes

This patch narrows distribution task reservations while keeping the
base_path namespace protected in the safer way discussed in review:

  • ordinary updates that leave base_path unchanged reserve only the
    distribution instance
  • create and update operations that create or change a base_path also
    reserve a dedicated domain-scoped distribution.base_path resource
  • delete does not take the new distribution.base_path lock
  • the legacy domain:distributions reservation is still added in shared
    mode for mixed-version upgrade compatibility

The implementation also handles partial PATCH requests correctly by
falling back to instance.base_path when base_path is omitted from the
request body.

As part of the final cleanup, the shared_resources plumbing now lives in
the generic async mixins in base.py, while publication.py only defines
the distribution-specific lock policy.

Test coverage

Adds a functional test covering the reservation behavior for:

  • create
  • partial update with no base_path in the payload
  • partial update with unchanged base_path
  • partial update with changed base_path
  • delete

Performance notes

Tested with Satellite 6.20 Stream,
~175 capsules, 15 Pulp workers.

For completed pulpcore.app.tasks.base.ageneral_update tasks, lock usage
shifted from all domain-wide reservations to a mix of domain and
instance-scoped reservations:

Lock type Before patch After patch
Domain-wide 110 26
Instance-only 0 42

Block wait times improved substantially:

Metric Before (domain-locked) After domain-locked After instance-locked
P95 6.98s 75ms 28ms
P99 10.12s 105ms 29ms
Max 10.16s 121ms 29ms

These results are consistent with removing unnecessary serialization for
ordinary distribution updates while preserving explicit protection when the
base_path namespace can change.

mdellweg
mdellweg previously approved these changes Aug 11, 2026
@mdellweg

Copy link
Copy Markdown
Member

The lb test is failing because pytest-django 4.14.0 is installed, while 4.13.0 dropped support for Django 4.2.

How this incompatible combination of versions could ever have been installed is beyond my comprehension.

@mdellweg
mdellweg force-pushed the patchback/backports/3.105/d57f7275e4fb599f5488fe207fb1c8ae4472f4f5/pr-7896 branch from a52bc75 to 54d8de5 Compare August 11, 2026 12:22
Reserve per-distribution locks for ordinary updates while serializing creates and base_path changes on a dedicated domain-scoped distribution.base_path resource. Keep the legacy domain:distributions reservation in shared mode for mixed-version upgrade safety, and move the shared-resource plumbing into the generic async mixins so publication.py only defines the distribution-specific lock policy.

Co-authored-by: Cursor <cursoragent@cursor.com>
(cherry picked from commit d57f727)
@mdellweg
mdellweg force-pushed the patchback/backports/3.105/d57f7275e4fb599f5488fe207fb1c8ae4472f4f5/pr-7896 branch from 54d8de5 to 47f4a43 Compare August 11, 2026 12:32
@mdellweg
mdellweg merged commit 5f3c586 into 3.105 Aug 11, 2026
14 checks passed
@mdellweg
mdellweg deleted the patchback/backports/3.105/d57f7275e4fb599f5488fe207fb1c8ae4472f4f5/pr-7896 branch August 11, 2026 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants