Skip to content

Add ContentView resource with cross-domain scatter/gather utility - #7958

Draft
YasenT wants to merge 3 commits into
pulp:mainfrom
YasenT:content-view-resource
Draft

Add ContentView resource with cross-domain scatter/gather utility#7958
YasenT wants to merge 3 commits into
pulp:mainfrom
YasenT:content-view-resource

Conversation

@YasenT

@YasenT YasenT commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Add ContentView resource with cross-domain scatter/gather utility

Problem

Content Sources (via tang) performs batch searches across repository versions spanning multiple domains by opening a raw pgx connection straight to Pulp's Postgres. This has three problems:

  • It breaks under domain-DB-offloading — a single connection pool can't query tables that live on different physical RDS instances.
  • It bypasses all Pulp RBAC, since queries never pass through the API layer.
  • Callers must pass raw lists of repository version hrefs on every request, with no way to save, name, or share a search scope.
    Content Sources templates routinely mix repositories from a private org domain and shared/public domains, so cross-domain search is a first-class requirement, not an edge case.

What this adds

A new first-class Pulp resource, ContentView — a named, persistable object that composes distributions from multiple domains into a single searchable scope, replacing the raw-hrefs-per-request approach with a proper Pulp resource.

  • Model (ContentView): domain-scoped like any other Pulp resource, but its distributions ManyToManyField can reference Distributions from any domain the user has read access to. Each distribution already carries version-tracking semantics (tracks latest repo, pinned version, or a publication), so the content view doesn't need to duplicate that.
  • Full CRUD + RBAC: ContentViewViewSet extends NamedModelViewSet/RolesMixin/LabelsMixin with a standard DEFAULT_ACCESS_POLICY/LOCKED_ROLES, following existing pulpcore conventions. Any user in the same org can read/update/delete content views in domains they have access to.
  • Cross-domain scatter/gather utility (pulpcore.app.util_content_view, exported via pulpcore.plugin.util): resolve_content_view_distributions, group_versions_by_domain, and scatter_gather let plugins resolve a content view's distributions to their current repository versions, group them by domain, and run per-domain queries that are merged in Python — the single-domain case collapses to one query, multi-domain does a bounded per-domain over-fetch.
  • RBAC-aware resolution: user_can_view_domain respects domain-level core.view_domain permission checks during resolution, so a distribution the user has lost access to is silently excluded from search results rather than erroring; the content view detail endpoint's distributions_status field reports per-distribution status (ok, no_domain_access, deleted, etc.) so callers can see what's excluded and why.
  • Nested search endpoints for plugins: router_lookup = "content_view" on ContentViewViewSet lets plugins (e.g. pulp_rpm) register read-only search viewsets nested under /content-views/{uuid}/search/... via rest_framework_nested.routers, reusing an existing content type's queryset without competing for that model's canonical viewset.
  • Bugfix: get_viewset_for_model previously raised LookupError when a model had more than one registered viewset (its canonical viewset plus a plugin's nested, read-only search viewset reusing the same queryset) — a real bug hit during RPM sync once a plugin registers a nested search viewset. Fixed by disambiguating: if excluding nested viewsets (those with a parent_viewset) leaves exactly one candidate, that's the canonical viewset.

Out of scope (this PR)

  • RPM-specific search endpoints (search/rpm/*) — implemented separately in pulp_rpm on top of this.
  • Maven/NPM/Python content search.
  • UI implementation.

Testing

  • Unit tests for the model, cross-domain resolution logic, and scatter_gather (single- and multi-domain paths), including RBAC exclusion scenarios.
  • Unit tests for the get_viewset_for_model disambiguation fix, covering both the resolvable and genuinely-ambiguous cases.
  • Manually validated end-to-end (CRUD, cross-domain search, RBAC exclusion) against a live Postgres-backed instance alongside the companion pulp_rpm search endpoints.

YasenT and others added 3 commits August 11, 2026 15:59
Introduces a new first-class Pulp resource, ContentView: a named,
domain-scoped object composed of Distributions that may span multiple
domains, with full CRUD and RBAC. Plugins build cross-domain search
endpoints on top of it using the new resolve_content_view_distributions/
group_versions_by_domain/scatter_gather utilities, exposed via
pulpcore.plugin.*, without querying the database directly or bypassing
RBAC.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
A plugin may register an additional read-only, nested viewset that reuses
an existing content type's queryset for its own purposes (e.g. the RPM
ContentView search endpoints reusing Package/UpdateRecord/etc.) without
intending to compete for that model's canonical viewset. Since such
viewsets are always nested (they declare parent_viewset), exclude them
from the ambiguity check when exactly one non-nested candidate remains.

Without this fix, registering a second viewset against an existing
content model made get_viewset_for_model raise LookupError for that
model unconditionally, breaking RepositoryVersion content_summary hrefs
and master-viewset queryset scoping for every affected content type.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant