Skip to content

feat(cms): add limited seed import to gateway-sync pipeline#516

Open
lumberman wants to merge 4 commits intomainfrom
dreamy-cannon
Open

feat(cms): add limited seed import to gateway-sync pipeline#516
lumberman wants to merge 4 commits intomainfrom
dreamy-cannon

Conversation

@lumberman
Copy link
Collaborator

@lumberman lumberman commented Mar 23, 2026

Summary

  • Extends POST /gateway-sync/trigger to accept optional collectionIds, videoIds, and dryRun parameters for staging/local seed imports
  • Collection IDs (coverage-style top-level video IDs) are automatically resolved to their child video IDs via a new resolve-collection-video-ids.ts transformer
  • Limited imports are always additive — no soft-delete of unseen content
  • Dry-run mode returns the resolved ID mapping without writing CMS data
  • Gated behind GATEWAY_SYNC_ENABLE_LIMITED_IMPORT=true env var

Closes #514

Key Design Decisions

  • Behavior inferred, not declared: empty selection inputs run existing full sync; non-empty inputs run limited seed import
  • No separate variant crawl for limited imports: variants are fetched nested inside the selected-video query, avoiding a global videoVariants pagination
  • Variant data passed between phases via _collectedVariants on the selection object, so the video phase fetches once and the variant phase consumes
  • ID cap: maximum 500 total collectionIds + videoIds per request to prevent near-full imports

Testing

  • TypeScript compilation passes (pnpm --filter cms typecheck)
  • Prettier formatting passes (lint-staged)
  • Manual testing against gateway API needed for full validation

Post-Deploy Monitoring & Validation

  • What to monitor/search
    • Logs: [gateway-sync] Limited import — confirms limited mode activation
    • Logs: [gateway-sync] Missing collection IDs — flags bad operator input
  • Expected healthy behavior
    • Dry-run returns collectionVideoIds mapping with resolved child IDs
    • Limited import creates/updates only selected videos + variants
    • Full sync behavior unchanged when no selection inputs provided
  • Failure signal(s) / rollback trigger
    • Any soft-delete activity during limited imports = bug (should never happen)
    • GATEWAY_SYNC_ENABLE_LIMITED_IMPORT not set = limited imports rejected with clear error
  • Validation window & owner
    • Window: first staging seed import attempt
    • Owner: operator running the import

Compound Engineering v2.50.0
🤖 Generated with Claude Opus 4.6 (1M context, extended thinking) via Claude Code

lumberman and others added 2 commits March 23, 2026 17:36
@lumberman
docs(cms): add staging collection seed import plan
cb4e6a2
@lumberman @claude
feat(cms): add limited seed import to gateway-sync pipeline (#514)
5830838 
Extend the gateway-sync trigger to accept optional collectionIds,
videoIds, and dryRun parameters for staging/local seed imports.

Collection IDs (coverage-style top-level video IDs) are resolved to
their child video IDs automatically. Limited imports are additive —
no soft-delete of unseen content. A dry-run mode returns the resolved
mapping without writing data.

Gated behind GATEWAY_SYNC_ENABLE_LIMITED_IMPORT=true env var to
prevent accidental production use.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@railway-app railway-app bot temporarily deployed to forge / forge-pr-516 March 23, 2026 21:06 Destroyed
@railway-app
Copy link

railway-app bot commented Mar 23, 2026 •
edited
Loading

🚅 Deployed to the forge-pr-516 environment in forge

Service Status Web Updated (UTC)
@forge/cms ✅ Success (View Logs) Web Mar 24, 2026 at 2:47 pm
2 services not affected by this PR
  • @forge/web
  • @forge/manager

@lumberman @claude
fix(cms): resolve bugs found during limited import local testing
1bad213 
- fix: null-guard Apollo data in sync-keywords to handle gateway errors gracefully
- fix: batch SELECTED_VIDEOS_QUERY in 10-ID chunks to avoid gateway complexity limit
- fix: include parent collection video in resolvedVideoIds alongside children
- docs: add local testing runbook and bug report with verified fixes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@lumberman lumberman

Labels

cms feat

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

feat(cms): add staging collection seed import

1 participant

@lumberman