Skip to content

feat: move to continue as new pattern (CM-1277)#4262

Merged
ulemons merged 7 commits into
mainfrom
feat/maven-fetcher-continue-as-new
Jun 25, 2026
Merged

feat: move to continue as new pattern (CM-1277)#4262
ulemons merged 7 commits into
mainfrom
feat/maven-fetcher-continue-as-new

Conversation

@ulemons

@ulemons ulemons commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Summary

Replaces the per-minute Maven schedule (launching a new Temporal workflow every minute) with a single long-running ingestMavenPackages workflow that uses continueAsNew — the same pattern already in use by the npm, go, and other workers. The workflow processes one batch of 2000 packages per cycle, calls continueAsNew as long as there is work, and exits cleanly when the queue is drained. A daily Temporal schedule restarts it the next day.

Changes

  • maven/workflows.ts — replaces mavenCriticalWorkflow / mavenNonCriticalWorkflow (single-shot, no retry) with ingestMavenPackages: processes one batch per cycle, calls continueAsNew if work remains, returns when the queue is empty
  • maven/schedule.ts — schedule ID maven-critical (every minute, 15 min timeout) → maven-ingestion (daily at midnight, workflowRunTimeout: 24h); startup now deletes the legacy maven-critical schedule to prevent ghost runs after deploy
  • maven/activities.ts — removes dead processMavenNonCriticalBatch (no callers since non-critical schedule was never activated)
  • activities.ts — removes processMavenNonCriticalBatch from the activities index export
  • bin/maven-worker.ts, workflows/index.ts — updated to reference the new names

Type of change

  • Bug fix
  • New feature
  • Refactor / cleanup
  • Performance improvement
  • Chore / dependency update
  • Documentation

JIRA ticket

CM-1277


Note

Medium Risk
Changes Maven ingestion cadence and Temporal workflow lifecycle (24h runs vs 1-minute launches), which can affect how quickly the universe is polled until the first daily run completes; deploy-time legacy schedule deletion is intentional but environment-specific if schedules were customized.

Overview
Maven package enrichment moves from minute-by-minute one-shot workflows to a single long-running ingestMavenPackages workflow that processes one batch per cycle, continueAsNews while work remains, and exits when the queue is empty—aligned with npm/go workers.

Scheduling: Temporal schedule maven-critical (every minute, 15m execution timeout) is replaced by maven-ingestion (daily 0 9 * * *, fixed workflow id maven-daily-enrichment, 24h run timeout). Worker startup deletes the legacy maven-critical schedule to avoid duplicate runs after deploy.

Cleanup: processMavenNonCriticalBatch and mavenNonCriticalWorkflow are removed (non-critical path was never scheduled). Activity proxy gains retry (max 5) on the critical batch path.

Reviewed by Cursor Bugbot for commit 837668d. Bugbot is set up for automated code reviews on this repo. Configure here.

@ulemons ulemons self-assigned this Jun 25, 2026
Copilot AI review requested due to automatic review settings June 25, 2026 07:41
@ulemons ulemons added the Feature Created by Linear-GitHub Sync label Jun 25, 2026
@ulemons ulemons changed the title feat: move to continue as new pattern feat: move to continue as new pattern (CM-1277) Jun 25, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the packages_worker Maven ingestion to follow the established long-running Temporal workflow pattern (used by npm/go): a single ingestMavenPackages workflow drains work in bounded cycles and uses continueAsNew instead of launching a new workflow every minute.

Changes:

  • Replaces the one-shot Maven workflows with ingestMavenPackages, which runs up to 5 batches per cycle and continueAsNews while there’s work.
  • Replaces the per-minute Temporal schedule with a daily maven-ingestion schedule and adds startup cleanup to delete the legacy maven-critical schedule.
  • Removes the unused non-critical batch activity and related exports/registrations.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
services/apps/packages_worker/src/workflows/index.ts Updates workflow exports to register ingestMavenPackages instead of legacy Maven workflows.
services/apps/packages_worker/src/maven/workflows.ts Implements bounded loop + continueAsNew Maven ingestion workflow.
services/apps/packages_worker/src/maven/schedule.ts Switches to daily schedule and attempts to delete the legacy schedule at startup.
services/apps/packages_worker/src/maven/activities.ts Removes the unused non-critical activity implementation.
services/apps/packages_worker/src/bin/maven-worker.ts Updates worker startup to register the new Maven schedule function.
services/apps/packages_worker/src/activities.ts Removes non-critical Maven activity export from the activity index.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread services/apps/packages_worker/src/maven/schedule.ts
Comment thread services/apps/packages_worker/src/maven/workflows.ts Outdated
Comment thread services/apps/packages_worker/src/maven/schedule.ts
Comment thread services/apps/packages_worker/src/maven/workflows.ts
Copilot AI review requested due to automatic review settings June 25, 2026 08:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.

Comment thread services/apps/packages_worker/src/maven/workflows.ts

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 0caf5d3. Configure here.

Comment thread services/apps/packages_worker/src/maven/workflows.ts
ulemons added 4 commits June 25, 2026 10:18
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Copilot AI review requested due to automatic review settings June 25, 2026 08:18
@ulemons ulemons force-pushed the feat/maven-fetcher-continue-as-new branch from 42bf5db to b6c85c9 Compare June 25, 2026 08:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Comment thread services/apps/packages_worker/src/maven/workflows.ts
Comment thread services/apps/packages_worker/src/maven/schedule.ts
ulemons added 2 commits June 25, 2026 10:25
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Copilot AI review requested due to automatic review settings June 25, 2026 08:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Comment thread services/apps/packages_worker/src/maven/workflows.ts
Comment thread services/apps/packages_worker/src/maven/schedule.ts
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
@ulemons ulemons merged commit 57bea95 into main Jun 25, 2026
16 checks passed
@ulemons ulemons deleted the feat/maven-fetcher-continue-as-new branch June 25, 2026 08:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Feature Created by Linear-GitHub Sync

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants