worker: Add indexing dashboard in development by backspace · Pull Request #4189 · cardstack/boxel

backspace · 2026-03-13T18:34:44Z

I’m often wondering when developing how much is left to index:

This adds an event sink for index runners to report on their progress and a worker manager dashboard to report that.

github-actions · 2026-03-13T19:08:57Z

Host Test Results

1 files ±0 1 suites ±0 2h 31m 22s ⏱️ + 1m 36s
2 022 tests ±0 2 007 ✅ ±0 15 💤 ±0 0 ❌ ±0
2 037 runs ±0 2 022 ✅ ±0 15 💤 ±0 0 ❌ ±0

Results for commit 58ada6f. ± Comparison against base commit 0db3b27.

♻️ This comment has been updated with latest results.

…oard-cs-10443

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fc9fd377b9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

packages/realm-server/worker-manager.ts

packages/runtime-common/index-runner.ts

habdelra · 2026-03-14T13:17:58Z

wow! 🤩

habdelra · 2026-03-14T13:20:56Z

you can actually have multiple workers in development using env vars (i think it’s ALL_PRIORITY_WORKER_COUNT and HIGH_PRIORITY_WORKER_COUNT), can this handle showing multiple workers indexing simultaneously?

Copilot

Pull request overview

Adds a development-time “indexing dashboard” by emitting indexing progress events from IndexRunner through the worker/task stack, collecting them in the realm-server worker manager, and rendering a simple HTML dashboard (plus JSON endpoint).

Changes:

Introduces an IndexingProgressEvent flow from IndexRunner → tasks → Worker → realm-server worker process IPC → worker-manager.
Adds an in-memory IndexingEventSink with tests and exposes /_indexing-dashboard + /_indexing-status (non-ECS).
Updates CI to include the new realm-server test module.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
packages/runtime-common/worker.ts	Adds `IndexingProgressEvent` type and `reportProgress` plumbing into task args.
packages/runtime-common/tasks/index.ts	Extends `TaskArgs` with optional `reportProgress` callback.
packages/runtime-common/tasks/indexer.ts	Passes `reportProgress` through to `IndexRunner` as `onProgress`.
packages/runtime-common/index-runner.ts	Emits indexing progress events (start, per-file, finish).
packages/realm-server/worker.ts	Sends progress events over IPC to worker-manager.
packages/realm-server/worker-manager.ts	Adds event sink, dev-only routes, and parses `progress
packages/realm-server/indexing-event-sink.ts	New: stores active indexing + limited history snapshots.
packages/realm-server/handlers/handle-indexing-dashboard.ts	New: renders HTML dashboard from the sink snapshot.
packages/realm-server/tests/indexing-event-sink-test.ts	New: validates event sink behavior for start/visit/finish and concurrency across realms.
packages/realm-server/tests/index.ts	Registers the new test module.
.github/workflows/ci.yaml	Adds `indexing-event-sink-test.ts` to the realm-server test matrix.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

packages/realm-server/indexing-event-sink.ts

packages/runtime-common/index-runner.ts

backspace · 2026-03-16T15:06:02Z

packages/realm-server/worker.ts

+  function reportProgress(event: IndexingProgressEvent) {
+    if (process.send) {
+      process.send(`progress|${JSON.stringify(event)}`);
+    }
+  }


fixed in 281eefc

packages/realm-server/handlers/handle-indexing-dashboard.ts

packages/realm-server/worker-manager.ts

backspace · 2026-03-16T15:52:26Z

packages/realm-server/indexing-event-sink.ts

+  /** Active indexing state per realm */
+  #active = new Map<string, RealmIndexingState>();
+
+  /** Recently completed indexing runs (most recent first) */
+  #history: RealmIndexingState[] = [];
+
+  /** Max history entries to keep */
+  #maxHistory = 50;
+
+  handleEvent(event: IndexingProgressEvent): void {
+    switch (event.type) {
+      case 'indexing-started': {
+        this.#active.set(event.realmURL, {
+          realmURL: event.realmURL,
+          jobId: event.jobId,
+          jobType: event.jobType ?? 'unknown',
+          status: 'indexing',
+          totalFiles: event.totalFiles ?? 0,
+          filesCompleted: 0,
+          files: event.files ?? [],
+          completedFiles: [],
+          startedAt: Date.now(),
+          lastUpdatedAt: Date.now(),
+        });
+        break;


fixed in aeb0ff9

backspace · 2026-03-16T14:19:19Z

you can actually have multiple workers in development using env vars (i think it’s ALL_PRIORITY_WORKER_COUNT and HIGH_PRIORITY_WORKER_COUNT), can this handle showing multiple workers indexing simultaneously?

When I ran the realm server for this I had WORKER_HIGH_PRIORITY_COUNT=4 WORKER_ALL_PRIORITY_COUNT=4 so maybe it’s not being reflected, or something else is missing, I’ll look into it.

backspace · 2026-03-16T14:36:41Z

The reason you only see one active job despite having 8 workers is that realms are started sequentially in packages/realm-server/server.ts:279-284:
  // ideally we'd like to use a Promise.all to start these and the ordering
  // will just fall out naturally from cross realm invalidation. Until we have
  // that we should start the realms in order.
  for (let realm of this.realms) {
    await realm.start();
  }
Each realm.start() is awaited before the next one begins, so only one from-scratch-index job exists in the queue at a time
during startup. Your 8 workers are available, but only one has work at any given moment. Once that realm finishes indexing,
the next realm starts, and so on.

The comment in the code even acknowledges this — they'd like to use Promise.all but haven't implemented cross-realm
invalidation ordering yet. If/when that changes to parallel startup, you'd see multiple active jobs on the dashboard.

Seems true?

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

habdelra · 2026-03-16T15:32:11Z

The reason you only see one active job despite having 8 workers is that
realms are started sequentially in packages/realm-server/server.ts:279-284:
  // ideally we'd like to use a Promise.all to start these and the ordering
  // will just fall out naturally from cross realm invalidation. Until we have
  // that we should start the realms in order.
  for (let realm of this.realms) {
    await realm.start();
  }
Each realm.start() is awaited before the next one begins, so only one from-scratch-index job exists in the queue at a time
during startup. Your 8 workers are available, but only one has work at any given moment. Once that realm finishes indexing,
the next realm starts, and so on.
The comment in the code even acknowledges this — they'd like to use Promise.all but haven't implemented cross-realm
invalidation ordering yet. If/when that changes to parallel startup, you'd see multiple active jobs on the dashboard.
Seems true?

that's true--but that's just at startup. there are other scenarios after startup that could use multiple workers. like you run 2 different realm full reindex jobs, and then maybe do a incremental index because of an auto save. if you had it configured as such, that could be 3 concurrent jobs right there.

backspace · 2026-03-16T15:51:53Z

that's true--but that's just at startup. there are other scenarios after startup that could use multiple workers. like you run 2 different realm full reindex jobs, and then maybe do a incremental index because of an auto save. if you had it configured as such, that could be 3 concurrent jobs right there.

👍🏻 it does show concurrent jobs:

…oard-cs-10443

habdelra · 2026-03-16T15:53:26Z

could we include an update to the README.md to describe how to access this?

habdelra · 2026-03-16T15:54:16Z

I'm not sure if it's useful, but you might also want to show pending jobs in the queue too (just like you show completed jobs)

backspace · 2026-03-16T16:04:13Z

👍🏻 I added a mention in README and the dashboard now shows pending jobs:

backspace added 4 commits March 13, 2026 10:03

Add preliminary indexing dashboard

31843c5

Change to only show on localhost

ab51a9c

Move dashboard to worker (manager?)

ef95dc0

Add event sink test

8662411

backspace added 6 commits March 13, 2026 13:40

Add test to CI

a817683

Add formatting autofixes

ab41470

Fix test setup

1c963ab

Merge remote-tracking branch 'origin/main' into server/indexing-dashb…

bc5fe7f

…oard-cs-10443

Remove old approach

667d466

Remove dashboard in deployed environments

fc9fd37

backspace marked this pull request as ready for review March 13, 2026 23:33

backspace requested a review from a team March 13, 2026 23:33

chatgpt-codex-connector bot reviewed Mar 13, 2026

View reviewed changes

packages/realm-server/worker-manager.ts Outdated Show resolved Hide resolved

packages/runtime-common/index-runner.ts Outdated Show resolved Hide resolved

habdelra requested a review from Copilot March 14, 2026 13:17

Copilot started reviewing on behalf of habdelra March 14, 2026 13:18 View session

Copilot AI reviewed Mar 14, 2026

View reviewed changes

backspace and others added 5 commits March 16, 2026 09:42

Change to set for tracking files

33bc81a

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Remove dashboard processing outside development

b808d6a

Add handling for throws while indexing

c74d0f3

Remove sending outside of development

281eefc

Change to use set for tracking completion

153223c

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Change job key to be unique

aeb0ff9

Merge remote-tracking branch 'origin/main' into server/indexing-dashb…

3ef30c5

…oard-cs-10443

backspace added 2 commits March 16, 2026 09:00

Add dashboard mention to README

feae794

Add pending jobs listing

3f7842f

habdelra approved these changes Mar 16, 2026

View reviewed changes

backspace added 2 commits March 16, 2026 09:30

Fix lint error

1b8d3e4

Fix another lint error

58ada6f

backspace merged commit dc8558c into main Mar 16, 2026
109 of 110 checks passed

Conversation

backspace commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Host Test Results

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

habdelra commented Mar 14, 2026

Uh oh!

habdelra commented Mar 14, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

backspace Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

backspace Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

backspace commented Mar 16, 2026

Uh oh!

backspace commented Mar 16, 2026

Uh oh!

habdelra commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

backspace commented Mar 16, 2026

Uh oh!

habdelra commented Mar 16, 2026

Uh oh!

habdelra commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

backspace commented Mar 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

backspace commented Mar 13, 2026 •

edited

Loading

github-actions bot commented Mar 13, 2026 •

edited

Loading

habdelra commented Mar 16, 2026 •

edited

Loading

habdelra commented Mar 16, 2026 •

edited

Loading