Skip to content

feat: Dash dashboard example for live activation-patching heatmaps#280

Open
RhizoNymph wants to merge 1 commit into
feat/integrationfrom
feat/patch-dashboard
Open

feat: Dash dashboard example for live activation-patching heatmaps#280
RhizoNymph wants to merge 1 commit into
feat/integrationfrom
feat/patch-dashboard

Conversation

@RhizoNymph

Copy link
Copy Markdown
Owner

What

Adds examples/online_serving/patch_dashboard/ — a Dash dashboard for interactive activation-patching studies. It drives POST /v1/patch_sweep with stream: true and renders the SSE cell events as a live-updating (layers × positions) heatmap, one panel per hook, filling in as the server completes each cell.

  • Denoising and ablation sweeps from a browser form: capture-sourced (one-call auto-capture of the clean prompt) or zeros-sourced with optional mask dims; all three metrics; multi-hook grids (attention-vs-MLP decomposition) as stacked panels sharing one color scale.
  • Positions as all-prompt, substring span (server-resolved), or explicit indices; the x-axis is labeled with actual token text via /tokenize + incremental /detokenize (best-effort), HTML-escaped — raw special tokens like <s> otherwise parse as unclosed tags in plotly labels and corrupt the axis layout.
  • Live progress + cancel: progress bar over cells landed; Cancel closes the HTTP stream so the server aborts outstanding cells. Summary panel reports clean/corrupt baselines, noise floor, per-hook argmax, and skipped/voided cells.
  • sweep_client.py is Dash-free (requests only, no vllm import): payload build/validation, SSE parsing, thread-safe grid accumulator — reusable outside the dashboard and unit-tested without a server (37 tests in test_sweep_client.py).
  • Docs: pointers added to docs/features/activation_patching.md (Streaming, Related files) and the docs/OVERVIEW.md feature index.

Why

The sweep endpoint already streams per-cell SSE events precisely so a consumer can size and fill a live heatmap (startcell* → summary), but the only consumer was the ASCII walkthrough in openai_patch_client.py. This gives the streaming surface a real interactive front end and a reference SSE consumer in plain HTTP.

Not a duplicate: no existing example or open PR covers a UI/dashboard consumer of /v1/patch_sweep.

Tests run: pytest examples/online_serving/patch_dashboard/test_sweep_client.py (37 passed); end-to-end smoke driving the real Dash callbacks against a fake SSE server (live partial grids, token labels, cancel, and error paths verified; figure render checked). Ruff check/format clean. AI assistance was used for this change.

Sibling-PR conflicts: none expected (new example directory; docs touch only the patching feature doc and OVERVIEW index).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant