feat: Dash dashboard example for live activation-patching heatmaps#280
Open
RhizoNymph wants to merge 1 commit into
Open
feat: Dash dashboard example for live activation-patching heatmaps#280RhizoNymph wants to merge 1 commit into
RhizoNymph wants to merge 1 commit into
Conversation
Co-authored-by: Claude
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
examples/online_serving/patch_dashboard/— a Dash dashboard for interactive activation-patching studies. It drivesPOST /v1/patch_sweepwithstream: trueand renders the SSE cell events as a live-updating (layers × positions) heatmap, one panel per hook, filling in as the server completes each cell.zeros-sourced with optional mask dims; all three metrics; multi-hook grids (attention-vs-MLP decomposition) as stacked panels sharing one color scale./tokenize+ incremental/detokenize(best-effort), HTML-escaped — raw special tokens like<s>otherwise parse as unclosed tags in plotly labels and corrupt the axis layout.sweep_client.pyis Dash-free (requests only, no vllm import): payload build/validation, SSE parsing, thread-safe grid accumulator — reusable outside the dashboard and unit-tested without a server (37 tests intest_sweep_client.py).docs/features/activation_patching.md(Streaming, Related files) and thedocs/OVERVIEW.mdfeature index.Why
The sweep endpoint already streams per-cell SSE events precisely so a consumer can size and fill a live heatmap (
start→cell* →summary), but the only consumer was the ASCII walkthrough inopenai_patch_client.py. This gives the streaming surface a real interactive front end and a reference SSE consumer in plain HTTP.Not a duplicate: no existing example or open PR covers a UI/dashboard consumer of
/v1/patch_sweep.Tests run:
pytest examples/online_serving/patch_dashboard/test_sweep_client.py(37 passed); end-to-end smoke driving the real Dash callbacks against a fake SSE server (live partial grids, token labels, cancel, and error paths verified; figure render checked). Ruff check/format clean. AI assistance was used for this change.Sibling-PR conflicts: none expected (new example directory; docs touch only the patching feature doc and OVERVIEW index).