Skip to content

Implementation Checkpoint#31

Open
ContinuumLS wants to merge 24 commits into
mainfrom
feature-ai
Open

Implementation Checkpoint#31
ContinuumLS wants to merge 24 commits into
mainfrom
feature-ai

Conversation

@ContinuumLS

@ContinuumLS ContinuumLS commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR brings the feature-ai branch up to a more capable real-time dashboard for the HamSCI Ground Magnetometer network inline with the minimum requirements. It adds live data ingestion over WebSocket and MQTT, a redesigned configuration panel, a tabbed multi-source interface, on-the-fly data processing (orientation transforms and moving-average filtering), configurable visualization, and the supporting Docker/AI-governance infrastructure.

All connection, processing, and multi-source behavior was verified end-to-end in a headless browser against local fake WebSocket/MQTT hosts; the moving-average algorithm also has unit tests.

Issues closed

Highlights

Data ingestion & connections (#19, #20)

  • A per-source connection manager (js/transit.js) with pluggable WebSocket and MQTT-over-WebSocket transports, both emitting the same normalized readings. MQTT supports a configurable broker/topic, optional username/password auth (with an "Auth failed" status), and automatic reconnection. Vendors mqtt.js.
  • A redesigned, opaque Configuration panel (sidebar): source name, a WebSocket / MQTT / File source-type selector with conditional fields, input validation, a single Connect action, and localStorage persistence with auto-reconnect on load.

Multiple data sources (#21)

  • A tab bar for monitoring several stations at once. Each tab has an independent connection and data buffer; all tabs stay connected and keep buffering in the background, while the active tab drives the shared plots/spreadsheet/trends. Add / switch / close, per-tab status dots, live rename, and a source cap for performance.

Data processing (#22, #24)

  • Orientation transforms: per-source 90° X/Y/Z rotations applied to the displayed field.
  • Moving-average filtering: a trailing, time-based filter (10 / 30 / 60 s) overlaid on the raw data, with the raw series faded; handles data gaps gracefully. Includes unit tests (js/filter_test.js).

Visualization (#17, #18, #28)

  • Configurable time window with autoscroll that the user can pause by zooming/panning.
  • Total-field magnitude shown in the Current Reading panel, the spreadsheet, and as its own plot trace.
  • Sparkline "Trends" for a compact at-a-glance overview.
  • A static spreadsheet header over an independently scrolling body, and a layout bounded to the viewport.

Performance (#9)

  • Per-source measurement and sparkline buffers are capped (rolling window) so a continuous 1 Hz feed doesn't grow unbounded.

Infrastructure & governance

  • Docker/Deno serving, project docs, and the AI-governance scaffold. All AI-assisted commits are disclosed in ai/ai_usage_log.md per University of Scranton / HamSCI / NSF policy.

Testing

  • Headless-browser (Playwright) verification of: WebSocket and MQTT connect/receive/render, MQTT auth accept/reject, connection-panel validation/persistence/auto-reconnect, file load, rotation, moving-average overlay, and the full multi-tab flow (two independent live sources, background buffering, persistence + reconnect-on-reload, clean switch between sources).
  • Unit tests for the moving-average filter (deno test js/filter_test.js).

Notes

  • Adds a vendored dependency: vendor/mqtt.min.js (MQTT.js 5.x).
  • The Light/Dark mode toggle is present in the UI but not yet wired up.

🤖 Generated with Claude Code (minor edits by author)

ContinuumLS and others added 24 commits June 9, 2026 11:44
- config.toml now mounts to /etc/mag-usb/
- Add logs directory for container to dump .log files
…_project_template

- Add CLAUDE.md with project metadata (PI, NSF grants, stack, repo structure)
- Add .claude/settings.json, commands/commit.md, rules/ai-governance.md, rules/js-code.md
- Add ai/ai_usage_log.md with initial session entry
- Update .gitignore with Claude Code local-state exclusions

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add Project Management section linking to the HamSCI gmag_webui
project board (https://github.com/orgs/HamSCI/projects/7/views/1).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a trailing, time-based moving-average filter (FR-DP-04) with a
configurable window (10s/30s/60s) and on/off toggle. Smoothed
H/E/Z/Magnitude/Temperature traces overlay the raw data, which fades
while the filter is on. The window is time-based, so data gaps are
handled gracefully.

- js/filter.js: trailing moving-average algorithm (+ unit tests)
- js/data/plots.json: 5 hidden overlay traces (indices 5-9)
- index.html: Moving Average controls in Data Postprocessing
- js/index.js: settings + migration, overlay recompute/append, control
  wiring, file-upload integration

Also fix a pre-existing ReferenceError in the magclose reset handler
(undefined mainTraces/slTraces) that the new trace set interacted with;
replaced with a resetPlots() that re-initializes both plots.

Closes #24

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ersistence

- Bound the page to the viewport (no more page scroll/break) and make the
  spreadsheet fill its box and scroll internally
- Replace the recoiling sticky table header with a static header over an
  independently scrolling body, with aligned columns
- Rebuild all spreadsheet rows on coordinate-rotation save so existing rows
  reflect the change, not just new ones
- Keep the Date column on one line; widen the right column for readability
- Apply the saved moving-average filter state on load so the overlay shows
  after a page refresh when the filter is enabled

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Redesign the config sidebar into an opaque, instrument-style panel with
collapsible Connection / Processing / Display sections.

- Connection: source name, WebSocket/MQTT/File type selector with
  conditional fields, a single Connect button with input validation,
  localStorage persistence, load-on-startup, and auto-reconnect
- MQTT fields are present but visual-only (live ingestion is #19)
- Refactor transit.js from a cookie-based, WebSocket-only script into a
  connection manager (window.MagConnection) emitting the same
  magread/magclose/magerror events; index.js rendering is unchanged
- Anchor the sidebar flush between the header and footer

Refs #20

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Vendor MQTT.js 5.15.1 and add an MQTT transport to the connection
manager: connect to a broker over WebSocket, subscribe to a topic, parse
each message in the same JSONL reading format and dispatch magread, with
optional username/password auth and MQTT.js-driven reconnection.

- Validate broker URL + topic at Connect time
- Surface a dedicated "Auth failed" status (and stop retrying) when the
  broker rejects credentials
- WebSocket <-> MQTT source switching works; WebSocket path unchanged

Verified end-to-end against local aedes MQTT-over-WS brokers, including a
credential-required broker (accept + reject).

Closes #19

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Restructure the single-source dashboard onto a sources[] + activeSourceId
model so multiple data sources can each keep independent state. No visible
behavior change yet; this is groundwork for the tabbed interface (#21).

- transit.js: rewritten as a DOM-free per-source connection factory,
  MagConnection.create(config, { onReading, onStatus }), supporting both
  WebSocket and MQTT transports
- index.js: per-source sessions (own measurement/sparkline buffers and
  connection), settings migration from the old connection+transform shape,
  per-source reading/status routing with only the active session rendering,
  connection-status mapping moved here, and per-source rotation
- Unconfigured sources no longer auto-connect

Single-source parity (WS/MQTT/auth/file/rotation/filter/persistence)
verified via headless browser.

Refs #21

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Build a tab bar so several magnetometer stations can be monitored at once,
each with an independent connection and state.

- Tabs: add / switch / close, per-tab connection-status dot, live rename,
  6-source cap, and an always-at-least-one-tab rule
- All tabs stay connected and keep buffering in the background; only the
  active session renders to the shared plots/spreadsheet/trends; switching
  re-renders from the active session; rotation is per-tab
- Fix residual rendering when switching sources (a previous source's axis
  ranges, range slider, and WebGL traces lingered): Plotly mutates the
  shared layout's axis objects with computed ranges, so deep-clone the
  layout per draw, purge + newPlot on reset, and key uirevision to the
  active source

Verified via headless browser with two independent live sources: create/
switch/close, background buffering, persistence + reconnect-all on reload,
and a clean switch to an empty source.

Closes #21

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment