Skip to content

Add $PMTXTS PPS timestamping (opt-in host timing telemetry)#5

Draft
peterlewis wants to merge 3 commits into
mitxela:masterfrom
peterlewis:pcc-firmware-proposals
Draft

Add $PMTXTS PPS timestamping (opt-in host timing telemetry)#5
peterlewis wants to merge 3 commits into
mitxela:masterfrom
peterlewis:pcc-firmware-proposals

Conversation

@peterlewis

@peterlewis peterlewis commented Jun 30, 2026

Copy link
Copy Markdown

Adds an opt-in (pps = on) $PMTXTS sentence emitted on each GPS PPS edge, carrying the sub-millisecond phase measurement plus a USB SOF-correlation tail for microsecond-class host timestamping.
Stack: base of the series (targets upstream master). The nine PRs form a linear stack; if you'd rather take them against an integration branch, create one and I'll retarget them all.
Try it now: fwt.bin · fwd.bin — the union of this PR stack, byte-identical to the merged result.
Docs: every feature in the stack is documented in docs/firmware-additions.md, shipped in the final (menu) PR.


Note

You can try this PR without hardware: the PCC web app runs the real clock4 firmware compiled to WebAssembly, and its clock is built from a rollup of the whole PR series together, so this PR's behaviour is live there alongside the others. The app's DEVICE → UPDATES panel shows the exact commit it was built from. Fair warning: the app around the firmware is beta and still has bugs — the firmware behaviour is the real thing, the app chrome less so.

Draft. One feature for the Mk IV: opt-in per-PPS host timestamping, found while building a host companion. Source-only — mk4-time/Core/Src/main.c and one documented config.txt key. Built and flashed on an STM32L476 time board and left running; it stays a draft because the qspi/output/*.bin release images still need rebuilding from the diff (see To do).

The two unrelated firmware fixes this branch used to carry — the charger-power hard-fault and the date-board text OOB — have moved to the hardening PR (#7), so this one is just the feature.

$PMTXTS per-PPS host timestamping (opt-in)

The clock disciplines to the PPS edge to sub-µs, but a host over CDC only ever sees NMEA, which arrives a few hundred ms after the second and is smeared by USB framing — so there is no host-visible timing-stability metric. (The stale cdc-pps branch approached this by toggling the CDC DCD line, which is capped at the ~1 ms USB framing floor.)

With pps = on in config.txt, one proprietary sentence per PPS edge over the existing CDC stream:

$PMTXTS,<seq>,<epoch>,<subms>,<systick>,<load>,<calerr>,<sincecal>,<temp>,<flags>[,<dwt_pps>,<sof_frame>,<dwt_sof>]*CC
field meaning
seq per-edge counter (gap detection)
epoch currentTime at the edge (Unix s, UTC)
subms/systick/load modelled sub-second position captured at the edge, before the SysTick reload — the phase error between the firmware's second and the true GPS edge
calerr the LSE cycle error calibrateRTC() already computes → ppm
sincecal seconds since last RTC calibration (holdover age)
temp STM32 internal die temperature (°C)
flags data_valid / had_pps / rtc_good

Time-critical fields are snapshotted in the PPS ISR (a handful of register reads); the sentence is snprintf-formatted and submitted to CDC from the main loop, never the ISR, serialised against the existing NMEA passthrough. With no enumerated host the record is dropped before any formatting, so the feature costs nothing when nobody is listening. The host's arrival time is USB-jittery, but the measurement is captured in firmware at the edge and carried in the payload, so transport jitter does not corrupt it. Absolute offset to UTC is not recoverable over USB — a host derives jitter, stability and drift, not absolute offset. mk4-time/Core/Src/main.c: capturePPS(), emitPPSTimestamp(), measure_temp(), the pps key, one main-loop hook.

USB SOF correlation — a good host timestamp despite USB jitter

$PMTXTS carries the phase measured at the edge, so USB jitter can't corrupt the value — but a host that wants to place the edge on its own clock is still limited by when the packet arrives, and USB is host-polled (arrival jitter ~6–110 ms here, depending on the host's serial buffering). This optional extension anchors each edge to a USB Start-Of-Frame so the host recovers the timing regardless of when it reads the packet:

  • enable DWT->CYCCNT (free-running, 12.5 ns) as a shared timebase;
  • latch (frame number, DWT) in the SOF interrupt every 1 ms;
  • append the edge's DWT and that anchor to the sentence: …,<dwt_pps>,<sof_frame>,<dwt_sof>.

A host that can read each USB frame's own hardware arrival time (e.g. macOS IOUSBDeviceInterface::GetBusFrameNumberWithTime) then places the edge at hostTime(sof_frame) + (dwt_pps − dwt_sof)/f_dwt, immune to delivery lateness. f_dwt self-calibrates from consecutive dwt_pps deltas, so no core-clock constant is assumed.

Measured (STM32L476 time board): ~6–110 ms raw arrival jitter → ~100–175 µs SOF-corrected — down at the clock's own PPS floor, i.e. the USB transport is no longer the limiting factor. This is where the stale cdc-pps DCD approach wanted to go but couldn't: a modem-status bit has no payload to carry the frame, so it can't beat the ~1 ms framing floor.

Design notes

  • The tail is optional and valid-gated — emitted only once a real SOF has latched, so the first edge after enumeration, with pps just toggled on, or a build with no USB SOF (the WASM emulator) sends the plain nine-field sentence, never a stale (0,0) anchor.
  • The SOF latch runs only when pps = on; the 1 kHz SOF interrupt is cheap and sits below the priority-0 display DMA, so it can't perturb the display. valid resets when off, so a re-enable can't reuse a stale anchor.
  • Backward-compatible — a nine-field parser reads the first nine fields and ignores the tail; the *CC checksum still covers the whole body. NMEA_BUF_SIZE 90→128 for the longer sentence.

Considered and ruled out: last-microsecond injection into the outgoing packet (the L476's USB-OTG core has no addressable packet memory to rewrite), and a DCD/serial-state pulse (one bit, no payload → can't carry the frame). Edge cases checked: torn reads (the anchor is read under __disable_irq in thread context), 16-bit atomicity (single aligned access on M4), DWT wrap (the PPS→SOF gap is always ≪ the 53.7 s wrap). usbd_conf.c: PCD_SOFCallback + Sof_enable; main.c: the DWT enable, the capturePPS() latch, the emit tail.

Temperature

The die-temperature field made it cheap to characterise the oscillators against temperature on the unit. The 10 MHz HSE — which holds time through a running GPS-loss holdover — measured flat, ≈ 0.011 ppm/°C (R² ≈ 0.37) over a ~6 °C sweep, i.e. running-holdover drift from temperature is sub-millisecond over any realistic outage, so no compensation is included. The 32 kHz LSE has a real tempco but it only bites across power-loss recovery, where the MCU is off and cannot compensate. The die-temp telemetry is kept because it is useful and proven; the steering code is not. (The sweep is narrow — see To do.)

Data (STM32L476 time board, GPS-locked)

$PMTXTS,24,1782815319,0,87,79999,0,112,33,7*0A
$PMTXTS,25,1782815320,0,85,79999,0,113,33,7*02
$PMTXTS,26,1782815321,0,85,79999,0,114,33,7*07
$PMTXTS,27,1782815322,0,85,79999,0,115,33,7*04
$PMTXTS,28,1782815323,0,85,79999,0,116,34,7*0E
  • boots clean; $PMTXTS streams at 1 Hz interleaved with NMEA; checksums valid; seq contiguous
  • phase jitter ≈ 10 ns RMS / 25 ns pk-pk (systick 85 ± 2 counts)
  • die temp 33–34 °C; full lock (flags = 7)

To do

Before this leaves draft:

  • Rebuild the qspi/output/*.bin release images from the source diff. The diff is source-only; the committed binaries are unchanged and would need regenerating to match.
  • Widen the HSE temperature characterisation. The "flat" conclusion rests on a ~6 °C sweep at R² ≈ 0.37 — enough to rule out a large tempco, not to state a firm coefficient. A broader range would confirm or revise it.
  • Confidence-gated display (proposed, not in this diff — kept here as the intended use of the telemetry): dim then blank the sub-second digits as holdover uncertainty grows. calerr and sincecal give a live time-uncertainty, so the display can show only the digits it can still stand behind, degrading like an analogue needle. — since realized downstream as significance_fade in PR Self-learning holdover temperature compensation #9 (gradual sub-second dimming from the live holdover uncertainty).

@peterlewis peterlewis force-pushed the pcc-firmware-proposals branch from 4f901ab to 02671b2 Compare July 4, 2026 09:00
@peterlewis peterlewis changed the title Fix charger-power hard-fault & date OOB write; add $PMTXTS PPS timestamping Add $PMTXTS PPS timestamping (opt-in host timing telemetry) Jul 4, 2026
peterlewis added a commit to peterlewis/clock4 that referenced this pull request Jul 6, 2026
The exact residue between the pure union of the five draft PRs (mitxela#5 $PMTXTS,
mitxela#6 astro pack, mitxela#7 hardening, mitxela#8 sidereal/solar, mitxela#9 tempcomp + significance-fade
+ tc_seed — mitxela#5 and mitxela#6 arrive as ancestors of mitxela#9 and mitxela#8) and the rollup tree that
has been emulator-verified and hardware-run:

- astro.c/astro.h/test_astro.c: rollup astro refinements not yet folded back into
  the refreshed PR mitxela#6/mitxela#8 heads (candidates for a future PR update)
- version.c: rollup version string
- qspi/output/*.bin: rollup's built firmware images
- .settings IDE noise

After this commit the megapack tree is BYTE-IDENTICAL to the previously validated
rollup (feaf970) while the merge ancestry proves containment of all five PR heads.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
peterlewis added a commit to peterlewis/clock4 that referenced this pull request Jul 7, 2026
…xela#9)

The source said 0.0.5 (misidentifying a rollup build as stock) while the
committed image embedded 0.0.6 from an earlier build lineage. Source and
freshly built, CRC-gated image now both say 0.0.6. No cuckoo — that work is
parked on its own branch and versions later.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
peterlewis and others added 3 commits July 12, 2026 11:45
With `pps = on` in config.txt, emit one proprietary NMEA sentence per PPS edge
over the existing CDC stream, carrying the sub-second phase captured at the edge
plus calibration / holdover / die-temperature telemetry. Time-critical fields are
snapshotted in the PPS ISR; formatting and CDC submission happen in the main loop,
serialised against the NMEA passthrough. With no enumerated host the record is
dropped before any formatting.

mk4-time/Core/Src/main.c: capturePPS(), emitPPSTimestamp(), measure_temp(), a
`pps` config key and one main-loop hook. qspi/config.txt documents the key.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ver USB

Experimental attack on the ~6ms host-driven USB read jitter that swamps the
clock's true ~180us precision — without a hardware PPS wire. The OTG_FS core
rules out mitxela's last-microsecond FIFO injection, so instead we let the host
anchor each PPS to a USB Start-Of-Frame, whose own arrival time the host can
read in hardware (macOS IOKit GetBusFrameNumberWithTime).

- Enable DWT->CYCCNT (free-running 12.5ns core-cycle counter) as the timebase
  both the PPS edge and each SOF are latched against. Unaffected by tempcomp
  SysTick steering (counts raw core clock), so it's a stable monotonic ruler.
- Enable the USB SOF interrupt; PCD_SOFCallback latches (DWT, 11-bit frame from
  DSTS[13:8]) every 1ms, as its first act for minimal latency.
- capturePPS() latches DWT at the edge; emitPPSTimestamp() appends the tail
  ,dwt_pps,sof_frame,dwt_sof to $PMTXTS (read under __disable_irq, no torn read).
- Bump NMEA_BUF_SIZE 90->128 for the longer sentence.

Host places the edge at hostTime(sof_frame) + (dwt_pps-dwt_sof)/f_dwt, immune to
delivery lateness; dwt_pps deltas self-calibrate f_dwt (no core-clock assumption).
Backward-compatible: lenient $PMTXTS parsers read the first 9 fields. Bench-only;
+1kHz SOF ISR (~0.05% CPU) is the cost to watch. Builds clean (0 errors).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adversarial audit of the SOF-correlation change surfaced two real issues:
- Stale first anchor: pps_sof_dwt/frame init to 0, so the first PPS emitted
  after enumeration (or with pps just toggled on, or on the emulator which has
  no USB SOF) carried a (0,0) anchor → a ~53s/−729ms host error on that record.
  Now a pps_sof_valid flag gates the tail: emit the plain 9-field sentence until
  a real SOF has latched. Backward-compatible; also makes the emulator correct.
- SOF work ran unconditionally. Gate the latch on pps_ts_enabled (the SOF IRQ
  still fires — cheap, below the priority-0 display DMA — but does nothing when
  the feature is off); clear valid when off so a re-enable can't reuse a stale
  anchor. config.txt documents that pps=on enables the SOF tail.
Builds clean (0 errors). Refuted findings (torn read, 16-bit atomicity, buffer
truncation, display preemption, scope) left as-is.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
peterlewis added a commit to peterlewis/clock4 that referenced this pull request Jul 12, 2026
Comprehensive documentation for every feature in PRs mitxela#5-mitxela#10 and the three
follow-ons (ADEV, star transits, the menu), written to slot into the
project's documentation page: all new display modes with exact row renders,
every config.txt key (menu-settable ones marked, precedence explained), every
serial command and emitted sentence field-by-field ($PMTXTS/$PMTXTC/$PMADEV/
$PMHDEV/$PMSTAR), a full user guide for the on-device menu, the SETTINGS.BIN
and STARS.BIN file contracts, and the 0.0.5/0.0.2 version train with the
per-build-tag rationale. Describes the ROLLUP build (the tested union this
tree IS), with a real unit's learned tempcomp model as the worked example.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
peterlewis added a commit to peterlewis/clock4 that referenced this pull request Jul 12, 2026
…xela#9)

The source said 0.0.5 (misidentifying a rollup build as stock) while the
committed image embedded 0.0.6 from an earlier build lineage. Source and
freshly built, CRC-gated image now both say 0.0.6. No cuckoo — that work is
parked on its own branch and versions later.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
peterlewis added a commit to peterlewis/clock4 that referenced this pull request Jul 12, 2026
Comprehensive documentation for every feature in PRs mitxela#5-mitxela#10 and the three
follow-ons (ADEV, star transits, the menu), written to slot into the
project's documentation page: all new display modes with exact row renders,
every config.txt key (menu-settable ones marked, precedence explained), every
serial command and emitted sentence field-by-field ($PMTXTS/$PMTXTC/$PMADEV/
$PMHDEV/$PMSTAR), a full user guide for the on-device menu, the SETTINGS.BIN
and STARS.BIN file contracts, and the 0.0.5/0.0.2 version train with the
per-build-tag rationale. Describes the ROLLUP build (the tested union this
tree IS), with a real unit's learned tempcomp model as the worked example.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@peterlewis peterlewis force-pushed the pcc-firmware-proposals branch from 199a215 to 1fdbf33 Compare July 12, 2026 12:03
peterlewis added a commit to peterlewis/clock4 that referenced this pull request Jul 12, 2026
…xela#9)

The source said 0.0.5 (misidentifying a rollup build as stock) while the
committed image embedded 0.0.6 from an earlier build lineage. Source and
freshly built, CRC-gated image now both say 0.0.6. No cuckoo — that work is
parked on its own branch and versions later.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
peterlewis added a commit to peterlewis/clock4 that referenced this pull request Jul 12, 2026
Comprehensive documentation for every feature in PRs mitxela#5-mitxela#10 and the three
follow-ons (ADEV, star transits, the menu), written to slot into the
project's documentation page: all new display modes with exact row renders,
every config.txt key (menu-settable ones marked, precedence explained), every
serial command and emitted sentence field-by-field ($PMTXTS/$PMTXTC/$PMADEV/
$PMHDEV/$PMSTAR), a full user guide for the on-device menu, the SETTINGS.BIN
and STARS.BIN file contracts, and the 0.0.5/0.0.2 version train with the
per-build-tag rationale. Describes the ROLLUP build (the tested union this
tree IS), with a real unit's learned tempcomp model as the worked example.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant