macOS: two float32 tests fail under AppleClang 21 / Xcode 26.5 (Float32Parity.ToneRowWithNarrowbandGuard, itu_echo.EchoStability<float>)

[tap]

The macOS AppleClang CI job fails two tests. It is a toolchain/runner-image regression, not a code change — main last passed on the same tree four days earlier. Filing this so the evidence does not stay buried in a comment on an unrelated PR (#30).

The failure

99% tests passed, 2 tests failed out of 181
The following tests FAILED:
  114 - itu_echo.EchoStability<float>            (Failed)
  179 - Float32Parity.ToneRowWithNarrowbandGuard (Failed)

tests/test_float32.cpp:166: Failure
Expected: (max_level_dbm0a(rr.out, r.rs.fs, static_cast<size_t>(10.0 * r.rs.fs), rr.out.size())) < (r.gate),
  actual: 42.337837234992982 vs -55
fs 16000

Full log: macOS AppleClang job 90074362335 (the itu_echo assertion detail is in there too).

Not a code change

The run above is from PR #30, whose entire diff is one file — .github/pull_request_template.md, 40 insertions, no C++. Its base commit is 0682238, which is also the current main HEAD, and main's own last CI run on that exact commit ("Bump TapHouse drift pin to v5", 2026-07-23) concluded success. Same tree: green on 2026-07-23, red on 2026-07-27.

main has not been re-run since, so it is untested rather than known-good on the new toolchain. Re-running it would make that explicit.

Environment change

The macOS runners now report AppleClang 21.0.0.21000101 / Xcode 26.5 and ship CMake 4. The image move is independently corroborated: the same update broke configure in tap/AmbiTap-Max#18 and tap/MuTap-Max#14 with Compatibility with CMake < 3.5 has been removed from CMake, which required a min-api pin bump.

Scope

Green in the same run: Linux Clang, Linux GCC, Windows MSVC, Hexagon cross (QEMU), Cortex-M55 cross (QEMU), ASan + UBSan, Instruction-count ratchet, Suppressor branch-free/branchy parity, clang-format, drift.

It is also not a blanket AppleClang 21 problem across the family — tap/AmbiTap#24's build-test (macos-appleclang) job is green on the same runners today, as are SampleRateTap's jobs. What is distinctive here is that MuTap carries the family's tightest numerical assertions (measured ITU-T thresholds).

This is probably not floating-point drift

42.34 dBm0(A) against a −55 dBm0(A) gate is roughly a 97 dB miss. That is far too large for single-precision rounding differences. It reads more like the narrowband guard or the suppressor not engaging at all under this build than a threshold that needs widening. Please resist retuning the gate until the mechanism is understood — these thresholds are the ITU-T compliance evidence.

Leading hypothesis: the vDSP float32 FFT backend

Both failing tests are on the float32 path, and macOS is the only platform where the float32 FFT backend differs. Per DspTap's CMakeLists.txt, TAP_DSP_FFT_ACCELERATE defaults ON for Apple and routes the float32 real FFT through Apple's vDSP instead of Ooura; Linux and Windows use Ooura. Double is untouched (always Ooura), and the documented contract is that "on the float32 path vDSP agrees with Ooura to single-precision rounding (not bit-identical)".

vDSP is SDK-provided, so an Xcode 26.5 bump can change its numerics without any compiler codegen change being involved.

One-flag bisection — highest-value next step. Build macOS with the Apple backend forced off and re-run the two tests:

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DTAP_DSP_FFT_ACCELERATE=OFF
cmake --build build
ctest --test-dir build --output-on-failure -R 'Float32Parity.ToneRowWithNarrowbandGuard|itu_echo.EchoStability'

• Tests pass → it is vDSP, and the float32 vDSP/Ooura agreement contract is broken on the new SDK.
• Tests still fail → it is AppleClang 21 codegen; next probe -ffp-contract=off and compare -O0/-O1/-O2, since FMA contraction in the suppressor's recursions is the usual suspect.

Related context

This repo pins submodules/dsptap at 9cbfbca, which is 11 commits behind DspTap main (dfbe18d). So the vDSP backend MuTap actually compiles is not the one DspTap's own green macos-vdsp job certifies. That does not explain a failure that appeared this week — the pin has not moved — but it does mean DspTap's green macOS CI is not evidence about the code path failing here, and a pin bump is worth trying as part of the investigation.

Suggested next steps

1. Re-run main's CI to confirm the failure is independent of any PR.
2. Run the -DTAP_DSP_FFT_ACCELERATE=OFF bisection above on a Mac with Xcode 26.5.
3. Before adjusting any threshold, instrument whether the narrowband guard / suppressor gain is actually engaging on the failing run — the magnitude suggests a state problem, not a tolerance problem.
4. Consider bumping submodules/dsptap so the backend under test matches the one DspTap certifies.

Not reproducible outside macOS: Linux Clang and Linux GCC both pass, so this needs a Mac on the new toolchain.

[tap]

Correcting my earlier diagnosis here: these two failures are intermittent, not deterministic, which rules out the vDSP-backend explanation as stated.

Evidence, all from CI on this repo:

Run	Commit	macOS result
#32 run 1	761b6d5 (docs only)	failed itu_echo.EchoStability<float> + Float32Parity.ToneRowWithNarrowbandGuard
#32 run 2	e989c56 (CI flag + CMake alias; nothing in the DSP path)	passed all 181
main	d2a3abc (2026-07-27)	failed
main	0682238 (2026-07-23)	passed
main	2e024739 (2026-07-21)	failed

The failure amounts from run 1:

itu_echo.EchoStability<float>            4.5505849575754098 vs 4.5      (1.1% miss)
Float32Parity.ToneRowWithNarrowbandGuard 42.337837234992982 vs -55      (~97 dB miss)

A fixed backend difference cannot pass on one run and fail on the next with the same DSP code. So "macOS's vDSP float32 FFT computes different numbers" cannot be the whole story.

What fits instead: nondeterministic state, most likely an uninitialized read. That single root cause explains all three otherwise-awkward facts — a marginal 1.1% miss and a catastrophic 97 dB miss coexisting; the failure being macOS-mostly (different allocator fill behaviour, not different arithmetic); and Linux ASan+UBSan staying green, because ASan does not detect uninitialized reads at all.

Suggested reordering of the investigation:

1. MemorySanitizer (or Valgrind memcheck) on the two failing tests. This repo has no MSan job today; that is the gap that would have caught this. Note MSan needs an instrumented libc++ to avoid false positives, so Valgrind may be the faster first look.
2. Re-run the macOS job a few times on an unchanged commit to measure the failure rate — useful for knowing when a candidate fix is actually confirmed rather than lucky.
3. -DTAP_DSP_FFT_ACCELERATE=OFF bisection — still worth doing, but now as a second step. If the bug is uninitialized state, turning off vDSP may change the failure rate without being the cause, which is exactly the kind of result that misleads.

Caveats on my evidence: the two #32 runs are different commits (identical DSP path, but I cannot prove from logs that the test binaries were byte-identical), and I could not reproduce either failure directly — this container has no macOS, and both tests pass on Linux, where I ran the full 181 clean.

This also means the submodules/dsptap pin being held back at 9cbfbca is probably not buying anything for this investigation, since the suspected cause is not in the FFT backend. Worth unblocking that separately once there's a confirmed diagnosis.

---

Generated by Claude Code

[tap]

Correcting a detail in my comment above, in the direction that makes the case stronger.

I said the two contradicting runs were on different commits and hedged that I couldn't prove the test binaries were identical. That was wrong. They are the same commit:

Run	Head SHA	Trigger	Result
30374907771	e989c569	push	failed (macOS: itu_echo.EchoStability<float>, Float32Parity.ToneRowWithNarrowbandGuard)
30380046998	e989c569	pull_request	passed all 181

Same SHA, same tree, same workflow, same runner image — opposite outcomes. There is no compiler-, flag-, or backend-difference explanation available; the nondeterminism is in the program or its environment. The earlier caveat about binary identity does not apply.

Everything else in my previous comment stands, and the uninitialized-read hypothesis is now the leading one by elimination rather than by analogy. MSan or Valgrind on those two tests remains the cheapest next probe.

Practical note for anyone reading a red PR in this repo: because ci.yml triggers on both push and pull_request, a commit gets two runs, and GitHub shows the union of their check runs on the PR. One flaky red macOS job therefore keeps the PR looking red even after a later run of the identical commit goes green. #32 was in exactly that state; I re-ran the failed job rather than pushing an empty commit.

---

Generated by Claude Code

[tap]

New data point from re-running the failed macOS job on the same commit e989c569 (attempt 2), and it narrows things usefully. The set of failing tests is not stable across runs.

	attempt 1	attempt 2
itu_echo.EchoStability<float>	fail	fail
g168_adapted.ToneStability<float>	pass	fail
Float32Parity.ToneRowWithNarrowbandGuard	fail	fail
everything else (178 tests)	pass	pass
	2 failed / 181	3 failed / 181

Three runs of one commit, three different outcomes: 0 failures, 2 failures, 3 failures. A deterministic numerical difference would fail the same set every time, so this confirms nondeterminism independently of the earlier pass/fail pair.

The important pattern: every failure is on the float32 path. All three are <float> instantiations or the Float32Parity suite, and in both attempts the <double> counterparts of the very same tests passed — itu_echo.EchoStability<double> and g168_adapted.ToneStability<double> were green each time. 178 of 181 tests pass on every run; the flakiness is confined to float32.

This reverses the advice I gave in my comment above, and I want to be explicit about that. I said the -DTAP_DSP_FFT_ACCELERATE=OFF bisection should be the second step behind MSan, on the grounds that a generic uninitialized read anywhere could explain things. The float32-only confinement makes that wrong: double always goes to Ooura, and float32 is exactly where vDSP is substituted on macOS. So the two facts together —

• nondeterministic run to run, and
• confined to the float32 path

— point at uninitialized or improperly-reset state inside the float32/vDSP path specifically, rather than at arithmetic differences (my original framing) or at a generic uninitialized read (my revised framing). Both of my earlier readings were too broad.

Suggested order now:

1. -DTAP_DSP_FFT_ACCELERATE=OFF on macOS, run the suite several times. This is now the highest-value probe, but as a discriminator rather than a numerical comparison: if the flakiness vanishes with Ooura float32, the bug is in the vDSP wrapper (setup object lifetime, a scratch buffer not zeroed, state reused across prepare() calls). If it persists, the bug is elsewhere in the float32 path and vDSP is a red herring. Either result is decisive, which is what the previous framings lacked.
2. Several repeats per configuration, since one green run proves nothing here — attempt 1 vs the pull_request run already demonstrated that.
3. MSan or Valgrind on the three tests, still worthwhile, and still the thing that would have caught this: Linux ASan+UBSan is green on every run and ASan does not detect uninitialized reads.

Two caveats unchanged: I cannot reproduce any of this locally (no macOS here; all 181 pass on Linux, including under ASan+UBSan), and all of the above is inference from CI logs rather than from a debugger.

One consequence worth acting on separately: if the fault is in the vDSP float32 path, holding submodules/dsptap back at 9cbfbca may be actively unhelpful — the pin predates 11 commits of upstream work that could include relevant FFT-path changes. Worth checking whether anything in that range touches the vDSP backend before assuming the hold is protective.

---

Generated by Claude Code

[tap]

Ran the investigation on Linux. Three hypotheses eliminated — including both of mine — and the remaining suspect is now localized with a number attached.

Eliminated, with evidence

1. Data race — no. There is no threading anywhere in include/: no std::thread, std::async, #pragma omp, dispatch_* or pthread_*. The library is single-threaded, which also explains why there is no TSan job to have caught anything. And ctest runs one process per test, so cross-test contamination is out too.

2. Uninitialized read in shared float32 code — no. This was my leading hypothesis and it is refuted:

• Valgrind memcheck, --track-origins=yes, on Float32Parity.ToneRowWithNarrowbandGuard and g168_adapted/0.ToneStability: ERROR SUMMARY: 0 on both. Zero uninitialised reads, zero errors of any kind.
• MALLOC_PERTURB_ = 0, 1, 42, 85, 170, 255 on all three failing tests: all pass, every pattern. If an uninitialised heap read mattered, varying the fill byte would have moved something.

3. Knife-edge narrowband-guard latch — no. This was my other hypothesis, from fd_kalman.h's own note that float32 reads -20 against a -49.3 gate without the guard and -72 with it — so the guard is the only thing holding this row. At 16 kHz/256 the hold is max(8, 187/3) = 62 blocks, and a single non-concentrated block halves the counter, so latching needs ~62 consecutive good blocks. That looked like a probabilistic latch.

It isn't. A public-API probe (process_block + narrowband_frozen()) shows it latches at block 61 — the earliest possible — in every variation I tried: three amplitudes (-40, -16, 0 dB), on-bin and off-bin (1000/1031 Hz), 440 Hz and 3 kHz, and ten different start phases. conc for a tone sits far above 0.8, not near it.

4. Uncertified FFT geometry — no. Both sizes this chain uses are covered: canceller N = 2 × 256 = 512, residual suppressor N = analysis_blocks × 256 = 2048. Those are exactly DspTap's CertifiedGeometries parity parameters.

Established: how much perturbation this row can actually absorb

I replicated the failing 16 kHz row standalone and injected a relative perturbation into the float32 forward FFT (patched copy of fft.h in scratch — the repo was untouched, verified clean).

Baseline reproduces the committed number: -64.78 vs the recorded -64.8.

FFT perturbation (relative)	level dBm0(A)	vs gate -55
0 (baseline)	-64.78	pass
1e-9 / 1e-8	-64.78	pass
1e-7	-69.02	pass
4e-7 ← vDSP's documented agreement	-72.83	pass
1e-6	-73.72	pass
1e-5	-59.44	pass
1e-4	-42.32	FAIL
1e-3	-20.02	FAIL

Two things fall out of this:

• A perturbation the size of vDSP's documented accuracy does not break this row. fft.h states vDSP agrees with Ooura to < 4e-7 relative; at 4e-7 the row reads -72.83 and actually improves. Breaking it needs ≥ 1e-4 — roughly 250× the documented contract.
• The 1e-3 row lands at -20.02, matching fd_kalman.h's documented guard-off figure ("float -20 against a -49.3 gate") almost exactly. That is a good consistency check that the failure mode really is the constraint-churn weight walk.

So MuTap's float32 tolerance is not the problem. For macOS to fail this row, the vDSP path has to be deviating from Ooura by ~1e-4 or worse — not approximating its contract, violating it. And the observed CI failure of +42.34 is worse than even the 1e-3 row, which points at something structural rather than a small numeric deviation.

The missing gate

DspTap has exactly the right test — fft_backend_parity.ForwardMatchesOoura, at N=512 and N=2048, tolerance 5e-6 with "de-risk harness measured 1.3e-7..2.0e-7". But it lives in DspTap, and MuTap's CI never runs it. The only dsptap references in ci.yml compile the Ooura .c files for the branchless-parity job. So the vDSP wrapper at the version MuTap pins (9cbfbca) has no macOS parity gate in the pipeline that reports these failures.

One concrete unchecked-error bug

fft.h's accelerate_real_fft_f32::init() does not check vDSP_create_fftsetup's return:

m_setup = std::shared_ptr<std::remove_pointer_t<FFTSetup>>(
    vDSP_create_fftsetup(static_cast<vDSP_Length>(m_log2n), kFFTRadix2), vDSP_destroy_fftsetup);

That call returns NULL if it cannot allocate, and the result goes straight into vDSP_fft_zrip(m_setup.get(), …) — undefined behaviour. Allocation failure on a loaded CI runner is exactly the shape of thing that is intermittent. Worth fixing on its own merits whether or not it is the cause here; happy to put up the patch.

Revised next steps

1. On a Mac, at MuTap's pinned 9cbfbca: run fft_backend_parity repeatedly (~50 iterations, N=512 and 2048), and tighten the tolerance from 5e-6 toward the measured 2e-7. Given the numbers above, a real cause has to show up as an excursion ≳1e-4 — that should be unmissable once you look repeatedly rather than once.
2. Add that parity test to MuTap's macOS job (or invoke the submodule's tests), since that is the gate that is missing.
3. -DTAP_DSP_FFT_ACCELERATE=OFF remains the confirmatory A/B, with several repeats per configuration.
4. Add the NULL check.

Limits of this

Everything about vDSP itself is inference — I have no macOS here, and all 181 tests pass on Linux under plain, ASan+UBSan, Valgrind and every MALLOC_PERTURB_ pattern. The perturbation model is also a crude stand-in: I injected an alternating ±eps per bin, and a real backend difference with a different error structure could conceivably break the row at a smaller eps than 1e-4. The 250× headroom is large enough that I do not think that changes the conclusion, but it is worth knowing the model is synthetic.

---

Generated by Claude Code