Blazor time to first UI regression: blazorwasmbenchmark

[pr-benchmarks]

Scenario	Environment	Date	Old (ms)	New (ms)	Change	Dependencies
blazorwasmbenchmark [Fixed]		07/28/2026 23:41:02	382	387	1.31 % (5) 👎	Changes Name Version Diff

PowerBI Dashboard

[oroztocil]

@LoopedBard3 Do you know why the Dependencies column is empty? How can we check the diff related to the regression?

[javiercn]

< 5% is likely noise.

[ilonatommy]

The point is: why do we report <5%, maybe we should have some threshold? And even if not, the missing column is concerning.

[LoopedBard3]

The empty Dependencies column isn't a rendering bug — the blazorwasmbenchmark scenario just has no dependency/version data for the regression bot to diff.

Unlike the server scenarios (e.g. #67745), this one builds from a Dockerfile (src/Components/benchmarkapps/Wasm.Performance/dockerfile). crank normally records the framework versions (aspNetCoreVersion / netCoreAppVersion) by attaching to the process via the Microsoft.Crank.EventSources EventSource, but that channel isn't available for container-based jobs — so the stored result carries neither a scanned dependency list nor those versions. With nothing to compare, RegressionBot.ComputeChanges → CreateDotNetDependencies emits no rows, leaving the table empty.

Fix: #68119 makes the Wasm.Performance Driver emit aspNetCoreVersion/netCoreAppVersion over the same stdout #StartJobStatistics channel it already uses for its other measurements. Once that merges, the bot can synthesize the Microsoft.AspNetCore.App / Microsoft.NETCore.App rows (with dotnet/dotnet compare links) just like the other scenarios. It''ll take a couple of post-merge nightly runs before a diff can actually show up.

[LoopedBard3]

On the <5% point: the time-to-first-ui probe uses a standard-deviation threshold, not a percentage one. From regressions.blazor.config.yml:

- path: jobs.application.results['blazorwasm/time-to-first-ui']
  threshold: 2

unit defaults to StDev (per crank''s regressionbot.schema.json — unit is "StDev" | "Percent" | "Absolute", default "StDev"; threshold default 1), so a regression is filed when the value shifts by >= 2x the standard deviation of the trailing 7 runs and stays shifted for 2 consecutive runs. The actual comparison per unit lives in RegressionBot/Program.cs (switch (probe.Unit)): StDev -> |change| >= threshold * stdev, Percent -> |change| >= average * (threshold/100), Absolute -> |change| >= threshold.

time-to-first-ui is very stable, so its stdev is small — which is exactly why a 5 ms / 1.31% move clears 2σ and gets reported. It''s statistically real, just not necessarily meaningful.

To suppress sub-5% noise we have a couple of options:

• Switch the probe to a percentage floor — unit: Percent with threshold: 5 — so it only fires when the change is >=5% of the average (this is exactly what the download-size probe already does with unit: Percent). It keeps the "2 consecutive runs" persistence check but drops the statistical-significance gating.
• Or just raise the σ threshold (e.g. 2 -> 3) to require a larger, more consistent shift — still statistics-based, but no hard 5% floor.

crank doesn''t currently support requiring both σ and % in one probe (multiple probes are OR''d, which would make it more sensitive, not less), so a combined gate would need a crank change. If we agree <5% is noise here, my inclination is unit: Percent, threshold: 5 for time-to-first-ui.

[ilonatommy]

crank doesn''t currently support requiring both σ and % in one probe (multiple probes are OR''d, which would make it more sensitive, not less), so a combined gate would need a crank change. If we agree <5% is noise here, my inclination is unit: Percent, threshold: 5 for time-to-first-ui.

2σ sound sensible as a general rule for all the scenarios, we don't want to loose the signal about real regressions. I think it's better to keep all the reporting rules aligned across scenarios, it will be easier to reason about the reports in the future.

It looks like in this very case the report was really noise (it already got an update from the new measurement acknowledging the regression is gone). It's fine to have the regressions over-reported than under-reported.

Will #68119 also fix other missing data? In comparison to #67745 this report is missing deviation columns.

[LoopedBard3]

On the missing Deviation/StDev columns — that''s a separate, purely cosmetic cause from the Dependencies one, so #68119 doesn''t touch it.

The reports render from Liquid templates in regressions.blazor.config.yml, and the first-ui template (used for blazorwasm/time-to-first-ui) simply never defined those two columns. Its header was:

| Scenario | Environment | Date | Old (ms) | New (ms) | Change | Dependencies |

whereas the rps template behind #67745 has | Change | Deviation | StDev | Dependencies |. It looks like first-ui was modeled on the start-time template, which also omits them — even though the sibling download-size template in the same Blazor config already includes them.

The data itself was never missing — regression.StandardDeviation is always computed by the RegressionBot, and deviation is just Change / StandardDeviation. So it''s a template-only fix: aspnet/Benchmarks#2194 adds both columns to first-ui, bringing it in line with the other regression report templates. Future time-to-first-ui regressions will then show Deviation (in σ) and StDev alongside the change.