test/goroot: synchronize native run expectations - #2264
Conversation
|
The regular pull-request workflows do not run https://github.com/cpunion/llgo/actions/runs/30736831931 It covers macOS and Ubuntu with Go 1.24.11 and Go 1.26.5, split across four shards. Go 1.25.7 is not part of the workflow matrix; its Darwin arm64 and Linux amd64 cases were run manually before opening this PR. |
![fennoai[bot]](https://avatars.githubusercontent.com/in/1924301?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review: retire stale native run xfails
This PR deletes 368 lines from test/goroot/xfail.yaml, removing native run/runoutput expected-failure entries so those goroot cases are now asserted to pass. I verified the mechanics of the deletion:
- The file still parses as valid YAML (309 entries remain).
- No duplicate
(version, platform, directive, case)entries remain. - Every removed
(case, platform)pair is fully removed — no partial-version leftovers. - The
host_skips/timeouts/flakes/xfailssections are all still well-formed; no orphanedreason:lines.
The change is structurally clean and low-risk. Two points worth confirming before merge:
1. finprofiled.go removal — verify the hang is actually fixed (CI-time risk).
Among the removed entries are the darwin/arm64 finprofiled.go xfails (go1.24 + go1.25) whose reason was "LLGo finalizer profiling run hangs beyond the 1m timeout." There is no timeouts: override remaining for this case, so it now runs under the default run-timeout of 1 minute (runner_test.go:49). If the hang has not genuinely been resolved by other work, this case will now consume the full 1m and then hard-fail on every affected darwin/arm64 run — converting a fast expected-failure into a slow hard failure. Please confirm finprofiled.go now completes well under the timeout.
2. This removal is only safe if every removed case now passes.
Per runner_test.go:350, a case still listed as an xfail that succeeds triggers t.Fatalf("unexpected success for xfail case"), but there is no reverse guard: a removed case that still fails will surface as a hard CI failure. This is inherently CI-verified, not statically knowable — I'm flagging it so the removals are validated against a green goroot run on all affected platforms (darwin/arm64 and linux/amd64, go1.24/1.25/1.26) before merge.
Security and documentation reviews found no issues (the test/goroot/README.md references to xfail.yaml are generic and remain accurate).
| directive: run | ||
| case: tinyfin.go | ||
| reason: go1.25 goroot run failure on linux/amd64 | ||
|
|
There was a problem hiding this comment.
Minor: this deletion leaves a stray blank line here — it is now the only blank line in the entire xfails sequence, so the two surrounding groups read as a formatting gap. Since you're already editing this exact region, dropping the blank line would keep the sequence uniformly formatted. Cosmetic / non-blocking (YAML ignores it).
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
LLGo baseline benchmarks
Program measurements
Core language and compiler benchmarks
Compared only with the latest matching platform in the main series. |
|
Interim analysis of the manually dispatched GOROOT matrix:
Full run: https://github.com/cpunion/llgo/actions/runs/30736831931 |
cpunion
changed the title
|
Addressed the full-matrix failures in For the review concern specifically,
The removed xfails were not responsible for any failure in that run. The red jobs were existing XPASS entries, missing Linux expectations, host thread limits, and resource-guard thresholds. The update moves nondeterministic pass/fail cases to A new macOS/Ubuntu × Go 1.24/1.26 × 4-shard validation is running here: https://github.com/cpunion/llgo/actions/runs/30739767737 |
|
Additional check while the updated full matrix is queued: the two macOS shards that had been masked by the old swap-resource cascade contained 18 remaining xfail cases. I reran all of them locally with the same toolchain generations and the updated resource settings:
This closes the only coverage gap left by the first run; no additional stale XPASS or unclassified failure was found. |
|
Merged current
I also checked the completed dedicated matrix run after the earlier update. Its five red jobs were not caused by a removed xfail: four were Local validation:
|
1 participant
Summary
runandrunoutputxfail entries that pass on latestmainflakes, so an XPASS does not fail the matrixThe tested baseline is
2310ff87f1cfa57dda5d4736f98b34ee679bca6d, which is still the latestmain. Scope is limited totest/goroot/xfail.yamland.github/workflows/goroot.yml; there are no compiler/runtime, wasm, DWARF, shard-count, or timeout changes.Removed stale entries
Darwin arm64
38 Go 1.24/1.25 entries across 19 cases, including finalizer, indexing, nil, conversion, fixedbugs, channel, and ordered-map cases.
Linux amd64
append.go,recover3.go,switch.go,tinyfin.go, and five fixedbugs casesindex0.go,nil.go, and seven fixedbugs cases that were also verified across all three Go versionsFull-matrix findings
The first 16-job native run on commit
84d44952bcovered macOS and Ubuntu, Go 1.24.11 and Go 1.26.5, with four shards. No failing subtest matched an xfail removed by this PR for the same platform and Go version.Its failures came from existing XPASS entries, missing platform expectations, two macOS thread-limit cases, a macOS low-swap cascade, and Linux
fixedbugs/bug449.gopeaking at about 4.11 GiB RSS. This update classifies those outcomes directly. Deterministic implementation failures remain xfailed; pass-or-fail outcomes are flakes.Initial run: https://github.com/cpunion/llgo/actions/runs/30736831931
Updated run for commit
5032172ba: https://github.com/cpunion/llgo/actions/runs/30739767737Validation
Removed cases
Configuration and runner
go test ./test/goroot -run "^Test" -skip "^TestGoRootRunCases$" -count=1git diff --check