Fixing Unit Tests#42
Closed
npoulad1 wants to merge 2 commits into
Closed
Conversation
…uffle Fixes a family of wave64-only correctness bugs surfaced by the block scan and device radix sort unit tests on AMDGPU (gfx94x), with no throughput regression on the Genesis G1 RL scaling benchmark (233k env*steps/s, within noise of stock). - block.py: rewrite block inclusive/exclusive scan to a pure-LDS double-buffered Hillis-Steele pass, and rewrite the wave64 radix-rank match phase to a ballot-based CUB MatchAny. Removes the i64 LDS atomic_or + divergent cross-half shuffle hazards that miscompiled on wave64. - runtime.cpp: fix amdgpu_cross_half_shuffle_i32 lane select to key on (target_lane & 32) instead of (target_lane ^ self_lane) & 32. The old form inverted the selection for top-half readers, corrupting every cross-half read on lanes 32..63 (block scans via shuffle_up first diverged at thread 32). - llvm_context.cpp: widen the permlane64 LDS-emulation fences from wavefront to workgroup scope so SIMemoryLegalizer actually emits the s_waitcnt lgkmcnt(0) that orders the LDS round-trip; add a trailing fence to close the across-call write-after-read hazard between back-to-back permlane64 calls. - codegen_amdgpu.cpp: force-inline any range_for body that touches LDS (addrspace(3)) regardless of size. Out-of-line bodies get a separate LDS frame, so cross-thread SharedArray publish/observe (radix histogram bins, block reductions) read stale memory -- the >200-instruction inline gate left the radix-rank body out-of-line and nondeterministically wrong. Detection descends through ConstantExprs since the SharedArray addrspacecast folds to a constant. Co-authored-by: Cursor <cursoragent@cursor.com>
Fix the two pre-commit failures from CI on this branch: - block.py: remove the now-unused ``i64`` import (F401). The ballot-based radix-rank rewrite no longer references the i64 primitive type. - llvm_context.cpp: apply clang-format line-length reflow to the permlane64 LDS-fence comment block. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes a family of wave64-only (gfx94x) correctness bugs surfaced by the block-scan
and device radix-sort unit tests on AMDGPU. It eliminates the entire radix/sort/rank failure class with no
regressions.