Skip to content

[AMDGPU] Keep fdiv correctly rounded under fast_math (fixes modulo bug #749)#773

Open
ZJLi2013 wants to merge 1 commit into
Genesis-Embodied-AI:mainfrom
ZJLi2013:amd-fdiv-correctly-rounded
Open

[AMDGPU] Keep fdiv correctly rounded under fast_math (fixes modulo bug #749)#773
ZJLi2013 wants to merge 1 commit into
Genesis-Embodied-AI:mainfrom
ZJLi2013:amd-fdiv-correctly-rounded

Conversation

@ZJLi2013

Copy link
Copy Markdown

Fixes: #749

Summary

On AMDGPU, float % (int loaded from a field) returns the unreduced dividend instead of the remainder (reported in #749 as the "AMDGPU modulo bug"). The root cause is not modulo-specific: AMDGPU float division is being lowered to an approximate reciprocal, so exactly-divisible cases like 180.0 / 180.0 return 0.99999994. Any floor(a / b) — spatial hashing, grid indexing (support_res), bucketization — silently lands one cell off. Modulo just makes it obvious because it is lowered as a - floor(a/b)*b.

Fix: clear the afn (ApproxFunc) fast-math flag for the emitted FDiv so division stays correctly rounded. afn is meant for transcendentals, not basic arithmetic; the change is a no-op on CPU/CUDA (their fdiv is already correctly rounded).

Root cause

compile_config.fast_math defaults to true. TaskCodeGenLLVM::initialize_context() sets global fast-math flags on the IRBuilder, including setApproxFunc():

llvm::FastMathFlags fast_flags;
fast_flags.setNoInfs();
fast_flags.setNoSignedZeros();
fast_flags.setAllowReassoc();
fast_flags.setApproxFunc();          // afn
builder->setFastMathFlags(fast_flags);

Every instruction, including BinaryOpType::divCreateFDiv, inherits afn. The AMDGPU backend uses afn on fdiv to select the fast v_rcp_f32-based reciprocal path (~2.5 ULP) instead of a correctly-rounded division. x86 and NVPTX ignore afn for fdiv (their hardware divide is correctly rounded regardless), so the bug is AMDGPU-only.

Why float % literal looks fine but float % field does not: a constant divisor is folded at compile time (constant_fold.cpp, exact), while a runtime field value takes the fdiv path and hits the approximate reciprocal.

Trace of floor(180.0) % mod[None] (mod=180), storing every intermediate as f32:

step AMDGPU (before) CPU
x / mf (float / field-int) 0.99999994 1.0
floor(x / mf) 0.0 1.0
floor(x / const_180) 1.0 1.0
x - floor(x/mf)*mf (mod) 180.0 0.0

Setting fast_math=False makes floor(x/mf) = 1.0 and mod = 0 on AMDGPU, confirming the trigger is afn-on-fdiv.

Fix

In TaskCodeGenLLVM::visit(BinaryOpStmt) for BinaryOpType::div (real branch), clear afn for the emitted FDiv via a FastMathFlagGuard, leaving the builder's other fast-math flags untouched. mod and floordiv are lowered to BinaryOpType::div in demote_operations.cpp, so this one spot covers float / int, floordiv, and % alike.

Two non-obvious points worth flagging for review:

  • The flag must be set on the builder (so it applies as CreateFDiv builds the instruction). A post-hoc inst->setFastMathFlags(...) after CreateFDiv did not take effect on AMDGPU in practice.
  • Emitted AMDGPU IR for 180.0 / field, before → after:
- %12 = fdiv reassoc ninf nsz afn float 1.800000e+02, %11   ; approximate -> 0.99999994
+ %12 = fdiv reassoc ninf nsz     float 1.800000e+02, %11   ; correctly rounded -> 1.0

Testing

Standalone quadrants reproducer from #749 (qd_modulo_bug.py, cases A–F) plus a step-by-step intermediate-value diagnostic. Built from source and validated on:

  • R9700 (gfx1201 / RDNA4), quadrants 1.1.1 (main @ 2e2b43b40)
  • MI300 (gfx942 / CDNA3), quadrants 1.0.3
case before (amdgpu) after (amdgpu) cpu
float % int-literal 0 0 0
float % int-from-field 180 0 0
int(float) % field 0 0 0
floor(180.0 / field_180) 0.0 1.0 1.0

No change to non-floor float results; fast_math approximation for transcendentals (sin/cos/exp/…) is retained.

Notes / scope

  • The fix is in the shared LLVM codegen, not AMDGPU-specific, but is only observable on AMDGPU (CPU/CUDA fdiv already ignores afn). Clearing the flag keeps behavior consistent across backends.
  • Independent of the shuffle correctness / DPP work on other AMD branches; this is a pure arithmetic-precision fix.

Under fast_math the global IRBuilder fast-math flags include afn
(ApproxFunc). The AMDGPU backend uses that to lower fdiv to an
approximate v_rcp_f32 reciprocal (~2.5 ULP), so exact divisions such
as 180.0/180.0 return 0.99999994. This breaks floor(a/b) and the
a - floor(a/b)*b modulo expansion, surfacing as the "AMDGPU modulo
bug" where float % (runtime int) returns the unreduced dividend.

Clear afn on the emitted FDiv so division stays correctly rounded
(matching CPU/CUDA, where it is a no-op). afn is intended for
transcendentals, not basic arithmetic.

Fixes: quadrants#749
Co-authored-by: Cursor <cursoragent@cursor.com>
@ZJLi2013 ZJLi2013 changed the title Keep fdiv correctly rounded under fast_math (fixes modulo bug #749) [AMDGPU] Keep fdiv correctly rounded under fast_math (fixes modulo bug #749) Jul 10, 2026
@hughperkins

Copy link
Copy Markdown
Collaborator

Could we gate this change on using AMD please? That would make me feel more comfortable.

@hughperkins hughperkins added the awaiting-contributor-action awaiting-contributor-action label Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting-contributor-action awaiting-contributor-action

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AMDGPU modulo bug

2 participants