The recent paper "SOAP, Muon, and Beyond" (arXiv 2607.20548) points readers to this repository as its released code. Readers who install the library and use Muon's constructor defaults get a materially different optimizer than the paper's experiments on four axes:
| axis |
paper recipe |
library default (muon.py @ HEAD) |
| momentum |
0.9 |
0.95 |
| decoupled weight decay |
0.1 |
0.01 |
| orthogonalization |
16-step PolarExpress |
5-step quintic |
| update-RMS correction |
~0.2 (sqrt((1-b1)/(1+b1))) |
extra_scale_factor=1.0 (off) |
Since the paper itself argues optimizer comparisons are uninterpretable without matched update RMS, a short README note or a checked-in "paper recipe" reference config would prevent exactly the silent config mismatch the paper warns about. The only current breadcrumb is the extra_scale_factor docstring's optional 0.2 mention, which cites arXiv 2502.16982 rather than the recipe. Happy to PR the table above into the README if that's welcome.
The recent paper "SOAP, Muon, and Beyond" (arXiv 2607.20548) points readers to this repository as its released code. Readers who install the library and use
Muon's constructor defaults get a materially different optimizer than the paper's experiments on four axes:muon.py@ HEAD)extra_scale_factor=1.0(off)Since the paper itself argues optimizer comparisons are uninterpretable without matched update RMS, a short README note or a checked-in "paper recipe" reference config would prevent exactly the silent config mismatch the paper warns about. The only current breadcrumb is the
extra_scale_factordocstring's optional 0.2 mention, which cites arXiv 2502.16982 rather than the recipe. Happy to PR the table above into the README if that's welcome.