OKLS optimizer - #265

Draft

mkhona-nvidia wants to merge 10 commits into

NVIDIA-NeMo:mainfrom

mkhona-nvidia:okls-optimizer

Commits on Jul 28, 2026

feat: add scaled CANS inverse root utility
mkhona-nvidia
committed
0941b95
chore: update CANS copyright year
mkhona-nvidia
committed
1a36f62
style: use explicit symmetrization scale
mkhona-nvidia
committed
4aa6f49
refactor: broadcast batched CANS identity
mkhona-nvidia
committed
603af79
test: cover batched CANS accuracy
mkhona-nvidia
committed
c25cb3a
test: expand CANS accuracy coverage
mkhona-nvidia
committed
64445c9
refactor: clarify CANS inverse-root function name
mkhona-nvidia
committed
0b63790
feat: add Online KL-Shampoo optimizer
mkhona-nvidia
committed
673fb97
refactor: update OKLS CANS inverse-root call
mkhona-nvidia
committed
379ba77

Commits on Jul 29, 2026

fix: use decoupled weight decay in OKLS
mkhona-nvidia
committed
42cf97f