feat: add standard PPO training with GAE and value critic (Transformers/FSDP) - #256

Merged

tpx818 merged 7 commits into

modelscope:mainfrom

xxyyrr598:add-ppo

Aug 6, 2026

Commits on Aug 4, 2026

refactor: unify PPO epochs via training.num_train_epochs and extract ray NPU resource detection
xxyyrr598
committed
5edbde8
feat(loss): add loss_agg_mode to PPOLoss, default to token-mean
xxyyrr598
committed
1ce486d