fix(reward): extract \boxed{} before grading RLPR WebInstruct math (#170)#254
Open
minion1227 wants to merge 1 commit into
Open
fix(reward): extract \boxed{} before grading RLPR WebInstruct math (#170)#254minion1227 wants to merge 1 commit into
minion1227 wants to merge 1 commit into
Conversation
…ini-router#170) _check_rlpr_webinstruct fed the raw candidate prose into math_equal / normalize_math_answer, neither of which strips \boxed{...}. The RLPR format hint tells the worker to box math answers, so every correctly boxed WebInstruct math item was a false negative (scored 0), understating that benchmark's accuracy and corrupting its reward signal. Extract the answer first, mirroring _check_math: cand_math = extract_boxed(cand) or extract_last_number(cand) or cand gold_math = extract_boxed(gold) or gold The 'or cand' fallback preserves free-text answers unchanged; behaviour only diverges when an answer is actually extracted. Distinct from the MMLU-Pro choice path (mini-router#116/mini-router#122). Adds tests/test_reward_rlpr_webinstruct_boxed.py (offline): boxed repro, box-beats-stray-number, number-in-prose fallback, boxed fraction, boxed gold, plus guards that the plain-number/free-text/letter paths are unchanged. 5 cases fail on old code; full root suite 261 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #170
Problem
_check_rlpr_webinstructinsrc/trinity/orchestration/reward.pyscores WebInstruct-verified RLPR items (WebInstruct-verified-val_Avg2). For a math answer it fed the raw candidate prose straight intomath_equal/normalize_math_answer— unlike_check_math, which callsextract_boxed/extract_last_numberfirst. Sincenormalize_math_answerdoesn't strip\boxed{...}, a correctly boxed answer never matched the bare gold and scored 0.0 — even thoughformat_hint("rlpr")explicitly instructs the worker to "use a boxed final answer for math items". The requested output format was the one graded wrong.Impact: every WebInstruct-verified RLPR math item answered in the requested
\boxed{...}format is a false negative — understating accuracy on that benchmark and corrupting the per-benchmark reward signal.Fix
Extract the answer from candidate (and gold) before the math compare, mirroring
_check_math:The
or candfallback is load-bearing: when a candidate has neither a box nor a number (a free-text answer),cand_mathis the whole candidate, so non-math text answers grade exactly as before — the behaviour only diverges when an answer is actually extracted.Distinct from #116 / #122
Those cover the RLPR MMLU-Pro choice path (letters E–J). This is the WebInstruct math branch — a different code path and a different fix. The letter-choice branch here is untouched.
Tests
Adds
tests/test_reward_rlpr_webinstruct_boxed.py(offline, pure stdlib — no network/GPU):1/2; a boxed gold is unwrapped"Paris"), and letter-choice paths are unchanged; empty candidate/gold stillFalse5 of the new cases fail on the old code and pass on the fix; the 10 guard cases pass either way. Full root suite: 261 passed.
ruffclean on the changed lines (the 3 pre-existing findings inreward.pyare on unrelated lines).