Several reported scores are incompatible with the released test split sizes

[KhanCold]

Hi, thanks for releasing SkillOpt. I found a possible mismatch between the paper results and the currently released dataset splits.

The released code computes the hard score as the mean of per-example scores. For LiveMath, the evaluator is binary exact match, and the released manifest contains 124 test items:

• data/livemathematicianbench_id_split/split_manifest.json
• skillopt/envs/livemathematicianbench/evaluator.py
• skillopt/utils/scoring.py

However, several LiveMath results in Table 1 cannot be obtained from 124 binary examples after rounding to one decimal:

Score	Implied result
22.4	28/125
27.2	34/125
52.0	65/125
31.2	39/125
29.6	37/125
41.6	52/125

These values all fall exactly on a 125-item grid, suggesting that some experiments may have used a 125-item test split rather than the released 124-item split.

I found a similar issue for DocVQA. Its released test split contains 374 items, but the reported score 87.6 is not attainable in a single binary run:

• 327/374 = 87.4%
• 328/374 = 87.7%

There are also some denominator-incompatible cells for OfficeQA and SpreadsheetBench under the released hard-score implementation.

Could you please clarify:

1. Did the paper use an earlier version of the dataset splits?
2. Were the reported values averaged across multiple runs?
3. Were any failed or invalid examples removed before aggregation?
4. Could the exact paper splits and aggregation protocol be released?

Publishing per-example results or the exact result manifests would make the reported table much easier to reproduce.

Related: #108