Rename: synthpanel references → althing#350
Conversation
Companion to DataViking-Tech/Althing#587 (synthpanel → althing). - provider: module/class renamed to althing/AlthingProvider; 'synthpanel' kept as a registry alias and SynthPanelProvider as an import alias; CLI fallback tries the althing binary then the legacy synthpanel one; direct-API import falls back to pre-rename synth_panel installs - leaderboard taxonomy: althing/* display entries plus historical synthpanel/* alias keys so recorded results keep resolving; display names show the new brand for both - config_id: 'synthpanel' accepted as a runner prefix (historical ids); golden test restored to the pre-rename id + new althing golden added - FINDINGS.md: naming note added; generated sections + leaderboard.json regenerated (includes one recomputed SPS per #305 publish policy) - leaderboard-results/, docs/papers, docs/reports, dated audits: untouched (historical record) - core dependency: synthpanel>=0.2.0 → althing>=1.7.0 (merge AFTER althing publishes to PyPI) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
the-data-viking
added
the
vrt-baseline-update
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0779d157e4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return dict(_NULL_COST_FIELDS) | ||
| try: | ||
| from synth_panel.cost import lookup_pricing_by_provider | ||
| from althing.cost import lookup_pricing_by_provider |
There was a problem hiding this comment.
When publishing the retained synthpanel/... result artifacts, the new althing.cost lookup receives the unchanged legacy provider ID and fails to resolve pricing. The generated data demonstrates the regression: product rows in site/src/data/leaderboard.json have their previously populated cost fields replaced with null (for example, lines 89–95), while raw-model costs remain populated. Normalize the legacy prefix to althing/ before lookup, or provide a compatibility fallback, so historical Althing rows are not removed from cost comparisons and Pareto charts.
Useful? React with 👍 / 👎.
| "claude-haiku" in f | ||
| and "synthpanel" in f | ||
| and "althing" in f | ||
| and " t=0.85" in f |
There was a problem hiding this comment.
The experiment now requires althing in every candidate filename, but the inspected leaderboard-results/ directory contains 271 synthpanel filenames and zero althing filenames, consistent with the commit's policy of retaining historical artifact IDs. Consequently find_best_file() finds no optimal-temperature Haiku file and the experiment returns at lines 282–284 without producing results. Accept both the legacy and current runner names in these matchers.
Useful? React with 👍 / 👎.
| # Deprecated alias — kept for external imports that predate the rename. | ||
| SynthPanelProvider = AlthingProvider |
There was a problem hiding this comment.
For integrations that previously used from synthbench.providers.synthpanel import SynthPanelProvider, this alias is unreachable because the commit deletes synthbench.providers.synthpanel and defines it only inside the renamed althing module. Such consumers fail with ModuleNotFoundError before Python can evaluate this alias. Keep a compatibility shim at the old module path that re-exports the renamed class.
Useful? React with 👍 / 👎.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
github-actions
Bot
removed
the
vrt-baseline-update
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2 participants
Companion to DataViking-Tech/Althing#587.
Live code/config
althing.py/AlthingProvider;--provider synthpanelstays working via registry alias; CLI fallback triesalthingthen legacysynthpanelbinary; direct-API import falls back to pre-rename installssynthpanel/*alias keys → recorded results resolve; display names show Althing for both idsconfig_idaccepts the historicalsynthpanelrunner prefix; pre-rename golden hash restored (stability guarantee) + new althing golden addedPreserved as historical record (untouched)
leaderboard-results/,docs/papers/,docs/reports/, dated audits, citation-seeding planRegenerated artifacts
FINDINGS.md generated sections +
site/src/data/leaderboard.json— regeneration also picked up one recomputed SPS (opinionsqa 3-model ensemble 0.835 → 0.877, the #305 recompute-at-publish policy; publish-data flagged the stale committed value). Flagging explicitly since it changes a public leaderboard number.Tests
Full suite: 1159 passed, 0 failed.
Core dependency is now
althing>=1.7.0— do not merge until althing 1.7.0 is live on PyPI (via SynthPanel#587's publish flow). Site rendering changes → VRT baseline dance applies.🤖 Generated with Claude Code