From 064e609becb62521b1b9f062864704e1eead6fea Mon Sep 17 00:00:00 2001 From: OpenClaw Date: Fri, 17 Jul 2026 00:02:24 -0500 Subject: [PATCH] fix(site,docs): round-4 buyer-review defects - /docs now serves a real documentation index (links to on-site docs + canonical GitHub docs) instead of falling back to the homepage - Blog re-dated honestly (Updated 2026-07-16) and SPS figures reconciled to the live leaderboard (opinionsqa 0.877, subpop 0.858, globalopinionqa 0.813; generated 2026-07-17) - Homepage evidence section: subpop 0.831 -> 0.858, margins corrected, Pew card refreshed to the 2026-07-16 re-blend run (gaps 20.3/16.0/4.3, JSD 0.035, mean 0.208), gap phrasing no longer reads as an error bound, human/synthetic bars legible without color - README: panel_verdict success-path contradiction resolved (it IS emitted, per src/synth_panel/mcp/server.py), sampling-mode claim tightened, Docker pin -> :1.5.7 / :1.5, Versions table extended through the 1.5.x line, SynthBench section de-staled (0.90 claim, github.io links) - Hero copy: no-API-key claim scoped to run_prompt + <=3-persona polls - Calibration doc: example manifest bumped to 1.5.7, 4-model mix labeled as an example blend (not the leaderboard ensemble) - panel-run doc: --checkpoint-dir default matches actual CLI behavior (off unless passed; resume falls back to ~/.synthpanel/checkpoints) - /mcp: credential store path (~/.config/synthpanel/credentials.json) vs data dir (~/.synthpanel/) stated explicitly - sitemap: /docs added, blog lastmod bumped - Markdown renditions regenerated (scripts/render_site_markdown.py) Co-Authored-By: Claude Fable 5 --- README.md | 31 +- .../synthpanel-vs-commercial-alternatives.md | 47 ++- ...synthpanel-vs-commercial-alternatives.html | 33 +- .../synthpanel-vs-commercial-alternatives.md | 10 +- site/docs/calibration/index.html | 12 +- site/docs/calibration/index.md | 6 +- site/docs/index.html | 332 ++++++++++++++++++ site/docs/index.md | 63 ++++ site/docs/panel-run/index.html | 10 +- site/docs/panel-run/index.md | 6 +- site/index.html | 55 +-- site/index.html.j2 | 55 +-- site/index.md | 20 +- site/mcp/index.html | 9 + site/mcp/index.md | 2 + site/sitemap.xml | 7 +- 16 files changed, 581 insertions(+), 117 deletions(-) create mode 100644 site/docs/index.html create mode 100644 site/docs/index.md diff --git a/README.md b/README.md index 5785aa99..7a2e6c5b 100644 --- a/README.md +++ b/README.md @@ -79,10 +79,10 @@ echoed on every persisted-panel success envelope and every typed error [docs/methodology.md](docs/methodology.md). **Zero-config inside any MCP host that speaks sampling** (Claude Desktop, -Claude Code, Cursor, Windsurf) — drop the config in and run a panel with **no -API key set**. The host runs the model on your behalf, using its own -subscription. Bring your own provider key when you want reproducibility, -ensembles, or larger panels. Personas and instruments are plain YAML; every +Claude Code, Cursor, Windsurf) — drop the config in and run `run_prompt` and +small `run_quick_poll` calls (up to 3 personas) with **no API key set**. The +host runs the model on your behalf, using its own subscription. Full panels, +ensembles, and reproducible model pins need your own provider key. Personas and instruments are plain YAML; every response is schema-validated with per-turn cost telemetry. ## Why @@ -142,8 +142,9 @@ run without one. Read `synthesis.recommendation` for the headline call, `synthesis.disagreements` for dissent, and `rounds[].results[]` for the per-panelist transcript. See [docs/response-contract.md](docs/response-contract.md) for the full v1.0.0 -envelope (note: `panel_verdict` fields like `convergence` and `flags[]` are -defined but not yet emitted on the success path — see the note in the lede). +envelope (as of v1.0.6, `panel_verdict` — `headline`, `convergence`, +`dissent_count`, `flags[]` — is emitted on the success path of the MCP panel +tools; see the note in the lede for the sampling-mode/ensemble exceptions). **Structured polling is the agent default.** For pick-one, Likert, confidence, and tagged-themes questions, pass a bounded @@ -307,7 +308,9 @@ arguments to override. Provider keys are read from environment variables (`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `GOOGLE_API_KEY`/`GEMINI_API_KEY`, `XAI_API_KEY`) — pass whichever your model requires. -Pin to a specific version (`:0.11.0`) in production rather than `:latest`. +Pin to a version tag in production rather than `:latest` — either the exact +release (`:1.5.7`) or the floating major.minor tag (`:1.5`), both published on +every stable release. ## Use as a Python Library @@ -1365,6 +1368,14 @@ modes. | Version | Highlights | |---------|-----------| +| 1.5.x *(current)* | `synthpanel mcp install --host auto` one-liner host registration; four canonical persona packs; deterministic `poll-summary` rollups; MCP auto-picks a fast model for ≥10-persona panels; `results list`/`results show` for the saved-results store; `python -m synthpanel` alias; fail-loud empty attachments and typed `response_schema` enforcement; synthesis routing follows the panel's provider; PyPI Trusted Publishing + auto-tag version-drift guard | +| 1.4.0 | Explicit `cost_estimated_usd` / `cost_actual_usd` (+ `per_model_breakdown`) on `EnsembleResult`, `ModelRunResult`, and `SynthesisResult` — estimate-vs-bill reconciliation without inspecting raw usage | +| 1.3.0 | `trafilatura` moved to the `[full]` extra so bare `pip install synthpanel` works on pyodide/Cloudflare Python Workers (no `lxml` C extension required) | +| 1.2.0 | Pyodide-safe / async-DI surface for `synth_panel.ensemble.synthesize_panel` — drive the judge call through your own async LLM client | +| 1.1.0 | `synth_panel.ensemble` frozen as the supported public API for the ensemble/deliberation core | +| 1.0.6 | Cross-town dogfood friction sweep on the v1.0.x line; `panel_verdict` emitted on the MCP success path; release-dogfood protocol codified | +| 1.0.0 | Frozen MCP contract (`schemas/v1.0.0.json`, `schema_version` echoed on every response and typed error); multimodal attachments (images, URLs, PDFs, inline HTML) with stratified delivery and a URL-fetch security perimeter | +| 0.12.0 | `--best-model-for TOPIC[:DATASET]` SynthBench-driven model picker, `--submit-to-synthbench`, `synthpanel pack calibrate`, six bundled packs deepened 5 → 15 personas | | 0.11.0 | `sp-i2ub` scaled-orchestration epic: panelist-level checkpointing with `--resume ` and auto-checkpoint on SIGINT/SIGTERM (every K=25 panelists), `--max-cost ` mid-run projected-total cost gate that halts gracefully with valid partial JSON, and valid-partial-JSON discipline on every abort path (rate-exhaustion, SIGINT, cost-gate, panelist failure) with `run_invalid: true` + specific `abort_reason` and exit code 2; 6-bug loudness sweep turning silent failures loud across alias parse, synthesis partial payload, MCP `extend_panel`, condition evaluator, orchestrator follow-up exceptions, and the `test_aliases` fixture; auto-tag now fails loudly on unlabeled release PRs; `pip-audit` ignores CVE-2026-3219 in pip 26.0.1 | | 0.10.0 | `synthpanel report` post-hoc Markdown renderer for saved panel results (behind `[report]` extra); inline SynthBench calibration via `panel run --calibrate-against DATASET:QUESTION` with auto-derived `pick_one` schema and `per_question[key].calibration` sub-object wire format; decentralized pack registry — `pack import gh:/` with `--unverified`, `pack search`, `pack list --registry`, 24h cache + offline fallback; optional `version:` field on persona packs with opt-in shadow warning | | 0.9.9 | `--synthesis-strategy=auto` now routes to map-reduce on context overflow instead of hard-failing; OpenRouter alias resolution tightened for sub-1¢ local-table sanity checks; `--personas-merge` warns (or errors via `--personas-merge-on-collision`) on name collisions with bundled packs; version single-sourced from `src/synth_panel/__version__.py` with templated site render | @@ -1388,10 +1399,10 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, testing, and how t ## Benchmarked on SynthBench -synthpanel's ability to produce representative synthetic respondents is independently measured by [SynthBench](https://dataviking-tech.github.io/synthbench/), an open benchmark for synthetic survey quality. +synthpanel's ability to produce representative synthetic respondents is measured by [SynthBench](https://synthbench.org), an open benchmark for synthetic survey quality (a sibling DataViking project, operated by the SynthPanel maintainers). -- **Want proof it works?** Browse the [leaderboard](https://dataviking-tech.github.io/synthbench/leaderboard/) — ensemble blending of 3 models hits SPS 0.90 (90% human parity). -- **Got a great configuration?** [Submit your scores](https://dataviking-tech.github.io/synthbench/submit/) and compare against baselines. +- **Want proof it works?** Browse the [leaderboard](https://synthbench.org) — the 3-model ensemble blend scores SPS 0.877 on opinionsqa, 0.858 on subpop, and 0.813 on globalopinionqa (leaderboard generated 2026-07-17; numbers move as the board recomputes). +- **Got a great configuration?** [Submit your scores](https://synthbench.org) and compare against baselines. - **Contributing an adapter?** Heavy PRs with substantial behavior changes benefit from SynthBench results — reviewers can evaluate empirical quality, not just code. See [docs/adapter-guide.md](docs/adapter-guide.md) for the full adapter workflow. ## License diff --git a/docs/blog/synthpanel-vs-commercial-alternatives.md b/docs/blog/synthpanel-vs-commercial-alternatives.md index 5c0f76da..ea8aea87 100644 --- a/docs/blog/synthpanel-vs-commercial-alternatives.md +++ b/docs/blog/synthpanel-vs-commercial-alternatives.md @@ -1,13 +1,14 @@ # SynthPanel vs Synthetic Users vs FocusPanel.ai — When Open-Source MCP Beats the SaaS -*Published 2026-04-15 · 1,580 words · ~7 min read* +*Published 2026-04-15 · Updated 2026-07-16 · ~7 min read* > **TL;DR — SynthPanel is the open-source, LLM-agnostic alternative to commercial > synthetic-respondent tools like Synthetic Users, FocusPanel.ai, Delve.ai, and > POPJAM. It runs as both a CLI and a 12-tool MCP server you can drop into Claude > Code, Cursor, Windsurf, or Zed. Install with `pip install synthpanel[mcp]`. -> MIT-licensed. Scores 0.90 SPS on the SynthBench human-parity benchmark via -> 3-model ensembles.** +> MIT-licensed. Its 3-model ensemble leads the SynthBench leaderboard at +> SPS 0.86–0.88 (opinionsqa 0.877, subpop 0.858; leaderboard generated +> 2026-07-17 — numbers move as the board recomputes).** If you are a researcher, PM, or founder searching for a way to run synthetic focus groups with AI personas in 2026, the first five results you are going to @@ -74,8 +75,9 @@ give you this — they *are* the vendor. **Honest model comparison.** Run the same instrument through Claude Sonnet 4.6, GPT-4o, and Gemini 2.5 Flash, blend at `--models sonnet:0.34,gpt-4o:0.33,gemini:0.33`, and the framework emits per-model distributions *and* the weighted ensemble. -This is how SynthPanel earns its 0.90 SynthBench score: you cannot get that -number from any single model alone. +This is how SynthPanel tops the SynthBench leaderboard: the blended ensemble +beats every single model on each dataset — by roughly 0.03–0.05 SPS — which +you cannot get from any single model alone. ## 3. Why YAML Instruments Matter @@ -186,20 +188,27 @@ like this — commercial or open-source — is over-claiming. Use synthetic panels to *pre-screen* and *iterate*. Validate the interesting findings with real participants before you publish, launch, or sell. -## 6. Quantitative Proof: SynthBench SPS 0.90 - -Claims are cheap. SynthPanel was the first harness to publish head-to-head -results on [SynthBench](https://synthbench.org), an independent open -benchmark for synthetic-respondent quality. SynthBench computes a -Synthetic-Parity Score (SPS) — the fraction of responses from a synthetic -panel that match the distribution of a real-human control group on the same -instrument. - -A 3-model ensemble of SynthPanel personas — blended via -`synthpanel panel run --models haiku:0.33,gemini:0.33,gpt-4o-mini:0.34 --blend` -— scores **SPS 0.90 on the current SynthBench leaderboard**. That is 90% -human parity, measured by an independent third party, on a benchmark whose -data and scoring code are both open. +## 6. Quantitative Proof: SynthBench SPS 0.86–0.88 + +Claims are cheap. SynthPanel publishes head-to-head results on +[SynthBench](https://synthbench.org), an open, reproducible benchmark for +synthetic-respondent quality. Its data and scoring code are public — though, +in the interest of full disclosure, SynthBench is a sibling DataViking project +operated by the SynthPanel maintainers, not an independent third party. +SynthBench computes a **Synthetic-Parity Score (SPS)** — how closely a +synthetic panel's response distribution matches a real-human control group on +the same instrument. + +A 3-model ensemble of SynthPanel personas — the leaderboard's "SynthPanel +Ensemble (3-model)" entry, an equal-weight blend of Claude Haiku 4.5, +Gemini 2.5 Flash Lite, and GPT-4o-mini, run via +`synthpanel panel run --models haiku:0.33,gemini-flash-lite:0.33,gpt-4o-mini:0.34 --blend` +— leads the current SynthBench leaderboard (generated 2026-07-17) at +**SPS 0.877** on opinionsqa and **0.858** on subpop (0.813 on +globalopinionqa). That is roughly 0.03–0.05 above the best single model on +each dataset, and 0.10–0.11 above the random baseline (~0.71–0.76), on a +benchmark whose data and scoring code are both open. Leaderboard numbers move +as the board recomputes — check the live board for the current figures. Commercial competitors have not, at time of writing, published SPS scores on the same benchmark. If they do, the comparison will be apples-to-apples — and diff --git a/site/blog/synthpanel-vs-commercial-alternatives.html b/site/blog/synthpanel-vs-commercial-alternatives.html index edc9fb8e..878d9a96 100644 --- a/site/blog/synthpanel-vs-commercial-alternatives.html +++ b/site/blog/synthpanel-vs-commercial-alternatives.html @@ -9,7 +9,7 @@ + @@ -68,8 +69,9 @@ "@context": "https://schema.org", "@type": "TechArticle", "headline": "SynthPanel vs Synthetic Users vs FocusPanel.ai — When Open-Source MCP Beats the SaaS", - "description": "SynthPanel is the open-source, LLM-agnostic alternative to commercial synthetic-respondent tools. CLI + 12-tool MCP server, MIT-licensed, tops the SynthBench leaderboard (SPS ~0.88 on opinionsqa/subpop).", + "description": "SynthPanel is the open-source, LLM-agnostic alternative to commercial synthetic-respondent tools. CLI + 12-tool MCP server, MIT-licensed, tops the SynthBench leaderboard (SPS 0.86–0.88 on opinionsqa/subpop).", "datePublished": "2026-04-15", + "dateModified": "2026-07-16", "author": { "@type": "Organization", "name": "DataViking", "url": "https://dataviking.tech" }, "publisher": { "@type": "Organization", "name": "synthpanel", "url": "https://synthpanel.dev" }, "about": [ @@ -133,7 +135,7 @@

- Positioning · 2026-04-15 · ~7 min read + Positioning · 2026-04-15 · Updated 2026-07-16 · ~7 min read

pip install synthpanel[mcp]. MIT-licensed. Its 3-model ensemble leads the SynthBench leaderboard - at SPS ~0.88 (opinionsqa 0.877, subpop 0.879; snapshot - 2026-07-13). + at SPS 0.86–0.88 (opinionsqa 0.877, subpop 0.858; + leaderboard generated 2026-07-17).

@@ -274,7 +276,7 @@

and the framework emits per-model distributions and the weighted ensemble. This is how SynthPanel tops the SynthBench leaderboard: the blended ensemble beats every single model on each - dataset — by roughly 0.03–0.06 SPS — which you cannot get from any + dataset — by roughly 0.03–0.05 SPS — which you cannot get from any single model alone.

@@ -449,7 +451,7 @@

- 6. Quantitative Proof: SynthBench SPS ~0.88 + 6. Quantitative Proof: SynthBench SPS 0.86–0.88

Claims are cheap. SynthPanel publishes head-to-head results on @@ -466,13 +468,16 @@

real-human control group on the same instrument.

- A 3-model ensemble of SynthPanel personas — blended via - synthpanel panel run --models haiku:0.33,gemini:0.33,gpt-4o-mini:0.34 --blend - — leads the current SynthBench leaderboard (generated 2026-07-13) at + A 3-model ensemble of SynthPanel personas — the leaderboard’s + “SynthPanel Ensemble (3-model)” entry, an equal-weight + blend of Claude Haiku 4.5, Gemini 2.5 Flash Lite, and GPT-4o-mini, + run via + synthpanel panel run --models haiku:0.33,gemini-flash-lite:0.33,gpt-4o-mini:0.34 --blend + — leads the current SynthBench leaderboard (generated 2026-07-17) at SPS 0.877 on opinionsqa and - 0.879 on subpop (0.813 on - globalopinionqa). That is roughly 0.03–0.06 above the best single - model on each dataset, and 0.10–0.12 above the random baseline + 0.858 on subpop (0.813 on + globalopinionqa). That is roughly 0.03–0.05 above the best single + model on each dataset, and 0.10–0.11 above the random baseline (~0.71–0.76), on a benchmark whose data and scoring code are both open. Leaderboard numbers move as the board recomputes — check the live board for the current figures. diff --git a/site/blog/synthpanel-vs-commercial-alternatives.md b/site/blog/synthpanel-vs-commercial-alternatives.md index 4192d40e..6b8313b9 100644 --- a/site/blog/synthpanel-vs-commercial-alternatives.md +++ b/site/blog/synthpanel-vs-commercial-alternatives.md @@ -2,13 +2,13 @@ [DataViking](https://dataviking.tech) -Positioning · 2026-04-15 · ~7 min read +Positioning · 2026-04-15 · Updated 2026-07-16 · ~7 min read # SynthPanel vs Synthetic Users vs FocusPanel.ai — When Open-Source MCP Beats the SaaS The open-source, LLM-agnostic synthetic focus group tool — and why it shows up in an AI coding agent's toolbelt but not in a SaaS dashboard. -**TL;DR** — SynthPanel is the open-source, LLM-agnostic alternative to commercial synthetic-respondent tools like *Synthetic Users*, *FocusPanel.ai*, *Delve.ai*, and *POPJAM*. It runs as both a CLI and a 12-tool MCP server you can drop into Claude Code, Cursor, Windsurf, or Zed. Install with `pip install synthpanel[mcp]`. MIT-licensed. Its 3-model ensemble leads the SynthBench leaderboard at **SPS ~0.88** (opinionsqa 0.877, subpop 0.879; snapshot 2026-07-13). +**TL;DR** — SynthPanel is the open-source, LLM-agnostic alternative to commercial synthetic-respondent tools like *Synthetic Users*, *FocusPanel.ai*, *Delve.ai*, and *POPJAM*. It runs as both a CLI and a 12-tool MCP server you can drop into Claude Code, Cursor, Windsurf, or Zed. Install with `pip install synthpanel[mcp]`. MIT-licensed. Its 3-model ensemble leads the SynthBench leaderboard at **SPS 0.86–0.88** (opinionsqa 0.877, subpop 0.858; leaderboard generated 2026-07-17). If you are a researcher, PM, or founder searching for a way to run synthetic focus groups with AI personas in 2026, the first five results you are going to hit are all closed-source SaaS. This article exists so the sixth result — the open-source one — actually shows up in your answer. @@ -48,7 +48,7 @@ Three things fall out of LLM-agnosticism for free: **No vendor lock-in.** If Anthropic's prices change, or OpenAI deprecates a model, or Google introduces a smarter one, nothing in your research corpus breaks. Your personas, instruments, and saved panel results are all plain YAML and JSON. The client swaps underneath them. A SaaS competitor cannot give you this — they *are* the vendor. -**Honest model comparison.** Run the same instrument through Claude Sonnet 4.6, GPT-4o, and Gemini 2.5 Flash, blend at `--models sonnet:0.34,gpt-4o:0.33,gemini:0.33`, and the framework emits per-model distributions *and* the weighted ensemble. This is how SynthPanel tops the SynthBench leaderboard: the blended ensemble beats every single model on each dataset — by roughly 0.03–0.06 SPS — which you cannot get from any single model alone. +**Honest model comparison.** Run the same instrument through Claude Sonnet 4.6, GPT-4o, and Gemini 2.5 Flash, blend at `--models sonnet:0.34,gpt-4o:0.33,gemini:0.33`, and the framework emits per-model distributions *and* the weighted ensemble. This is how SynthPanel tops the SynthBench leaderboard: the blended ensemble beats every single model on each dataset — by roughly 0.03–0.05 SPS — which you cannot get from any single model alone. ## 3. Why YAML Instruments Matter @@ -136,11 +136,11 @@ This list is in the README. Any research harness that does not publish a list li Use synthetic panels to *pre-screen* and *iterate*. Validate the interesting findings with real participants before you publish, launch, or sell. -## 6. Quantitative Proof: SynthBench SPS ~0.88 +## 6. Quantitative Proof: SynthBench SPS 0.86–0.88 Claims are cheap. SynthPanel publishes head-to-head results on [SynthBench](https://synthbench.org), an open, reproducible benchmark for synthetic-respondent quality. Its data and scoring code are public — though, in the interest of full disclosure, SynthBench is a sibling DataViking project operated by the SynthPanel maintainers, not an independent third party. SynthBench computes a **Synthetic-Parity Score (SPS)** — how closely a synthetic panel's response distribution matches a real-human control group on the same instrument. -A 3-model ensemble of SynthPanel personas — blended via `synthpanel panel run --models haiku:0.33,gemini:0.33,gpt-4o-mini:0.34 --blend` — leads the current SynthBench leaderboard (generated 2026-07-13) at **SPS 0.877** on opinionsqa and **0.879** on subpop (0.813 on globalopinionqa). That is roughly 0.03–0.06 above the best single model on each dataset, and 0.10–0.12 above the random baseline (~0.71–0.76), on a benchmark whose data and scoring code are both open. Leaderboard numbers move as the board recomputes — check the live board for the current figures. +A 3-model ensemble of SynthPanel personas — the leaderboard’s “SynthPanel Ensemble (3-model)” entry, an equal-weight blend of Claude Haiku 4.5, Gemini 2.5 Flash Lite, and GPT-4o-mini, run via `synthpanel panel run --models haiku:0.33,gemini-flash-lite:0.33,gpt-4o-mini:0.34 --blend` — leads the current SynthBench leaderboard (generated 2026-07-17) at **SPS 0.877** on opinionsqa and **0.858** on subpop (0.813 on globalopinionqa). That is roughly 0.03–0.05 above the best single model on each dataset, and 0.10–0.11 above the random baseline (~0.71–0.76), on a benchmark whose data and scoring code are both open. Leaderboard numbers move as the board recomputes — check the live board for the current figures. Commercial competitors have not, at time of writing, published SPS scores on the same benchmark. If they do, the comparison will be apples-to-apples — and whoever wins on the leaderboard wins on the leaderboard. diff --git a/site/docs/calibration/index.html b/site/docs/calibration/index.html index 5b47b9a8..80d58794 100644 --- a/site/docs/calibration/index.html +++ b/site/docs/calibration/index.html @@ -170,10 +170,18 @@ models: [haiku:0.30, gemini-flash-lite:0.30, qwen3-plus:0.20, deepseek-v3:0.20] extractor: pick_one:auto-derived panelist_cost_usd: 0.6451 - calibrated_at: 2026-04-26T14:23:00Z - synthpanel_version: 0.11.1 + calibrated_at: 2026-07-16T14:23:00Z + synthpanel_version: 1.5.7 methodology_url: https://synthpanel.dev/docs/calibration + +

+ The 4-model models: mix above is + an example blend, not the SynthBench leaderboard ensemble — + the leaderboard’s “SynthPanel Ensemble (3-model)” + entry is an equal-weight blend of Claude Haiku 4.5, Gemini 2.5 Flash + Lite, and GPT-4o-mini. +

diff --git a/site/docs/calibration/index.md b/site/docs/calibration/index.md index 288ede23..a1ba2863 100644 --- a/site/docs/calibration/index.md +++ b/site/docs/calibration/index.md @@ -33,11 +33,13 @@ calibration: models: [haiku:0.30, gemini-flash-lite:0.30, qwen3-plus:0.20, deepseek-v3:0.20] extractor: pick_one:auto-derived panelist_cost_usd: 0.6451 - calibrated_at: 2026-04-26T14:23:00Z - synthpanel_version: 0.11.1 + calibrated_at: 2026-07-16T14:23:00Z + synthpanel_version: 1.5.7 methodology_url: https://synthpanel.dev/docs/calibration ``` +The 4-model `models:` mix above is an example blend, *not* the SynthBench leaderboard ensemble — the leaderboard’s “SynthPanel Ensemble (3-model)” entry is an equal-weight blend of Claude Haiku 4.5, Gemini 2.5 Flash Lite, and GPT-4o-mini. + ## What JSD means Calibration reports **Jensen-Shannon divergence** between the pack's extracted answer distribution and the dataset's published human distribution. JSD is bounded in `[0, 1]` (using log₂): diff --git a/site/docs/index.html b/site/docs/index.html new file mode 100644 index 00000000..dd66662f --- /dev/null +++ b/site/docs/index.html @@ -0,0 +1,332 @@ + + + + + + + Documentation — synthpanel · Run synthetic focus groups with any LLM + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + +
+

+ + Docs · Index +

+

+ Documentation +

+

+ Everything for running synthetic focus groups with synthpanel — + the CLI reference and MCP server guide live on this site; the full + docs set (methodology, response contract, production operations, and + more) is maintained in the GitHub repo. +

+
+ + +
+

+ On this site +

+
+ +
+

panel run CLI reference

+ → +
+

+ The full flag reference — multi-model ensembles, convergence + auto-stop, checkpointing & resume, cost gates, and SynthBench + calibration. +

+ + +
+

MCP server

+ → +
+

+ Drop-in config for Claude Code, Cursor, Windsurf, Zed, and + Claude Desktop — 12 tools, sampling mode, determinism, + observability, and troubleshooting. +

+ + +
+

Pack calibration

+ → +
+

+ Calibrate a persona pack against a known human baseline with + synthpanel pack calibrate — + JSD interpretation and methodology. +

+ + +
+

Recommended models

+ → +
+

+ SynthBench-validated model picks by use case, and the + --best-model-for auto-selector. +

+ +
+
+ + +
+

+ Full docs set on GitHub +

+

+ The canonical, always-current documentation lives in the repo’s + docs/ directory. Highlights: +

+
    +
  • + agent-quickstart.md + — end-to-end walkthrough: install → verify → dry-run → run → save → emit JSON, on both CLI and MCP surfaces. +
    • +
  • + mcp.md + — full MCP tool schemas, resources, prompt templates, and the model-resolution order. +
    • +
  • + response-contract.md + — the frozen v1.0.0 response envelope, field by field (panel_verdict, typed errors, schema_version). +
    • +
  • + structured-polling.md + — schema-enforced polling patterns (pick-one, Likert, confidence, tagged themes), the agent default. +
    • +
  • + production-operations.md + — running panels in production: checkpointing, cost gates, failure budgets, and observability. +
    • +
  • + methodology.md + — how panels are constructed, synthesized, and where synthetic research is (and isn’t) trustworthy. +
    • +
  • + ensemble.md + — multi-model ensemble blending (--models / --blend) and the public ensemble API. +
    • +
  • + reproducibility.md + — seeds, model pinning, and what “reproducible” can and cannot mean across providers. +
    • +
  • + model-packs.md + — bundled persona and instrument packs, and how to author your own. +
    • +
  • + registry.md + — the decentralized pack registry: pack import gh:<user>/<repo>, search, and verification. +
    • +
  • + migration-v1.md + — migrating from v0.12 to the frozen v1.0.0 contract. +
    • +
  • + synthbench-integration.md + — inline calibration against SynthBench baselines and opt-in leaderboard submission. +
    • +
+
+ + + +
+ + diff --git a/site/docs/index.md b/site/docs/index.md new file mode 100644 index 00000000..26bbda2f --- /dev/null +++ b/site/docs/index.md @@ -0,0 +1,63 @@ +# Documentation — synthpanel · Run synthetic focus groups with any LLM + +[DataViking](https://dataviking.tech) + +Docs · Index + +# Documentation + +Everything for running synthetic focus groups with synthpanel — the CLI reference and MCP server guide live on this site; the full docs set (methodology, response contract, production operations, and more) is maintained in the GitHub repo. + +## On this site + +### [`panel run` CLI reference](/docs/panel-run) + +→ + +The full flag reference — multi-model ensembles, convergence auto-stop, checkpointing & resume, cost gates, and SynthBench calibration. + +### [MCP server](/mcp) + +→ + +Drop-in config for Claude Code, Cursor, Windsurf, Zed, and Claude Desktop — 12 tools, sampling mode, determinism, observability, and troubleshooting. + +### [Pack calibration](/docs/calibration) + +→ + +Calibrate a persona pack against a known human baseline with `synthpanel pack calibrate` — JSD interpretation and methodology. + +### [Recommended models](/recommended-models) + +→ + +SynthBench-validated model picks by use case, and the `--best-model-for` auto-selector. + +## Full docs set on GitHub + +The canonical, always-current documentation lives in the repo’s [`docs/` directory](https://github.com/DataViking-Tech/SynthPanel/tree/main/docs). Highlights: + +- [agent-quickstart.md](https://github.com/DataViking-Tech/SynthPanel/blob/main/docs/agent-quickstart.md) — end-to-end walkthrough: install → verify → dry-run → run → save → emit JSON, on both CLI and MCP surfaces. + +- [mcp.md](https://github.com/DataViking-Tech/SynthPanel/blob/main/docs/mcp.md) — full MCP tool schemas, resources, prompt templates, and the model-resolution order. + +- [response-contract.md](https://github.com/DataViking-Tech/SynthPanel/blob/main/docs/response-contract.md) — the frozen v1.0.0 response envelope, field by field (`panel_verdict`, typed errors, `schema_version`). + +- [structured-polling.md](https://github.com/DataViking-Tech/SynthPanel/blob/main/docs/structured-polling.md) — schema-enforced polling patterns (pick-one, Likert, confidence, tagged themes), the agent default. + +- [production-operations.md](https://github.com/DataViking-Tech/SynthPanel/blob/main/docs/production-operations.md) — running panels in production: checkpointing, cost gates, failure budgets, and observability. + +- [methodology.md](https://github.com/DataViking-Tech/SynthPanel/blob/main/docs/methodology.md) — how panels are constructed, synthesized, and where synthetic research is (and isn’t) trustworthy. + +- [ensemble.md](https://github.com/DataViking-Tech/SynthPanel/blob/main/docs/ensemble.md) — multi-model ensemble blending (`--models` / `--blend`) and the public ensemble API. + +- [reproducibility.md](https://github.com/DataViking-Tech/SynthPanel/blob/main/docs/reproducibility.md) — seeds, model pinning, and what “reproducible” can and cannot mean across providers. + +- [model-packs.md](https://github.com/DataViking-Tech/SynthPanel/blob/main/docs/model-packs.md) — bundled persona and instrument packs, and how to author your own. + +- [registry.md](https://github.com/DataViking-Tech/SynthPanel/blob/main/docs/registry.md) — the decentralized pack registry: `pack import gh:/`, search, and verification. + +- [migration-v1.md](https://github.com/DataViking-Tech/SynthPanel/blob/main/docs/migration-v1.md) — migrating from v0.12 to the frozen v1.0.0 contract. + +- [synthbench-integration.md](https://github.com/DataViking-Tech/SynthPanel/blob/main/docs/synthbench-integration.md) — inline calibration against SynthBench baselines and opt-in leaderboard submission. diff --git a/site/docs/panel-run/index.html b/site/docs/panel-run/index.html index a366e5f3..17cb363a 100644 --- a/site/docs/panel-run/index.html +++ b/site/docs/panel-run/index.html @@ -524,7 +524,7 @@

Starting a checkpointed run

synthpanel panel run \
   --personas panel.yaml \
   --instrument survey.yaml \
-  --checkpoint-dir /tmp/runs \    # opts in; default: ~/.synthpanel/checkpoints
+  --checkpoint-dir /tmp/runs \    # passing the flag opts in to checkpointing
   --checkpoint-every 10           # flush every 10 completed panelists

@@ -540,6 +540,10 @@

Resuming

synthpanel panel run --resume <run-id>

+ Without --checkpoint-dir, resume + looks up the run under + $SYNTHPANEL_CHECKPOINT_ROOT or + ~/.synthpanel/checkpoints. When --personas and --instrument are omitted they are recovered from the checkpoint's saved CLI args. The resume refuses to start if the current @@ -560,8 +564,8 @@

Resuming

--checkpoint-dir PATH - ~/.synthpanel/checkpoints - Directory for per-run snapshots. Setting this opts in to checkpointing. + off (no snapshots) + Directory for per-run snapshots. Setting this opts in to checkpointing; omit it (and --resume) to run without snapshots. A --resume without this flag reads from $SYNTHPANEL_CHECKPOINT_ROOT or ~/.synthpanel/checkpoints. --checkpoint-every N diff --git a/site/docs/panel-run/index.md b/site/docs/panel-run/index.md index 7e5b6dc5..6ce48d45 100644 --- a/site/docs/panel-run/index.md +++ b/site/docs/panel-run/index.md @@ -154,7 +154,7 @@ For long or expensive runs, checkpointing writes per-panelist progress to disk s synthpanel panel run \ --personas panel.yaml \ --instrument survey.yaml \ - --checkpoint-dir /tmp/runs \ # opts in; default: ~/.synthpanel/checkpoints + --checkpoint-dir /tmp/runs \ # passing the flag opts in to checkpointing --checkpoint-every 10 # flush every 10 completed panelists ``` @@ -166,11 +166,11 @@ The run id is printed to stderr. Each checkpoint is written to ` synthpanel panel run --resume ``` -When `--personas` and `--instrument` are omitted they are recovered from the checkpoint's saved CLI args. The resume refuses to start if the current config (model, temperature, questions) does not match the checkpointed config — pass `--allow-drift` to downgrade this to a warning and continue (statistically inconsistent results). +Without `--checkpoint-dir`, resume looks up the run under `$SYNTHPANEL_CHECKPOINT_ROOT` or `~/.synthpanel/checkpoints`. When `--personas` and `--instrument` are omitted they are recovered from the checkpoint's saved CLI args. The resume refuses to start if the current config (model, temperature, questions) does not match the checkpointed config — pass `--allow-drift` to downgrade this to a warning and continue (statistically inconsistent results). | Flag | Default | Description | |---|---|---| -| `--checkpoint-dir PATH` | `~/.synthpanel/checkpoints` | Directory for per-run snapshots. Setting this opts in to checkpointing. | +| `--checkpoint-dir PATH` | off (no snapshots) | Directory for per-run snapshots. Setting this opts in to checkpointing; omit it (and `--resume`) to run without snapshots. A `--resume` without this flag reads from `$SYNTHPANEL_CHECKPOINT_ROOT` or `~/.synthpanel/checkpoints`. | | `--checkpoint-every N` | 25 | Flush a checkpoint every N completed panelists. | | `--resume RUN_ID` | — | Resume a checkpointed run. Skips already-completed panelists. | | `--allow-drift` | off | With `--resume`: downgrade config-mismatch errors to warnings. | diff --git a/site/index.html b/site/index.html index 12c1d632..e148787a 100644 --- a/site/index.html +++ b/site/index.html @@ -135,10 +135,11 @@

Zero-config inside Claude Desktop, Claude Code, Cursor, and other MCP hosts — the host's own model - powers every panelist, so you drop the config in and run a panel with - no API key set. Or bring your - own key (Claude, GPT, Gemini, Grok, local) for reproducibility, - ensembles, and larger panels. Personas and instruments are plain YAML + answers, so you can drop the config in and run one-shot prompts and + quick polls of up to 3 personas with + no API key set. Bring your + own key (Claude, GPT, Gemini, Grok, local) for full panels, + ensembles, and reproducible model pins. Personas and instruments are plain YAML — run from your terminal, a pipeline, or an AI agent's tool call over MCP (Model Context Protocol, the open standard that lets AI tools call external functions). @@ -604,10 +605,13 @@

- Green = real humans: GSS - 2024, NORC — public-use data, weighted shares + For every option, the top bar (green) is the real human share + and the bottom bar (blue) is the synthetic share — the same + values printed as text beside each option label. + Humans: GSS 2024, NORC — + public-use data, weighted shares (wtssnrps). - Blue = synthetic: + Synthetic: gemini-2.5-flash via the SynthBench harness, run 2026-07-15, 30 sampled responses.

@@ -665,46 +669,48 @@

Yes, I believe in this - synthetic within ±13.8 pts + gap: 20.3 pts
-
+
No, I do not believe in this - synthetic within ±15.1 pts + gap: 16.0 pts
-
+
Refused - synthetic within ±1.3 pts + gap: 4.3 pts
-
+

- Bars show |synthetic − human| per answer option, in - percentage points (shorter = closer). + Each bar is the measured option-level gap + |synthetic − human| for that answer option, in percentage + points (shorter bar = closer match). These are observed + differences on this one question, not an error bound.

Measured divergence from the real Pew respondents’ distribution: Jensen–Shannon - divergence 0.023 + divergence 0.035 (0 = identical distributions, 1 = disjoint).

- This item is one of the ensemble’s closest matches by JSD; + This item sits in the run’s closest decile by JSD; across the full 200-question SubPOP evaluation the same - run’s mean JSD is 0.209. View the underlying distributions + run’s mean JSD is 0.208. View the underlying distributions signed-in at synthbench.org, or recompute these exact numbers from the public class="text-emerald-400 underline decoration-emerald-400/40 underline-offset-2 hover:text-emerald-300" >SynthBench harness; the synthetic distribution and per-question JSD are published in - the repo (leaderboard-results/subpop_ensemble_3blend_20260714_171917.json, - run dated 2026-07-14). + the repo (leaderboard-results/subpop_ensemble_3blend_20260716_192413.json, + run dated 2026-07-16).

@@ -745,10 +751,10 @@

class="underline decoration-emerald-400/60 underline-offset-2 hover:text-emerald-200" >leaderboard - (generated 2026-07-15), the 3-model ensemble scores SPS 0.877 on - opinionsqa, 0.831 on subpop, and 0.813 on globalopinionqa — roughly - 0.01–0.05 above the best single model on each dataset and - 0.07–0.11 above the random baseline (~0.71–0.76). Leaderboard + (generated 2026-07-17), the 3-model ensemble scores SPS 0.877 on + opinionsqa, 0.858 on subpop, and 0.813 on globalopinionqa — roughly + 0.03–0.05 above the best single model on each dataset and + 0.10–0.11 above the random baseline (~0.71–0.76). Leaderboard numbers move as the board recomputes.

@@ -767,6 +773,7 @@

>SynthPanel vs Synthetic Users vs FocusPanel.ai — when open-source MCP beats the SaaS → + · updated diff --git a/site/index.html.j2 b/site/index.html.j2 index 08b4bba3..e62c3d2c 100644 --- a/site/index.html.j2 +++ b/site/index.html.j2 @@ -135,10 +135,11 @@

Zero-config inside Claude Desktop, Claude Code, Cursor, and other MCP hosts — the host's own model - powers every panelist, so you drop the config in and run a panel with - no API key set. Or bring your - own key (Claude, GPT, Gemini, Grok, local) for reproducibility, - ensembles, and larger panels. Personas and instruments are plain YAML + answers, so you can drop the config in and run one-shot prompts and + quick polls of up to 3 personas with + no API key set. Bring your + own key (Claude, GPT, Gemini, Grok, local) for full panels, + ensembles, and reproducible model pins. Personas and instruments are plain YAML — run from your terminal, a pipeline, or an AI agent's tool call over MCP (Model Context Protocol, the open standard that lets AI tools call external functions). @@ -604,10 +605,13 @@ synthpanel report <result-id> -o report.md

- Green = real humans: GSS - 2024, NORC — public-use data, weighted shares + For every option, the top bar (green) is the real human share + and the bottom bar (blue) is the synthetic share — the same + values printed as text beside each option label. + Humans: GSS 2024, NORC — + public-use data, weighted shares (wtssnrps). - Blue = synthetic: + Synthetic: gemini-2.5-flash via the SynthBench harness, run 2026-07-15, 30 sampled responses.

@@ -665,46 +669,48 @@ synthpanel report <result-id> -o report.md
Yes, I believe in this - synthetic within ±13.8 pts + gap: 20.3 pts
-
+
No, I do not believe in this - synthetic within ±15.1 pts + gap: 16.0 pts
-
+
Refused - synthetic within ±1.3 pts + gap: 4.3 pts
-
+

- Bars show |synthetic − human| per answer option, in - percentage points (shorter = closer). + Each bar is the measured option-level gap + |synthetic − human| for that answer option, in percentage + points (shorter bar = closer match). These are observed + differences on this one question, not an error bound.

Measured divergence from the real Pew respondents’ distribution: Jensen–Shannon - divergence 0.023 + divergence 0.035 (0 = identical distributions, 1 = disjoint).

- This item is one of the ensemble’s closest matches by JSD; + This item sits in the run’s closest decile by JSD; across the full 200-question SubPOP evaluation the same - run’s mean JSD is 0.209. View the underlying distributions + run’s mean JSD is 0.208. View the underlying distributions signed-in at synthbench.org, or recompute these exact numbers from the public class="text-emerald-400 underline decoration-emerald-400/40 underline-offset-2 hover:text-emerald-300" >SynthBench harness; the synthetic distribution and per-question JSD are published in - the repo (leaderboard-results/subpop_ensemble_3blend_20260714_171917.json, - run dated 2026-07-14). + the repo (leaderboard-results/subpop_ensemble_3blend_20260716_192413.json, + run dated 2026-07-16).

@@ -745,10 +751,10 @@ synthpanel report <result-id> -o report.md class="underline decoration-emerald-400/60 underline-offset-2 hover:text-emerald-200" >leaderboard - (generated 2026-07-15), the 3-model ensemble scores SPS 0.877 on - opinionsqa, 0.831 on subpop, and 0.813 on globalopinionqa — roughly - 0.01–0.05 above the best single model on each dataset and - 0.07–0.11 above the random baseline (~0.71–0.76). Leaderboard + (generated 2026-07-17), the 3-model ensemble scores SPS 0.877 on + opinionsqa, 0.858 on subpop, and 0.813 on globalopinionqa — roughly + 0.03–0.05 above the best single model on each dataset and + 0.10–0.11 above the random baseline (~0.71–0.76). Leaderboard numbers move as the board recomputes.

@@ -767,6 +773,7 @@ synthpanel report <result-id> -o report.md >SynthPanel vs Synthetic Users vs FocusPanel.ai — when open-source MCP beats the SaaS → + · updated diff --git a/site/index.md b/site/index.md index 40eb6857..2b5b601d 100644 --- a/site/index.md +++ b/site/index.md @@ -6,7 +6,7 @@ Run synthetic focus groups with any LLM. -Zero-config inside Claude Desktop, Claude Code, Cursor, and other MCP hosts — the host's own model powers every panelist, so you drop the config in and run a panel with **no API key set**. Or bring your own key (Claude, GPT, Gemini, Grok, local) for reproducibility, ensembles, and larger panels. Personas and instruments are plain YAML — run from your terminal, a pipeline, or an AI agent's tool call over MCP (Model Context Protocol, the open standard that lets AI tools call external functions). +Zero-config inside Claude Desktop, Claude Code, Cursor, and other MCP hosts — the host's own model answers, so you can drop the config in and run one-shot prompts and quick polls of up to 3 personas with **no API key set**. Bring your own key (Claude, GPT, Gemini, Grok, local) for full panels, ensembles, and reproducible model pins. Personas and instruments are plain YAML — run from your terminal, a pipeline, or an AI agent's tool call over MCP (Model Context Protocol, the open standard that lets AI tools call external functions). $ pip install synthpanel @@ -182,7 +182,7 @@ Dull humans 5.7% · synthetic 0.0% Don’t know humans 0.7% · synthetic 0.0% -Green = real humans: GSS 2024, NORC — public-use data, weighted shares (`wtssnrps`). Blue = synthetic: gemini-2.5-flash via the SynthBench harness, run 2026-07-15, 30 sampled responses. +For every option, the top bar (green) is the real human share and the bottom bar (blue) is the synthetic share — the same values printed as text beside each option label. Humans: GSS 2024, NORC — public-use data, weighted shares (`wtssnrps`). Synthetic: gemini-2.5-flash via the SynthBench harness, run 2026-07-15, 30 sampled responses. Measured divergence between the two distributions: Jensen–Shannon divergence 0.046 (0 = identical distributions, 1 = disjoint). @@ -194,20 +194,20 @@ Real Pew American Trends Panel question (Wave 96, via SubPOP item `BELIEVE_a_W96 Pew’s response data is license-gated (CC-BY-NC-SA), so this page shows only the *gap* between the SynthPanel 3-model ensemble (90 sampled responses) and the real Pew respondents — not the survey values themselves: -Yes, I believe in this synthetic within ±13.8 pts +Yes, I believe in this gap: 20.3 pts -No, I do not believe in this synthetic within ±15.1 pts +No, I do not believe in this gap: 16.0 pts -Refused synthetic within ±1.3 pts +Refused gap: 4.3 pts -Bars show |synthetic − human| per answer option, in percentage points (shorter = closer). +Each bar is the measured option-level gap |synthetic − human| for that answer option, in percentage points (shorter bar = closer match). These are observed differences on this one question, not an error bound. -Measured divergence from the real Pew respondents’ distribution: Jensen–Shannon divergence 0.023 (0 = identical distributions, 1 = disjoint). +Measured divergence from the real Pew respondents’ distribution: Jensen–Shannon divergence 0.035 (0 = identical distributions, 1 = disjoint). -This item is one of the ensemble’s closest matches by JSD; across the full 200-question SubPOP evaluation the same run’s mean JSD is 0.209. View the underlying distributions signed-in at synthbench.org, or recompute these exact numbers from the public [SubPOP dataset](https://huggingface.co/datasets/jjssuh/subpop) and the open [SynthBench harness](https://github.com/DataViking-Tech/SynthBench); the synthetic distribution and per-question JSD are published in the repo (`leaderboard-results/subpop_ensemble_3blend_20260714_171917.json`, run dated 2026-07-14). +This item sits in the run’s closest decile by JSD; across the full 200-question SubPOP evaluation the same run’s mean JSD is 0.208. View the underlying distributions signed-in at synthbench.org, or recompute these exact numbers from the public [SubPOP dataset](https://huggingface.co/datasets/jjssuh/subpop) and the open [SynthBench harness](https://github.com/DataViking-Tech/SynthBench); the synthetic distribution and per-question JSD are published in the repo (`leaderboard-results/subpop_ensemble_3blend_20260716_192413.json`, run dated 2026-07-16). -Powers the [SynthBench](https://synthbench.org) open benchmark — an open, reproducible evaluation of synthetic-respondent quality (public data + scoring code), operated by the SynthPanel maintainers rather than an independent third party. On the current [leaderboard](https://synthbench.org/data/leaderboard.json) (generated 2026-07-15), the 3-model ensemble scores SPS 0.877 on opinionsqa, 0.831 on subpop, and 0.813 on globalopinionqa — roughly 0.01–0.05 above the best single model on each dataset and 0.07–0.11 above the random baseline (~0.71–0.76). Leaderboard numbers move as the board recomputes. +Powers the [SynthBench](https://synthbench.org) open benchmark — an open, reproducible evaluation of synthetic-respondent quality (public data + scoring code), operated by the SynthPanel maintainers rather than an independent third party. On the current [leaderboard](https://synthbench.org/data/leaderboard.json) (generated 2026-07-17), the 3-model ensemble scores SPS 0.877 on opinionsqa, 0.858 on subpop, and 0.813 on globalopinionqa — roughly 0.03–0.05 above the best single model on each dataset and 0.10–0.11 above the random baseline (~0.71–0.76). Leaderboard numbers move as the board recomputes. ## Further reading -- [SynthPanel vs Synthetic Users vs FocusPanel.ai — when open-source MCP beats the SaaS →](/blog/synthpanel-vs-commercial-alternatives.html) 2026-04-15 +- [SynthPanel vs Synthetic Users vs FocusPanel.ai — when open-source MCP beats the SaaS →](/blog/synthpanel-vs-commercial-alternatives.html) 2026-04-15 · updated 2026-07-16 diff --git a/site/mcp/index.html b/site/mcp/index.html index 91d59848..1d0dd068 100644 --- a/site/mcp/index.html +++ b/site/mcp/index.html @@ -1480,6 +1480,15 @@

├── packs/instruments/ # Installed instrument packs (YAML) └── results/ # Panel results (JSON) + session data +

+ Note the two distinct locations: API keys saved by + synthpanel login live in the + credential store at + ~/.config/synthpanel/credentials.json + (mode 0600), while run data and + packs live under ~/.synthpanel/ + as shown above. +

diff --git a/site/mcp/index.md b/site/mcp/index.md index 02e3fcd5..ea4bdd57 100644 --- a/site/mcp/index.md +++ b/site/mcp/index.md @@ -368,6 +368,8 @@ Panel results, persona packs, and instrument packs are stored under `~/.synthpan └── results/ # Panel results (JSON) + session data ``` +Note the two distinct locations: API keys saved by `synthpanel login` live in the credential store at `~/.config/synthpanel/credentials.json` (mode `0600`), while run data and packs live under `~/.synthpanel/` as shown above. + ## Troubleshooting `command not found: synthpanel` The MCP host can't find the binary on its `PATH`. MCP subprocesses inherit the host's environment, not your shell's. Fix by installing globally (`pip install "synthpanel[mcp]"`), pointing `command` at an absolute path (e.g. `/Users/you/.venv/bin/synthpanel`), or running through `uvx` (`"command": "uvx", "args": ["--from", "synthpanel[mcp]", "synthpanel", "mcp-serve"]`). Claude Desktop on macOS is especially strict — use an absolute path or `uvx`. diff --git a/site/sitemap.xml b/site/sitemap.xml index fa6d5d1d..d0afd195 100644 --- a/site/sitemap.xml +++ b/site/sitemap.xml @@ -10,6 +10,11 @@ weekly 0.9 + + https://synthpanel.dev/docs + monthly + 0.8 + https://synthpanel.dev/docs/panel-run monthly @@ -27,7 +32,7 @@ https://synthpanel.dev/blog/synthpanel-vs-commercial-alternatives.html - 2026-04-15 + 2026-07-16 monthly 0.8