Merge colleague evals into scenario_design pipeline

[kcarnold]

What & why

The experiment app had two independently-developed systems validating the same colleague-LLM behavior against the same scenarios (confirmed by git history): the older evalColleague.ts + colleagueEval.ts (5 hardcoded criteria, single-turn probes, gpt-4o-mini judge) and the newer scripts/scenario_design/ pipeline (8 criteria in criteria.md, multi-turn archetype sims, gpt-4o judge). This consolidates onto the pipeline, with criteria.md as the single source of truth.

Changes

• probe.ts (new) — folds the old single-turn adversarial probes in as a pipeline phase. GENERIC_PROBES (scenario-agnostic) live in code; scenario-specific fact questions live in scenarios.json under chat.probes. Each probe is judged against only its mapped criteria (criteria.md slugs), reusing the pipeline's judge.
• Colleague model + reasoning effort are now scenario config (chat.model / chat.reasoningEffort, default gpt-5.5 / low), read by both the live app/api/chat/route.ts and the pipeline, so production and eval test the same thing. (Previously hardcoded gpt-5.2.)
• Latency — every colleague turn records latencyMs + reasoningTokens; probes fail if a response exceeds API_TIMEOUT_MS (the live app's existing 20s abort).
• Deleted evalColleague.ts, colleagueEval.ts, and fix.ts (the latter replaced by pointing a coding agent at the agent-readable *_judgments.json).
• Exported shared helpers from simulate.ts/judge.ts + guarded their main() so importing them doesn't execute them.
• Docs: new scripts/scenario_design/README.md + updated experiment/CLAUDE.md.

Verification

• npm run typecheck + lint clean; no dangling references.
• Live probe run on roomDoubleBooking: 6/7 pass, latencies 1.4–2.9s (gpt-5.5/low is well under the 20s budget — no fallback to gpt-5.4-mini needed).
• The one failing probe is a real finding: on "Anything else I should know?" the colleague volunteers logistical details (Information Gating violation) — a scenario system-prompt issue to address separately.

Note

This changes the live colleague model from gpt-5.2 → gpt-5.5 (intended).

Follow-ups

• Colleague chat: show typing indicator while the LLM is still generating #495 — typing indicator while the LLM is still generating
• Scenario validation: run probes inside a longer conversation (context depth) #496 — run probes inside a longer conversation (context depth)
• Make remaining LLM models configurable (writing-support route, judge/participant) #497 — parameterize remaining hardcoded models (writing-support, judge/participant)

🤖 Generated with Claude Code