Skip to content

chore(scaffold): default examples to gpt-5-mini; pre-approve esbuild build in templates#277

Merged
blove merged 3 commits into
mainfrom
blove/scaffold-gpt5-pnpm
Jun 27, 2026
Merged

chore(scaffold): default examples to gpt-5-mini; pre-approve esbuild build in templates#277
blove merged 3 commits into
mainfrom
blove/scaffold-gpt5-pnpm

Conversation

@blove

@blove blove commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Post-smoke-test cleanup (2 of 2 follow-up PRs). Finishes the move off gpt-4o-mini for user-facing examples and fixes the scaffold's non-interactive pnpm install.

  • Model → gpt-5-mini (the project's canonical default — the research template already uses it) in: the basic scaffold template route, root README.md (the raw-LangGraph + With-Dawn + 30-second snippets), @dawn-ai/sdk README, the landing Hero/FeatureRouting snippets, content/templates/AGENTS.md, content/prompts, and the @dawn-ai/evals llmJudge default.
  • Deliberately NOT swept: the known-ids registry, model-validation tests, recorded aimock fixtures (gpt-4o-mini-2024-07-18), api.mdx type-union, frozen superpowers plans, CHANGELOGs, and dated blog posts — those keep gpt-4o intentionally.
  • Scaffold pnpm.onlyBuiltDependencies: ["esbuild"] in both template package.json.template files — fixes ERR_PNPM_IGNORED_BUILDS: esbuild that breaks pnpm install non-interactively in CI/Docker (found while smoke-testing the docker blueprint).

Test Plan

  • pnpm build / typecheck / lint clean
  • @dawn-ai/devkit (9) + create-dawn-ai-app (5) tests pass
  • check-docs + pack:check pass; changeset gate passes
  • grep gpt-4o over the migrated dirs is empty; only the 10 intended files changed

🤖 Generated with Claude Code

…build in templates

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@vercel

vercel Bot commented Jun 27, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
dawnai Ready Ready Preview, Comment Jun 27, 2026 10:42pm

Request Review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated approval: this PR received an intelligent (AI) code review. See the review comments on this PR.

@github-actions

github-actions Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Claude finished @blove's task in 1m 31s —— View job


PR Review: chore(scaffold): default examples to gpt-5-mini; pre-approve esbuild build in templates

  • Read PR diff and changed files
  • Verify model ID validity in known-ids registry
  • Inspect llmJudge behavioral impact
  • Check changeset correctness
  • Review pnpm.onlyBuiltDependencies fix
  • Check for security/type-safety issues

Summary

Clean, well-scoped cleanup PR. No bugs, no security issues, no type-safety problems. Two small observations worth noting:


Observations

1. llmJudge default is a silent runtime behavior change

packages/evals/src/llm-judge.ts:23 changes the fallback model from gpt-4o-mini to gpt-5-mini. This is correct and intentional, but it's worth being explicit: any user calling llmJudge({criteria: "..."}) without passing model will silently start using gpt-5-mini after upgrading. The changeset description covers this, but the changelog entry (when generated) won't surface it as prominently as a minor bump would. Given the fixed group in changeset/config.json, all packages bump together regardless — so the bump level doesn't change user behavior here. Just noting it for awareness.

2. Changeset lists only @dawn-ai/evals, but devkit templates also changed

packages/devkit/templates/app-basic/package.json.template and packages/devkit/templates/app-research/package.json.template both changed. This is fine because all packages are in a fixed group in .changeset/config.json — bumping @dawn-ai/evals triggers a version bump for @dawn-ai/devkit and every other package too. No action needed.


What Looks Good

  • gpt-5-mini is in OPENAI_MODEL_IDS at packages/sdk/src/known-model-ids.ts:9 — the model ID is valid and won't trigger a validation warning.
  • pnpm.onlyBuiltDependencies: ["esbuild"] is correctly placed in both templates and is the right fix for ERR_PNPM_IGNORED_BUILDS in non-interactive installs.
  • The sweep exclusions (aimock recordings, model-validation test fixtures, api.mdx type union, changelogs, dated blog posts) are all well-reasoned — especially keeping gpt-4o-mini-2024-07-18 in recorded fixtures where replay fidelity matters.
  • No secrets, no injection points, no unsafe input handling introduced.

LGTM. The PR does exactly what it says and nothing more.

…iltDependencies

The scaffold templates now declare pnpm.onlyBuiltDependencies; reflect that in
the external expected fixtures (packageJson) and in createExpectedInternalFixture
(internal mode merges onlyBuiltDependencies with the repo overrides block).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Claude finished @blove's task in 1m 50s —— View job


PR Review: chore(scaffold): default examples to gpt-5-mini; pre-approve esbuild build in templates

  • Read PR diff and changed files
  • Verify model ID validity
  • Check pnpm.onlyBuiltDependencies fix correctness
  • Check llmJudge behavioral impact
  • Review test fixture updates
  • Check for security/type-safety issues

Summary

Clean, well-scoped cleanup PR. No bugs, no security issues, no type-safety problems. LGTM.


What Looks Good

Model migration is correct and complete:

  • gpt-5-mini is present at packages/sdk/src/known-model-ids.ts:9 — valid model ID, no validation warnings.
  • All targeted files (app-basic template route, both READMEs, landing components, content/prompts, AGENTS.md, llmJudge) are migrated. grep gpt-4o-mini over migrated dirs returns empty.
  • Intentional exclusions are well-reasoned: api.mdx type union, blog posts, aimock fixtures, and the known-ids registry itself keep gpt-4o-mini correctly.

pnpm.onlyBuiltDependencies: ["esbuild"] fix:

  • Correctly placed in both app-basic/package.json.template and app-research/package.json.template.
  • Test fixtures (basic.expected.json, custom-app-dir.expected.json) updated to match the new template output.
  • run-generated-app.test.ts:481 updated to include onlyBuiltDependencies: ["esbuild"] in the internal fixture builder — the test merges this with pnpm.overrides, so no duplication issue.
  • The handwritten runtime fixture (handwritten-runtime-app/package.json) correctly omits pnpm.onlyBuiltDependencies since it's not a scaffolded app.

Changeset and versioning:

  • Changeset bumps @dawn-ai/evals as patch. All packages are in the same fixed group in .changeset/config.json, so @dawn-ai/devkit and every other package bump together automatically — no gap here.

Minor Observations (non-blocking)

llmJudge default is a silent runtime change: Any caller using llmJudge({ criteria: "..." }) without an explicit model will start using gpt-5-mini after upgrading. This is intentional and correct — just worth being aware of in release notes.

Changeset only names @dawn-ai/evals: Devkit templates also changed, but this is covered by the fixed group bumping all packages together. No action needed.

@github-actions

github-actions Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Claude finished @blove's task in 1m 20s —— View job


PR Review: chore(scaffold): default examples to gpt-5-mini; pre-approve esbuild build in templates

  • Read PR diff and changed files
  • Verify model ID validity in known-ids registry
  • Check pnpm.onlyBuiltDependencies fix correctness
  • Check llmJudge behavioral impact
  • Review test fixture updates
  • Check for security/type-safety issues
  • Post findings

Summary

Clean, well-scoped cleanup PR. No correctness bugs, no security issues, no type-safety problems. LGTM.


What Looks Good

  • gpt-5-mini is a valid model ID — present at packages/sdk/src/known-model-ids.ts:9 in OPENAI_MODEL_IDS. No validation warnings will fire.
  • pnpm.onlyBuiltDependencies: ["esbuild"] is correctly placed in both app-basic and app-research package.json.template files. This is the right fix for ERR_PNPM_IGNORED_BUILDS in non-interactive pnpm installs (CI/Docker).
  • Research template is already consistentapp-research/src/app/research/index.ts and researcher/index.ts already use gpt-5-mini; this PR just catches up the basic template and documentation.
  • Test fixtures properly updatedbasic.expected.json and custom-app-dir.expected.json both gain the pnpm.onlyBuiltDependencies block, and run-generated-app.test.ts adds it to the fixture builder (createExpectedInternalFixture). Runtime fixtures don't reference these, so no other fixtures needed updating.
  • Sweep exclusions are well-reasonedgpt-4o-mini correctly remains in: the known-ids registry (it's a valid model), agent.test.ts test fixtures, api.mdx type union, dated blog posts, and CHANGELOG.

One Note Worth Mentioning

llmJudge default is a silent runtime behavior change (packages/evals/src/llm-judge.ts:23). Any caller using llmJudge({ criteria: "..." }) without passing model will start using gpt-5-mini after upgrading. This is intentional and the right call — just worth calling out that it affects eval cost/behavior for existing users who omit model. The patch bump level is arguably correct since users can always pass model explicitly, but anyone running automated evals without pinning a model will see a change.

No action required — just awareness.

@blove blove merged commit cdcda20 into main Jun 27, 2026
7 checks passed
@blove blove deleted the blove/scaffold-gpt5-pnpm branch June 27, 2026 22:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant