Skip to content

chore(skills): add /add-model and /validate-model commands#4475

Merged
waleedlatif1 merged 1 commit intostagingfrom
waleedlatif1/add-model-skills
May 6, 2026
Merged

chore(skills): add /add-model and /validate-model commands#4475
waleedlatif1 merged 1 commit intostagingfrom
waleedlatif1/add-model-skills

Conversation

@waleedlatif1
Copy link
Copy Markdown
Collaborator

Summary

  • add /add-model skill — adds a new LLM model to apps/sim/providers/models.ts with specs verified against the provider's live API docs (no hallucination)
  • add /validate-model skill — audits a model entry (or all models in a provider) against live docs and reports drift, dead capability flags, and unverified fields

Type of Change

  • Chore

Testing

Tested manually

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel
Copy link
Copy Markdown

vercel Bot commented May 6, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview, Comment May 6, 2026 5:55pm

Request Review

@cursor
Copy link
Copy Markdown

cursor Bot commented May 6, 2026

PR Summary

Low Risk
Low risk: adds new .claude/commands documentation/automation prompts only, with no production code paths or runtime behavior changed.

Overview
Adds two new Claude command definitions, /add-model and /validate-model, to standardize how contributors add or audit entries in apps/sim/providers/models.ts.

The new docs enforce live-docs verification (including two-source pricing checks), outline which capability flags are meaningful via a provider “consumption matrix,” and require structured verification/validation reports plus linting guidance.

Reviewed by Cursor Bugbot for commit fc733bd. Configure here.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 6, 2026

Greptile Summary

This PR adds two new Claude slash commands — /add-model and /validate-model — as Markdown skill files under .claude/commands/. They guide an agent through adding or auditing LLM model entries in apps/sim/providers/models.ts using live web fetches, a capability consumption matrix, and mandatory verification reports.

  • add-model.md: Defines a five-step workflow (live fetch → consumption matrix → pattern match → lint → verification report) with hard rules against hallucination, explicit lint-failure recovery, and a full anti-patterns list.
  • validate-model.md: Mirrors the audit side — reads existing entries, fetches live docs, runs a per-field checklist, and requires human confirmation before any auto-fix is applied; now correctly references add-model.md's canonical provider URL table instead of duplicating it.

Confidence Score: 5/5

Safe to merge — these are instruction-only Markdown skill files with no runtime code paths.

Both files are documentation/prompt files that guide agent behavior; they contain no executable code and require explicit human confirmation before any change to models.ts is applied. The two findings are wording ambiguities in validate-model.md that could confuse an agent at the margin but do not create incorrect data or break any code path.

validate-model.md — the dangling "update the other" sentence and the toolUsageControl checklist item are worth cleaning up before the commands see heavy use.

Important Files Changed

Filename Overview
.claude/commands/add-model.md New skill with thorough hard rules, consumption matrix, lint recovery guidance, and mandatory verification report format; no issues found.
.claude/commands/validate-model.md New validation skill; two minor instruction-quality issues: a dangling "update the other" sentence (the second table no longer exists) and an ambiguous toolUsageControl checklist item that conflicts with the consumption matrix.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A(["/add-model provider model-id"]) --> B[Step 1: Live-fetch provider docs + pricing]
    B --> C[Step 2: Apply Consumption Matrix]
    C --> D[Step 3: Read sibling entries in models.ts]
    D --> E[Step 4: Insert entry + bun run lint]
    E -->|lint fails| F[Fix syntax, re-run lint, note in report]
    F --> E
    E -->|lint passes| G[Step 5: Print Verification Report]
    G -->|any UNVERIFIED rows| H[Ask user to confirm before merging]
    G -->|all verified| I([Done])

    J(["/validate-model provider model-id"]) --> K[Step 1: Read entries from models.ts]
    K --> L[Step 2: Live-fetch provider docs]
    L --> M[Step 3: Re-grep consumption map]
    M --> N[Step 4: Run per-model checklist]
    N --> O[Step 5: Print Validation Report with source URLs]
    O --> P{Offer to fix?}
    P -->|yes| Q[Print diff → get confirmation → edit models.ts → bun run lint → re-check failed rows]
    P -->|no| R([Done])
    Q --> R
Loading

Reviews (2): Last reviewed commit: "chore(skills): add /add-model and /valid..." | Re-trigger Greptile

Comment thread .claude/commands/add-model.md Outdated
@waleedlatif1 waleedlatif1 force-pushed the waleedlatif1/add-model-skills branch from 9aff383 to fc733bd Compare May 6, 2026 17:51
@waleedlatif1
Copy link
Copy Markdown
Collaborator Author

Addressed all three Greptile outside-diff findings in fc733bd:

  1. Lint-failure recovery (add-model.md Step 4) — added explicit instructions: read the error, fix the syntax/typing issue in the entry just written (do not delete the entry — it's the work product), re-run lint, note the fix in a 'Lint adjustments' line in the verification report. Never report done with lint failing.

  2. o3-pro example (validate-model.md line 99) — replaced the misleading parenthetical. The checklist now uses grok-4.3 as the canonical example (xAI docs explicitly state reasoning_effort is not supported) and warns that some always-reasoning models like OpenAI's o-series DO accept the parameter — verify per-model rather than pattern-matching.

  3. Duplicated provider URL table (validate-model.md Step 2) — removed the duplicated table; validate-model.md now references the canonical table in add-model.md Step 1 with a note to update both together if either changes.

@waleedlatif1
Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1
Copy link
Copy Markdown
Collaborator Author

@cursor review

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit fc733bd. Configure here.

@waleedlatif1 waleedlatif1 merged commit ad88859 into staging May 6, 2026
14 checks passed
@waleedlatif1 waleedlatif1 deleted the waleedlatif1/add-model-skills branch May 6, 2026 17:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant