Add create-skill command with validation and documentation#94
Add create-skill command with validation and documentation#94ericelliott wants to merge 9 commits intomainfrom
Conversation
Add a new /create-skill command that guides AI agents through creating skills following the AgentSkills.io specification and SudoLang syntax. Includes size validation, verb-form naming conventions, and a skill template structure. - ai/skills/create-skill.mdc: main skill guide with process, size constraints, authoring guidelines, and example structure - ai/commands/create-skill.md: command entry point - Register command in please.mdc and agent-orchestrator.mdc https://claude.ai/code/session_012PBVMXigqq6Bd4WTUhi4b7
…plan review - Scan ai/ and aidd-custom/ for related skills to avoid overlap - Web search for best practices on the skill topic - Present plan to user for approval before drafting - Run skills-ref validate with symbolic JS size checker - Report warnings when frontmatter > 100 tokens or body > 160 LoC https://claude.ai/code/session_012PBVMXigqq6Bd4WTUhi4b7
Rewrite create-skill.mdc to comply with the official AgentSkills.io spec:
- Skills are directories with SKILL.md (not standalone .mdc files)
- Frontmatter uses name + description (required), plus optional license,
compatibility, metadata, and allowed-tools fields
- Name validation: lowercase alphanumeric + hyphens, 1-64 chars, no
consecutive hyphens, must match parent directory name
- Progressive disclosure: ~100 token metadata, <5000 token instructions
- Output to $projectRoot/aidd-custom/${skillName}/SKILL.md
- Symbolic JS validator includes name validation and tiered size warnings
(160 line guideline, 500 line spec limit, 5000 token guideline)
- Support for optional scripts/, references/, assets/ directories
https://claude.ai/code/session_012PBVMXigqq6Bd4WTUhi4b7
…g mistakes Add explicit constraints section documenting mistakes to avoid when generating new skills: - Must create directory/SKILL.md structure, not standalone files - Must use AgentSkills.io frontmatter fields (name+description), not aidd .mdc fields (alwaysApply, globs) - Must validate name field (lowercase, no consecutive hyphens, match dir) - Must use plain Markdown in SKILL.md body, not SudoLang - Must always run validator and report metrics before success https://claude.ai/code/session_012PBVMXigqq6Bd4WTUhi4b7
Move from flat ai/skills/create-skill.mdc to proper spec-compliant ai/skills/create-skill/SKILL.md directory structure. The skill now eats its own dog food. - AgentSkills.io frontmatter: name + description + metadata - Proper SudoLang usage: type definitions (SkillName, SkillDescription, TokenCount), typed interfaces (Frontmatter, Skill, SizeMetrics, SkillPlan), pipe composition in createSkill process - Reference productmanager.mdc as SudoLang exemplar (not as a skill) - Update command to reference new path https://claude.ai/code/session_012PBVMXigqq6Bd4WTUhi4b7
The AgentSkills.io spec has no alwaysApply equivalent. Document the design decision to use the spec's metadata extension point (metadata.alwaysApply: "true") for AIDD-specific context preloading on project init. - Document AIDD Extensions via metadata section with rationale - Add SkillPlan.alwaysApply field to plan interface - Add RequiredSections interface enforcing documentation quality: When to use, Steps/Process, Examples, Edge cases - Add documentation constraints to spec compliance checks - Update example SKILL.md to show all required sections and metadata.alwaysApply usage https://claude.ai/code/session_012PBVMXigqq6Bd4WTUhi4b7
Add alwaysApply question to gatherRequirements with guidance hints: recommend "yes" only for universally-applicable skills (coding standards, security checks), "no" for task-specific skills that activate on demand. https://claude.ai/code/session_012PBVMXigqq6Bd4WTUhi4b7
…lication Review findings addressed: - Extract inline JS validator to scripts/validate-skill.js as a proper testable module with 4 exports: parseSkillMd, validateName, calculateMetrics, checkThresholds - 16 unit tests written TDD-first with riteway assertions covering all functional requirements (name validation, metrics, thresholds) - Remove duplication: SudoLang Reference section (meta-guidance), TokenCount type (just number), Skill interface (duplicated tree diagram), Progressive Disclosure section (already in AIDD Extensions) - SKILL.md reduced from 279 to 237 lines while adding clarity - Inline 40-line JS blob replaced with reference to tested script https://claude.ai/code/session_012PBVMXigqq6Bd4WTUhi4b7
|
@cursoragent please /review |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| if (metrics.bodyTokens >= 5000) | ||
| warnings.push("Body exceeds 5000 token spec guideline"); | ||
| return warnings; | ||
| }; |
There was a problem hiding this comment.
Hard 500-line limit indistinguishable from soft warnings
Medium Severity
checkThresholds returns the 500-line hard "MUST" violation in the same warnings array as soft "SHOULD" guideline advisories. The spec and SizeMetrics definition clearly distinguish between "should be < 160" (soft) and "must be < 500" (hard), but callers cannot differentiate them without parsing message strings. A spec-violating skill could be treated as passing if warnings are non-blocking.
| if (metrics.bodyTokens >= 5000) | ||
| warnings.push("Body exceeds 5000 token spec guideline"); | ||
| return warnings; | ||
| }; |
There was a problem hiding this comment.
Validator has no CLI entry point despite documented usage
Medium Severity
validate-skill.js only exports functions — it contains no CLI entry point code. But both its JSDoc header (Usage: node validate-skill.js ./path-to-skill-directory) and SKILL.md instruct agents to run it as a CLI command. Executing node validate-skill.js ./path silently exits with no output and no validation, so the constraint "ALWAYS run scripts/validate-skill.js after creating the skill" is effectively a no-op that gives a false sense of validation passing.
Additional Locations (1)
There was a problem hiding this comment.
Pull request overview
This PR adds a new create-skill capability to the ai/ knowledge/automation layer, including a full skill specification (SKILL.md), a validator module with unit tests, and updates to command/rule indexes so the new skill/command is discoverable.
Changes:
- Added
ai/skills/create-skill/SKILL.mddefining the “create skill” process, constraints, and documentation requirements. - Added a JavaScript validator module (
validate-skill.js) plus Vitest unit tests (validate-skill.test.js) for parsing/name/size checks. - Updated
ai/indexes and rules to reference the new skills directory and/create-skillcommand.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| ai/index.md | Links the new skills/ section from the top-level AI index. |
| ai/commands/index.md | Adds create-skill to the commands index. |
| ai/commands/create-skill.md | Defines the /create-skill command and points to the skill guide. |
| ai/rules/please.mdc | Adds /create-skill to the global command list. |
| ai/rules/agent-orchestrator.mdc | Adds a create-skill reference to the orchestrator’s guide list. |
| ai/skills/index.md | Introduces a new skills directory index and links to create-skill/. |
| ai/skills/create-skill/index.md | Adds an index page describing the create-skill skill and subdirs. |
| ai/skills/create-skill/SKILL.md | Adds the full skill specification, workflow, constraints, and example template. |
| ai/skills/create-skill/scripts/index.md | Adds an index for scripts (currently inaccurate per PR contents). |
| ai/skills/create-skill/scripts/validate-skill.js | Adds validator utilities (currently documented as a CLI but not implemented as one). |
| ai/skills/create-skill/scripts/validate-skill.test.js | Adds unit tests for the validator utilities. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| /** | ||
| * Validate an AgentSkills.io SKILL.md file. | ||
| * | ||
| * Usage: node validate-skill.js ./path-to-skill-directory | ||
| */ |
There was a problem hiding this comment.
The header comment claims this file can be run as a CLI (node validate-skill.js ./path-to-skill-directory), but the module only exports functions and never reads argv / the filesystem or prints results. As written, running the command does nothing, which also makes the validation instructions in SKILL.md incorrect. Either implement a main() that loads SKILL.md from the provided directory and exits non-zero on validation failures, or change the usage/docs to reflect module-only usage.
| export const parseSkillMd = (content) => { | ||
| const match = content.match(/^---\n([\s\S]*?)\n---/); | ||
| if (!match) return { body: content, frontmatter: "" }; | ||
| return { | ||
| body: content.slice(match[0].length).trim(), | ||
| frontmatter: match[1], | ||
| }; |
There was a problem hiding this comment.
parseSkillMd() doesn’t actually parse YAML frontmatter (it only extracts the raw frontmatter string), and the regex only matches LF (\n) line endings. On CRLF files (common on Windows), frontmatter won’t be detected and validation will silently operate on the wrong body. Consider using the existing gray-matter dependency (already used elsewhere in the repo) or updating the regex/splitting to handle \r\n and return a structured frontmatter object if the intent is to “parse YAML”.
| warnings.push("Frontmatter exceeds 100 token guideline"); | ||
| if (metrics.bodyLines >= 160) | ||
| warnings.push("Body exceeds 160 line guideline"); | ||
| if (metrics.bodyLines >= 500) | ||
| warnings.push( | ||
| "Body exceeds 500 line spec limit - split into reference files", | ||
| ); | ||
| if (metrics.bodyTokens >= 5000) | ||
| warnings.push("Body exceeds 5000 token spec guideline"); |
There was a problem hiding this comment.
checkThresholds() treats the 500-line “spec limit” the same as soft guidelines by returning it as a warning string (and the messages say “exceeds” even when the value is exactly at the boundary, e.g. 160/500/5000). Since the SKILL.md constraints say the body MUST stay under 500 lines, it would be clearer to distinguish hard-limit errors vs guideline warnings (or include severity) and adjust messages to reflect inclusive thresholds (e.g. “at or above”).
| warnings.push("Frontmatter exceeds 100 token guideline"); | |
| if (metrics.bodyLines >= 160) | |
| warnings.push("Body exceeds 160 line guideline"); | |
| if (metrics.bodyLines >= 500) | |
| warnings.push( | |
| "Body exceeds 500 line spec limit - split into reference files", | |
| ); | |
| if (metrics.bodyTokens >= 5000) | |
| warnings.push("Body exceeds 5000 token spec guideline"); | |
| warnings.push("WARNING: Frontmatter is at or above 100 token guideline"); | |
| if (metrics.bodyLines >= 160) | |
| warnings.push("WARNING: Body is at or above 160 line guideline"); | |
| if (metrics.bodyLines >= 500) | |
| warnings.push( | |
| "ERROR: Body is at or above 500 line spec limit (hard limit) - split into reference files", | |
| ); | |
| if (metrics.bodyTokens >= 5000) | |
| warnings.push("WARNING: Body is at or above 5000 token spec guideline"); |
|
|
||
| This index provides an overview of the contents in this directory. | ||
|
|
||
| *This directory is empty.* |
There was a problem hiding this comment.
This file currently says “This directory is empty.”, but the PR adds validate-skill.js and validate-skill.test.js in this directory. Update the index to list the validator script (and optionally note the test file) so navigation/docs aren’t misleading.
| *This directory is empty.* | |
| - `validate-skill.js`: Script for validating skill definitions and configuration. | |
| - `validate-skill.test.js`: Tests for `validate-skill.js`. |
| ## Validation | ||
|
|
||
| After creating the skill, validate using the unit-tested validator at `scripts/validate-skill.js`: | ||
|
|
||
| ```bash | ||
| # If skills-ref is available: | ||
| skills-ref validate ./path-to-skill-directory | ||
|
|
||
| # Then run the symbolic size/name validator: | ||
| node ai/skills/create-skill/scripts/validate-skill.js ./path-to-skill-directory | ||
| ``` |
There was a problem hiding this comment.
The validation instructions tell users to run node ai/skills/create-skill/scripts/validate-skill.js ./path-to-skill-directory, but validate-skill.js currently has no CLI entrypoint and won’t read the directory or report anything. Either add a real CLI to the script (read SKILL.md, validate name vs directory, print metrics/warnings, set exit code) or change/remove these instructions to avoid giving users a no-op command.
| You are an agent orchestrator. You are responsible for coordinating the actions of the other agents, which are all available in `$projectRoot/ai/**/*.mdc` files: | ||
|
|
||
| Agents { | ||
| please: when user says "please", use this guide for general assistance, logging, committing, and proofing tasks | ||
| stack: when implementing NextJS + React/Redux + Shadcn UI features, use this guide for tech stack guidance and best practices | ||
| productmanager: when planning features, user stories, user journeys, or conducting product discovery, use this guide for building specifications and user journey maps | ||
| tdd: when implementing code changes, use this guide for systematic test-driven development with proper test isolation | ||
| javascript: when writing JavaScript or TypeScript code, use this guide for JavaScript best practices and guidance | ||
| log: when documenting changes, use this guide for creating structured change logs with emoji categorization | ||
| commit: when committing code, use this guide for conventional commit format with proper message structure | ||
| autodux: when building Redux state management, use this guide for creating and transpiling Autodux dux objects | ||
| javascript-io-network-effects: when you need to make network requests or invoke side-effects, use this guide for saga pattern implementation | ||
| ui: when building user interfaces and user experiences, use this guide for beautiful and friendly UI/UX design | ||
| requirements: when writing functional requirements for a user story, use this guide for functional requirement specification | ||
| create-skill: when creating a new agent skill, use this guide for AgentSkills.io specification and SudoLang skill authoring | ||
| } |
There was a problem hiding this comment.
This guide says the orchestrator coordinates agents available in $projectRoot/ai/**/*.mdc, but create-skill is introduced as a skill at ai/skills/create-skill/SKILL.md (not an .mdc). As written, the new create-skill entry doesn’t match the stated discovery mechanism; consider updating the text to include skills (and/or referencing the concrete SKILL.md path) or moving create-skill out of the “Agents” list.
| @@ -0,0 +1,8 @@ | |||
| ## 🛠️ Create Skill | |||
|
|
|||
| Use `ai/skills/create-skill/SKILL.md` to create a new agent skill following the AgentSkills.io specification. | |||
There was a problem hiding this comment.
I don't know if this is needed anymore. We still have this in the skills PR, too, but Cursor, Codex and Claude Code all recognize skills as slash commands automatically. I suspect this will be (or is already) industry standard.
There was a problem hiding this comment.
Should we also add the cabability to check for allowed properties?
const ALLOWED_PROPERTIES = new Set([
"name",
"description",
"license",
"metadata",
]);
function extractFrontmatter(content: string): ValidationResult | string {
if (!content.startsWith("---")) {
return { message: "No YAML frontmatter found", valid: false };
}
const match = content.match(/^---\n(.*?)\n---/s);
if (!match) {
return { message: "Invalid frontmatter format", valid: false };
}
return match[1];
}
function parseFrontmatter(
raw: string,
): ValidationResult | Record<string, unknown> {
try {
const parsed = yaml.load(raw);
if (
typeof parsed !== "object" ||
parsed === null ||
Array.isArray(parsed)
) {
return { message: "Frontmatter must be a YAML dictionary", valid: false };
}
return parsed as Record<string, unknown>;
} catch (e) {
return {
message: `Invalid YAML in frontmatter: ${e}`,
valid: false,
};
}
}| name: SkillName // required | ||
| description: SkillDescription // required | ||
| license // optional | ||
| compatibility: string(1-500) // optional, environment requirements |
There was a problem hiding this comment.
compatibility isn't in the spec. Your Frontmatter interface lists compatibility: string(1-500) as an optional field, but the agentskills.io spec doesn't include it. Should it live under metadata.compatibility?
There was a problem hiding this comment.
compatibility IS defined in the agentskills.io spec.
The full set of recognized frontmatter fields is name (required), description (required), license, compatibility, metadata, and allowed-tools (experimental). All others are ignored.
| license // optional | ||
| compatibility: string(1-500) // optional, environment requirements | ||
| metadata {} // optional, arbitrary key-value pairs (see AIDD Extensions below) | ||
| allowed-tools // optional, space-delimited tool list (experimental) |
There was a problem hiding this comment.
You note it's "experimental" but still list it as a top-level field. The spec doesn't define it either — it's a Claude Code extension. Should we move it under metadata or add a clear disclaimer that it's implementation-specific and not portable?
There was a problem hiding this comment.
The spec DOES define it. Top level of the correct place and this is a useful field to call out non-standard but indispensable tools such as rg (ripgrep) instead of grep|find, etc.
|
|
||
| Create agent skills following the [AgentSkills.io](https://agentskills.io/specification) specification. Skills use **verb form** naming (e.g., `format-code`, `create-skill`). Agents use **noun form** (e.g., `code-formatter`, `skill-creator`). | ||
|
|
||
| Read `ai/rules/sudolang/sudolang-syntax.mdc` for SudoLang syntax conventions. See `ai/rules/productmanager.mdc` for an example of well-written SudoLang (types, interfaces, composition). |
There was a problem hiding this comment.
The spec says:
Keep file references one level deep from SKILL.md. Use relative paths from the skill root.
Even if it's not possible to make these references one level deep (unless symlink?), should we make these paths relative?
There was a problem hiding this comment.
Yes! We absolutely should use reltive paths (NOT symlinks)
| @@ -0,0 +1,236 @@ | |||
| --- | |||
| name: create-skill | |||
There was a problem hiding this comment.
The name is missing the aidd- prefix. We also need to rename the folder.
There was a problem hiding this comment.
Should we add a constraint or instruction that skills should be written in SudoLang?
| 1. Detect the project's formatter configuration (prettier, biome, eslint, etc.) | ||
| 2. Identify files that need formatting | ||
| 3. Apply formatting rules | ||
| 4. Report changes made |
There was a problem hiding this comment.
The process ends at reportMetrics with no validation. Should we add a validation step that checks the generated skill against the spec:
# If skills-ref is available:
skills-ref validate ./path-to-skill-directory
# Then run the local validator:
node ai/skills/create-skill/scripts/validate-skill.js ./path-to-skill-directoryWithout this, frontmatter issues, name constraint violations, and size threshold breaches go undetected until someone tries to use the skill.
3 participants
Overview
This PR introduces the
create-skillcommand and supporting infrastructure for creating new agent skills following the AgentSkills.io specification.Changes
New Files
ai/skills/create-skill/SKILL.md- Comprehensive skill definition with:ai/skills/create-skill/scripts/validate-skill.js- Validator module exporting:parseSkillMd()- Parses YAML frontmatter and body from SKILL.md contentvalidateName()- Validates skill names (1-64 chars, lowercase alphanumeric + hyphens, no leading/trailing/consecutive hyphens, matches directory name)calculateMetrics()- Calculates size metrics (frontmatter tokens, body lines, body tokens)checkThresholds()- Checks against size guidelines (frontmatter < 100 tokens, body < 160 lines / < 500 lines hard limit, < 5000 tokens)ai/skills/create-skill/scripts/validate-skill.test.js- Comprehensive unit tests covering:ai/commands/create-skill.md- Command definition referencing the create-skill skillIndex files - Navigation and documentation:
ai/skills/index.mdai/skills/create-skill/index.mdai/skills/create-skill/scripts/index.mdModified Files
ai/commands/index.md- Added create-skill command entryai/index.md- Added skills directory referenceai/rules/please.mdc- Added /create-skill command referenceai/rules/agent-orchestrator.mdc- Added create-skill referenceKey Features
metadata.alwaysApply)Test Plan
Added comprehensive unit tests in
validate-skill.test.jscovering all validator functions:parseSkillMd(valid frontmatter, no frontmatter)validateName(valid names, uppercase, hyphens, length, empty, directory mismatch)calculateMetricscheckThresholds(within limits, frontmatter, 160-line, 500-line, 5000-token)All tests use the riteway/vitest assertion framework and follow project conventions.
https://claude.ai/code/session_012PBVMXigqq6Bd4WTUhi4b7
Note
Low Risk
Primarily adds new documentation/agent instructions plus a small, self-contained validator with unit tests; minimal impact on existing runtime behavior aside from new command/guide discovery.
Overview
Adds a new
/create-skillcommand andcreate-skillskill to scaffold AgentSkills.io-compliant skills, including a detailedSKILL.mdauthoring/validation workflow and required documentation/size constraints.Introduces a small JS validator (
parseSkillMd,validateName,calculateMetrics,checkThresholds) with Vitest coverage, and wires the new command/skill into theaiindexes plusplease.mdcandagent-orchestrator.mdcso it’s discoverable/usable.Written by Cursor Bugbot for commit 80de38e. This will update automatically on new commits. Configure here.