-
Notifications
You must be signed in to change notification settings - Fork 15
Add create-skill command with validation and documentation #94
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
0dcca3f
7e1e8b7
d158301
c8cd8e3
2d82edf
2601c61
15bf99e
f98bbdc
80de38e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| ## 🛠️ Create Skill | ||
|
|
||
| Use `ai/skills/create-skill/SKILL.md` to create a new agent skill following the AgentSkills.io specification. | ||
|
|
||
| Constraints { | ||
| Begin by asking the user clarifying questions about the skill's purpose, inputs, outputs, and constraints. | ||
| Before beginning, read and respect the constraints in please.mdc. | ||
| } | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should we add a constraint or instruction that skills should be written in SudoLang? |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,236 @@ | ||
| --- | ||
| name: create-skill | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The name is missing the |
||
| description: Create new agent skills following the AgentSkills.io specification. Use when the user wants to author, scaffold, or generate a new agent skill with proper structure, validation, and best practices. | ||
| metadata: | ||
| author: paralleldrive | ||
| version: "1.0" | ||
| --- | ||
| # Create Skill | ||
|
|
||
| Create agent skills following the [AgentSkills.io](https://agentskills.io/specification) specification. Skills use **verb form** naming (e.g., `format-code`, `create-skill`). Agents use **noun form** (e.g., `code-formatter`, `skill-creator`). | ||
|
|
||
| Read `ai/rules/sudolang/sudolang-syntax.mdc` for SudoLang syntax conventions. See `ai/rules/productmanager.mdc` for an example of well-written SudoLang (types, interfaces, composition). | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The spec says:
Even if it's not possible to make these references one level deep (unless symlink?), should we make these paths relative?
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes! We absolutely should use reltive paths (NOT symlinks) |
||
|
|
||
| ## Skill Structure | ||
|
|
||
| A skill is a directory containing a `SKILL.md` file with optional supporting directories: | ||
|
|
||
| ``` | ||
| skill-name/ | ||
| ├── SKILL.md # Required: YAML frontmatter + markdown instructions | ||
| ├── scripts/ # Optional: executable code agents can run | ||
| ├── references/ # Optional: documentation loaded on demand | ||
| └── assets/ # Optional: templates, images, data files | ||
| ``` | ||
|
|
||
| ## Types | ||
|
|
||
| type SkillName = string(1-64, lowercase alphanumeric + hyphens, no leading/trailing/consecutive hyphens, must match parent directory name) | ||
| type SkillDescription = string(1-1024, describes what skill does AND when to use it) | ||
|
|
||
| ## Interfaces | ||
|
|
||
| Frontmatter { | ||
| name: SkillName // required | ||
| description: SkillDescription // required | ||
| license // optional | ||
| compatibility: string(1-500) // optional, environment requirements | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The full set of recognized frontmatter fields is name (required), description (required), license, compatibility, metadata, and allowed-tools (experimental). All others are ignored. |
||
| metadata {} // optional, arbitrary key-value pairs (see AIDD Extensions below) | ||
| allowed-tools // optional, space-delimited tool list (experimental) | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You note it's "experimental" but still list it as a top-level field. The spec doesn't define it either — it's a Claude Code extension. Should we move it under metadata or add a clear disclaimer that it's implementation-specific and not portable?
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The spec DOES define it. Top level of the correct place and this is a useful field to call out non-standard but indispensable tools such as |
||
| } | ||
|
|
||
| ### AIDD Extensions via `metadata` | ||
|
|
||
| The AgentSkills.io spec uses progressive disclosure: only `name` and `description` (~100 tokens) are loaded at startup; the full SKILL.md body (< 5000 tokens recommended) loads only on activation; resources load on demand. | ||
|
|
||
| The spec has no `alwaysApply` equivalent. AIDD uses `metadata.alwaysApply` to mark skills whose full instructions should be preloaded into agent context on project init. Use sparingly - every always-applied skill consumes context budget. | ||
|
|
||
| ```yaml | ||
| metadata: | ||
| alwaysApply: "true" # AIDD extension: preload full SKILL.md on project init | ||
| ``` | ||
|
|
||
| SizeMetrics { | ||
| frontmatterTokens: number // should be < 100 | ||
| bodyLines: number // should be < 160, must be < 500 | ||
| bodyTokens: number // should be < 5000 | ||
| } | ||
|
|
||
| SkillPlan { | ||
| name: SkillName | ||
| purpose: SkillDescription | ||
| alwaysApply: boolean // should this skill preload on project init? | ||
| relatedSkills[] // existing skills found in discovery | ||
| bestPractices[] // findings from web research | ||
| proposedSections[] // planned SKILL.md structure | ||
| optionalDirs: ["scripts" | "references" | "assets"] | ||
| sizeEstimate: SizeMetrics | ||
| } | ||
|
|
||
| ## Process | ||
|
|
||
| createSkill(userRequest) { | ||
| gatherRequirements | ||
| |> discoverRelatedSkills | ||
| |> researchBestPractices | ||
| |> nameSkill | ||
| |> buildPlan | ||
| |> presentPlan | ||
| |> awaitApproval | ||
| |> draftSkillMd | ||
| |> writeSkill | ||
| |> validate | ||
| |> reportMetrics | ||
| } | ||
|
|
||
| gatherRequirements(userRequest) { | ||
| ask clarifying questions about: | ||
| - purpose: what problem does this skill solve? | ||
| - inputs and outputs: what does the skill consume and produce? | ||
| - constraints: any technical limitations or requirements? | ||
| - alwaysApply: should this skill be preloaded into agent context on every session? | ||
| hint: recommend "yes" only if the skill applies to nearly every task (e.g., coding standards, security checks) | ||
| hint: recommend "no" for task-specific skills that activate on demand (e.g., pdf-processing, deploy-app) | ||
| } | ||
|
|
||
| discoverRelatedSkills(skillTopic) { | ||
| searchPaths = ["$projectRoot/ai/", "$projectRoot/aidd-custom/"] | ||
| scan for SKILL.md, .mdc, and .md files | ||
| read frontmatter descriptions | ||
| identify overlapping or complementary skills | ||
| report: related skills, how to leverage them, overlap to avoid | ||
| } | ||
|
|
||
| researchBestPractices(skillTopic) { | ||
| use web search to find best practices for $skillTopic | ||
| summarize key findings relevant to skill authoring | ||
| } | ||
|
|
||
| nameSkill(topic) { | ||
| use verb form (e.g., `format-code`, `generate-report`) | ||
| validate against SkillName type constraints | ||
| } | ||
|
|
||
| buildPlan() => SkillPlan | ||
|
|
||
| presentPlan(plan: SkillPlan) { | ||
| show the user the full SkillPlan | ||
| ask if any changes are required | ||
| await explicit user approval | ||
| } | ||
|
|
||
| draftSkillMd(plan: SkillPlan) { | ||
| write Frontmatter with name + description (required), optional fields as needed | ||
| if (plan.alwaysApply) add `metadata.alwaysApply: "true"` | ||
| write body with required documentation sections (see Documentation Requirements below) | ||
| } | ||
|
|
||
| writeSkill(skillMd) { | ||
| write to `$projectRoot/aidd-custom/${skillName}/SKILL.md` | ||
| create optional directories (scripts/, references/, assets/) if planned | ||
| } | ||
|
|
||
| ## Validation | ||
|
|
||
| After creating the skill, validate using the unit-tested validator at `scripts/validate-skill.js`: | ||
|
|
||
| ```bash | ||
| # If skills-ref is available: | ||
| skills-ref validate ./path-to-skill-directory | ||
|
|
||
| # Then run the symbolic size/name validator: | ||
| node ai/skills/create-skill/scripts/validate-skill.js ./path-to-skill-directory | ||
| ``` | ||
|
Comment on lines
+133
to
+143
|
||
|
|
||
| The validator exports: `parseSkillMd`, `validateName`, `calculateMetrics`, `checkThresholds`. | ||
|
|
||
| ## Documentation Requirements | ||
|
|
||
| Every generated skill MUST include these documentation sections in the SKILL.md body: | ||
|
|
||
| RequiredSections { | ||
| "# Title" // skill name as heading | ||
| "## When to use" // clear activation criteria - when should an agent use this skill? | ||
| "## Steps" | "## Process" // step-by-step instructions for executing the skill | ||
| "## Examples" // concrete input/output examples demonstrating usage | ||
| "## Edge cases" // known limitations, error handling, boundary conditions | ||
| } | ||
|
|
||
| The `description` field is the skill's elevator pitch - it must be good enough for an agent to decide whether to activate the skill based on description alone. Write it as if it were the only thing an agent reads before deciding. | ||
|
|
||
| ## Constraints | ||
|
|
||
| Constraints { | ||
| // Structure | ||
| NEVER create a standalone .mdc or .md file as a skill | ||
| The output MUST be `$projectRoot/aidd-custom/${skillName}/SKILL.md` | ||
| The file MUST be named exactly `SKILL.md` (uppercase) | ||
|
|
||
| // Frontmatter | ||
| Frontmatter MUST include `name` and `description` as required fields | ||
| NEVER use non-spec keys as top-level frontmatter fields | ||
| Use `metadata` for AIDD extensions (e.g., `metadata.alwaysApply: "true"`) | ||
| The `name` field MUST satisfy the SkillName type | ||
| The `description` field MUST satisfy the SkillDescription type | ||
|
|
||
| // Size | ||
| SKILL.md body SHOULD stay under 160 lines and MUST stay under 500 lines | ||
| SKILL.md body SHOULD stay under 5000 tokens | ||
| If content exceeds limits, split into references/ directory files | ||
|
|
||
| // Documentation | ||
| SKILL.md body MUST include all RequiredSections | ||
| The description field MUST be specific enough for agent activation decisions | ||
| Include concrete examples - not just abstract instructions | ||
|
|
||
| // Validation | ||
| ALWAYS run `scripts/validate-skill.js` after creating the skill | ||
| ALWAYS run `skills-ref validate` if available | ||
| ALWAYS report size metrics and any warnings to the user | ||
| } | ||
|
|
||
| ## Example SKILL.md | ||
|
|
||
| ```markdown | ||
| --- | ||
| name: format-code | ||
| description: Format source code files according to project style guides and conventions. Use when code needs formatting, linting fixes, style consistency checks, or when a user mentions "format", "lint", or "prettier". | ||
| metadata: | ||
| author: my-org | ||
| version: "1.0" | ||
| alwaysApply: "true" | ||
| --- | ||
|
|
||
| # Format Code | ||
|
|
||
| Format source code according to project conventions and style guides. | ||
|
|
||
| ## When to use | ||
|
|
||
| Use this skill when: | ||
| - Code needs formatting or linting fixes | ||
| - A new file is created and needs style consistency | ||
| - The user asks for style or formatting help | ||
|
|
||
| ## Steps | ||
|
|
||
| 1. Detect the project's formatter configuration (prettier, biome, eslint, etc.) | ||
| 2. Identify files that need formatting | ||
| 3. Apply formatting rules | ||
| 4. Report changes made | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The process ends at # If skills-ref is available:
skills-ref validate ./path-to-skill-directory
# Then run the local validator:
node ai/skills/create-skill/scripts/validate-skill.js ./path-to-skill-directoryWithout this, frontmatter issues, name constraint violations, and size threshold breaches go undetected until someone tries to use the skill. |
||
|
|
||
| ## Examples | ||
|
|
||
| Given a project with `.prettierrc`, run: `npx prettier --write "src/**/*.ts"` | ||
| Given a project with `biome.json`, run: `npx @biomejs/biome format --write` | ||
|
|
||
| ## Edge cases | ||
|
|
||
| - If no formatter config exists, ask the user which style to use | ||
| - For mixed-language projects, apply per-language formatters | ||
| - If formatter conflicts with linter, prioritize linter configuration | ||
| ``` | ||
|
|
||
| Commands { | ||
| /create-skill - create a new agent skill following the AgentSkills.io specification | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # create-skill | ||
|
|
||
| This index provides an overview of the contents in this directory. | ||
|
|
||
| ## Subdirectories | ||
|
|
||
| ### 📁 scripts/ | ||
|
|
||
| See [`scripts/index.md`](./scripts/index.md) for contents. | ||
|
|
||
| ## Files | ||
|
|
||
| ### Create Skill | ||
|
|
||
| **File:** `SKILL.md` | ||
|
|
||
| Create new agent skills following the AgentSkills.io specification. Use when the user wants to author, scaffold, or generate a new agent skill with proper structure, validation, and best practices. | ||
|
|
| Original file line number | Diff line number | Diff line change | ||||||
|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,5 @@ | ||||||||
| # scripts | ||||||||
|
|
||||||||
| This index provides an overview of the contents in this directory. | ||||||||
|
|
||||||||
| *This directory is empty.* | ||||||||
|
||||||||
| *This directory is empty.* | |
| - `validate-skill.js`: Script for validating skill definitions and configuration. | |
| - `validate-skill.test.js`: Tests for `validate-skill.js`. |
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should we also add the cabability to check for allowed properties? const ALLOWED_PROPERTIES = new Set([
"name",
"description",
"license",
"metadata",
]);
function extractFrontmatter(content: string): ValidationResult | string {
if (!content.startsWith("---")) {
return { message: "No YAML frontmatter found", valid: false };
}
const match = content.match(/^---\n(.*?)\n---/s);
if (!match) {
return { message: "Invalid frontmatter format", valid: false };
}
return match[1];
}
function parseFrontmatter(
raw: string,
): ValidationResult | Record<string, unknown> {
try {
const parsed = yaml.load(raw);
if (
typeof parsed !== "object" ||
parsed === null ||
Array.isArray(parsed)
) {
return { message: "Frontmatter must be a YAML dictionary", valid: false };
}
return parsed as Record<string, unknown>;
} catch (e) {
return {
message: `Invalid YAML in frontmatter: ${e}`,
valid: false,
};
}
}
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes |
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,47 @@ | ||||||||||||||||||||||||||||||||||||||
| /** | ||||||||||||||||||||||||||||||||||||||
| * Validate an AgentSkills.io SKILL.md file. | ||||||||||||||||||||||||||||||||||||||
| * | ||||||||||||||||||||||||||||||||||||||
| * Usage: node validate-skill.js ./path-to-skill-directory | ||||||||||||||||||||||||||||||||||||||
| */ | ||||||||||||||||||||||||||||||||||||||
|
Comment on lines
+1
to
+5
|
||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
| export const parseSkillMd = (content) => { | ||||||||||||||||||||||||||||||||||||||
| const match = content.match(/^---\n([\s\S]*?)\n---/); | ||||||||||||||||||||||||||||||||||||||
| if (!match) return { body: content, frontmatter: "" }; | ||||||||||||||||||||||||||||||||||||||
| return { | ||||||||||||||||||||||||||||||||||||||
| body: content.slice(match[0].length).trim(), | ||||||||||||||||||||||||||||||||||||||
| frontmatter: match[1], | ||||||||||||||||||||||||||||||||||||||
| }; | ||||||||||||||||||||||||||||||||||||||
|
Comment on lines
+7
to
+13
|
||||||||||||||||||||||||||||||||||||||
| }; | ||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
| export const validateName = (name, dirName) => { | ||||||||||||||||||||||||||||||||||||||
| const errors = []; | ||||||||||||||||||||||||||||||||||||||
| if (name.length < 1 || name.length > 64) | ||||||||||||||||||||||||||||||||||||||
| errors.push("Name must be 1-64 characters"); | ||||||||||||||||||||||||||||||||||||||
| if (/[^a-z0-9-]/.test(name)) | ||||||||||||||||||||||||||||||||||||||
| errors.push("Name must be lowercase alphanumeric + hyphens only"); | ||||||||||||||||||||||||||||||||||||||
| if (/^-|-$/.test(name)) errors.push("Name must not start or end with hyphen"); | ||||||||||||||||||||||||||||||||||||||
| if (/--/.test(name)) errors.push("Name must not contain consecutive hyphens"); | ||||||||||||||||||||||||||||||||||||||
| if (name !== dirName) errors.push("Name must match directory name"); | ||||||||||||||||||||||||||||||||||||||
| return errors; | ||||||||||||||||||||||||||||||||||||||
| }; | ||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
| export const calculateMetrics = (frontmatter, body) => ({ | ||||||||||||||||||||||||||||||||||||||
| bodyLines: body.split("\n").length, | ||||||||||||||||||||||||||||||||||||||
| bodyTokens: Math.ceil(body.length / 4), | ||||||||||||||||||||||||||||||||||||||
| frontmatterTokens: Math.ceil(frontmatter.length / 4), | ||||||||||||||||||||||||||||||||||||||
| }); | ||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
| export const checkThresholds = (metrics) => { | ||||||||||||||||||||||||||||||||||||||
| const warnings = []; | ||||||||||||||||||||||||||||||||||||||
| if (metrics.frontmatterTokens >= 100) | ||||||||||||||||||||||||||||||||||||||
| warnings.push("Frontmatter exceeds 100 token guideline"); | ||||||||||||||||||||||||||||||||||||||
| if (metrics.bodyLines >= 160) | ||||||||||||||||||||||||||||||||||||||
| warnings.push("Body exceeds 160 line guideline"); | ||||||||||||||||||||||||||||||||||||||
| if (metrics.bodyLines >= 500) | ||||||||||||||||||||||||||||||||||||||
| warnings.push( | ||||||||||||||||||||||||||||||||||||||
| "Body exceeds 500 line spec limit - split into reference files", | ||||||||||||||||||||||||||||||||||||||
| ); | ||||||||||||||||||||||||||||||||||||||
| if (metrics.bodyTokens >= 5000) | ||||||||||||||||||||||||||||||||||||||
| warnings.push("Body exceeds 5000 token spec guideline"); | ||||||||||||||||||||||||||||||||||||||
|
Comment on lines
+37
to
+45
|
||||||||||||||||||||||||||||||||||||||
| warnings.push("Frontmatter exceeds 100 token guideline"); | |
| if (metrics.bodyLines >= 160) | |
| warnings.push("Body exceeds 160 line guideline"); | |
| if (metrics.bodyLines >= 500) | |
| warnings.push( | |
| "Body exceeds 500 line spec limit - split into reference files", | |
| ); | |
| if (metrics.bodyTokens >= 5000) | |
| warnings.push("Body exceeds 5000 token spec guideline"); | |
| warnings.push("WARNING: Frontmatter is at or above 100 token guideline"); | |
| if (metrics.bodyLines >= 160) | |
| warnings.push("WARNING: Body is at or above 160 line guideline"); | |
| if (metrics.bodyLines >= 500) | |
| warnings.push( | |
| "ERROR: Body is at or above 500 line spec limit (hard limit) - split into reference files", | |
| ); | |
| if (metrics.bodyTokens >= 5000) | |
| warnings.push("WARNING: Body is at or above 5000 token spec guideline"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hard 500-line limit indistinguishable from soft warnings
Medium Severity
checkThresholds returns the 500-line hard "MUST" violation in the same warnings array as soft "SHOULD" guideline advisories. The spec and SizeMetrics definition clearly distinguish between "should be < 160" (soft) and "must be < 500" (hard), but callers cannot differentiate them without parsing message strings. A spec-violating skill could be treated as passing if warnings are non-blocking.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Validator has no CLI entry point despite documented usage
Medium Severity
validate-skill.js only exports functions — it contains no CLI entry point code. But both its JSDoc header (Usage: node validate-skill.js ./path-to-skill-directory) and SKILL.md instruct agents to run it as a CLI command. Executing node validate-skill.js ./path silently exits with no output and no validation, so the constraint "ALWAYS run scripts/validate-skill.js after creating the skill" is effectively a no-op that gives a false sense of validation passing.


There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know if this is needed anymore. We still have this in the skills PR, too, but Cursor, Codex and Claude Code all recognize skills as slash commands automatically. I suspect this will be (or is already) industry standard.