From 3399e7570b158eb32474a44add8d0ccb0811cae3 Mon Sep 17 00:00:00 2001 From: Rachael Rose Renk <91027132+rachaelrenk@users.noreply.github.com> Date: Tue, 5 May 2026 11:41:47 -1000 Subject: [PATCH 1/7] docs: add AFDocs audit skill for agent-friendly docs monitoring Add a reusable skill (.agents/skills/afdocs-audit/) that runs the AFDocs scorecard against docs.warp.dev and produces a structured report. Follows the same pattern as the existing docs-seo-audit skill. Includes: - SKILL.md with instructions for running, reporting, and Slack notifications - afdocs_audit.mjs wrapper script that runs npx afdocs check --format json, parses the output, and produces a structured report with scores, issues, and fix guidance - references/known-exceptions.md documenting expected failures and false positives (content-start-position, markdown-content-parity, page-size) The skill is designed to be run by a scheduled Oz agent (weekly cadence) to monitor the docs site's agent-friendliness score over time. Co-Authored-By: Oz --- .agents/skills/afdocs-audit/SKILL.md | 141 ++++++++++++++ .../references/known-exceptions.md | 42 ++++ .../afdocs-audit/scripts/afdocs_audit.mjs | 183 ++++++++++++++++++ 3 files changed, 366 insertions(+) create mode 100644 .agents/skills/afdocs-audit/SKILL.md create mode 100644 .agents/skills/afdocs-audit/references/known-exceptions.md create mode 100755 .agents/skills/afdocs-audit/scripts/afdocs_audit.mjs diff --git a/.agents/skills/afdocs-audit/SKILL.md b/.agents/skills/afdocs-audit/SKILL.md new file mode 100644 index 0000000..db0dc39 --- /dev/null +++ b/.agents/skills/afdocs-audit/SKILL.md @@ -0,0 +1,141 @@ +--- +name: afdocs-audit +description: >- + Audit docs.warp.dev for agent-friendly documentation issues using the AFDocs + scorecard. Checks llms.txt, markdown availability, content negotiation, page + size, URL stability, and content structure. Use when asked to check agent + readiness, run an AFDocs audit, improve the docs score, or verify llms.txt + and markdown support. +--- + +# AFDocs Audit + +Run the [AFDocs scorecard](https://agentdocsspec.com/spec/) against docs.warp.dev and report results. + +## Running the audit + +From the docs repo root: + +```bash +node .agents/skills/afdocs-audit/scripts/afdocs_audit.mjs \ + --output /tmp/afdocs-report.json +``` + +The script runs `npx afdocs check https://docs.warp.dev --format json`, parses the output, and writes a structured report. + +### Options + +- `--output FILE` — Write the JSON report to a file (otherwise prints to stdout). +- `--url URL` — Override the site URL (default: `https://docs.warp.dev`). + +## Reading the report + +The JSON report contains: +- `score` — Overall score out of 100 +- `grade` — Letter grade (A+ through F) +- `total_checks` — Number of checks run +- `summary` — Counts by status (`pass`, `fail`, `warn`, `skip`) +- `categories` — Per-category scores and grades +- `issues` — Array of failing and warning checks with details and fix guidance + +Each issue includes: +- `id` — Check identifier (e.g., `llms-txt-directive-html`) +- `category` — Check category (e.g., `content-discoverability`) +- `status` — `fail` or `warn` +- `message` — Human-readable description +- `fix` — Suggested fix from the AFDocs spec + +### Known exceptions + +Before reporting, cross-reference every issue against the known exceptions in `references/known-exceptions.md`. Classify each issue into exactly one bucket: +- **Allowlisted** — known exceptions that are intentional (not problems) +- **Remaining** — genuine issues that need attention + +Only include a section if its count is > 0. Never list allowlisted issues under "Remaining." + +## Reporting results + +After running the audit, ALWAYS report the results to the user before taking any action. Include: + +1. **Score**: Overall score, grade, and comparison to last known score (82/100 B as of 2026-05-05) +2. **Failures first**: List every fail-severity check with its message and fix guidance. These are the most impactful. +3. **Warnings**: List warning-severity checks with context. +4. **Allowlisted**: Briefly note any known exceptions that were flagged. +5. **If all checks pass**: Explicitly tell the user everything looks clean. + +Example report format: +``` +AFDocs audit complete: 23 checks run, score 82/100 (B). + +**Failures (5):** +- llms-txt-directive-html: No llms.txt directive in HTML pages + Fix: Add a visually-hidden element near the top of each page with a link to /llms.txt +- content-negotiation: Server ignores Accept: text/markdown + Fix: Add middleware to serve .md variants when Accept: text/markdown is requested + +**Warnings (1):** +- llms-txt-coverage: 80% of sitemap pages covered (247/308) + +**Allowlisted (2):** +- page-size-markdown: 1 page over 50K (changelog — intentionally long) +- markdown-content-parity: 7 pages with minor diffs (Turndown escaping, not real content gaps) +``` + +After reporting, ask the user which issues they want to address. + +## Slack notification (optional) + +If instructed to send a report to Slack, post a summary after the audit completes. + +1. Check if `BUZZ_SLACK_TOKEN` environment variable exists. +2. If the token exists, send a summary to the specified channel. + +**Format:** + +``` +*AFDocs Audit — * +Score: /100 () | checks | pass, fail, warn + +*Failures ():* +• : + +*Warnings ():* +• : + +*Allowlisted ():* +• : +``` + +Send using: + +```bash +curl -X POST https://slack.com/api/chat.postMessage \ + -H "Authorization: Bearer $BUZZ_SLACK_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "channel": "", + "text": "", + "unfurl_links": false, + "unfurl_media": false + }' +``` + +If `BUZZ_SLACK_TOKEN` is not set, skip the notification and note that the token is required. + +## Dependencies + +Node.js 18+ with npm (for `npx afdocs`). No additional install required — `afdocs` is fetched on demand by npx. + +## Checks performed + +The AFDocs scorecard evaluates these categories: + +**Content Discoverability** — llms.txt existence, validity, size, link resolution, markdown links, and in-page directives +**Markdown Availability** — .md URL support and Accept: text/markdown content negotiation +**Page Size and Truncation Risk** — rendering strategy, page sizes (markdown and HTML), and content start position +**Content Structure** — tabbed content serialization, section header quality, code fence validity +**URL Stability and Redirects** — HTTP status codes and redirect behavior +**Observability and Content Health** — llms.txt coverage, markdown/HTML parity, cache headers +**Authentication and Access** — auth gate detection and alternative access paths + +Full spec: https://agentdocsspec.com/spec/ diff --git a/.agents/skills/afdocs-audit/references/known-exceptions.md b/.agents/skills/afdocs-audit/references/known-exceptions.md new file mode 100644 index 0000000..ce3c09a --- /dev/null +++ b/.agents/skills/afdocs-audit/references/known-exceptions.md @@ -0,0 +1,42 @@ +# AFDocs Known Exceptions + +This file lists checks that may flag as warnings or failures but are expected and intentional. When reporting audit results, classify these as "Allowlisted" rather than "Remaining." + +## content-start-position + +**Expected status**: fail or warn +**Reason**: 34/50 sampled pages have content starting past 50% of the HTML output. This is inherent to Starlight's layout — sidebar navigation, header markup, and JavaScript/CSS precede the `
` content area. +**Mitigation**: The llms.txt directive, `` in ``, and `Accept: text/markdown` content negotiation middleware all steer agents to the clean markdown version, bypassing the HTML boilerplate entirely. +**Action**: No fix needed. This is a structural property of Starlight sites. + +## markdown-content-parity + +**Expected status**: warn (7 pages, ~2% average difference) +**Reason**: False positive. The "missing" segments are numbered heading text like "2. Tabbed File Viewer" where Turndown correctly escapes the period (`### 2\. Tabbed File Viewer`) to prevent markdown parsers from interpreting it as a list item. The content IS present in the markdown — the AFDocs checker's text comparison doesn't account for markdown escaping. +**Affected pages** (as of 2026-05-05): +- `/agent-platform/cloud-agents/triggers/scheduled-agents-quickstart/` — step headings +- `/agent-platform/cloud-agents/integrations/github-actions/` — numbered use case headings +- `/support-and-community/troubleshooting-and-support/troubleshooting-login-issues/` — URLs with special chars +- `/reference/cli/quickstart/` — optional step headings +- `/guides/getting-started/welcome-to-warp/` — numbered section headings +- `/terminal/editor/vim/` — "See Vim docs:" link text +- `/guides/getting-started/10-coding-features-you-should-know/` — numbered feature headings +**Action**: No fix needed. Content is intact. + +## page-size-markdown / page-size-html + +**Expected status**: warn (1 page between 50K-100K chars) +**Reason**: The changelog page (`/changelog/`) is intentionally a single long page (~4,000 lines of MDX). It is excluded from `llms-full.txt` generation due to a `hast-util-to-text` stack overflow, but is still accessible at its URL and indexed by the sitemap. +**Action**: No fix needed unless the page grows significantly larger. Petra has indicated long pages are lower priority. + +## section-header-quality + +**Expected status**: skip +**Reason**: Only evaluated when tab panels contain section headers. Most sampled pages with tabs don't have headers inside the tab panels, so the check is skipped. +**Action**: None needed. + +## auth-alternative-access + +**Expected status**: skip +**Reason**: All docs pages are publicly accessible, so no alternative access path is needed. +**Action**: None needed. diff --git a/.agents/skills/afdocs-audit/scripts/afdocs_audit.mjs b/.agents/skills/afdocs-audit/scripts/afdocs_audit.mjs new file mode 100755 index 0000000..5c01080 --- /dev/null +++ b/.agents/skills/afdocs-audit/scripts/afdocs_audit.mjs @@ -0,0 +1,183 @@ +#!/usr/bin/env node +/** + * AFDocs audit wrapper script. + * + * Runs `npx afdocs check` against docs.warp.dev, parses the JSON output, + * and produces a structured report with scores, issues, and fix guidance. + * + * Usage: + * node .agents/skills/afdocs-audit/scripts/afdocs_audit.mjs + * node .agents/skills/afdocs-audit/scripts/afdocs_audit.mjs --output /tmp/report.json + * node .agents/skills/afdocs-audit/scripts/afdocs_audit.mjs --url https://preview.docs.warp.dev + */ + +import { execSync } from 'node:child_process'; +import { writeFileSync } from 'node:fs'; +import { resolve } from 'node:path'; + +const GRADE_THRESHOLDS = [ + [97, 'A+'], + [93, 'A'], + [90, 'A-'], + [87, 'B+'], + [83, 'B'], + [80, 'B-'], + [77, 'C+'], + [73, 'C'], + [70, 'C-'], + [67, 'D+'], + [63, 'D'], + [60, 'D-'], + [0, 'F'], +]; + +function scoreToGrade(score) { + for (const [threshold, grade] of GRADE_THRESHOLDS) { + if (score >= threshold) return grade; + } + return 'F'; +} + +function parseArgs(argv) { + const args = { output: null, url: 'https://docs.warp.dev' }; + for (let i = 0; i < argv.length; i++) { + if (argv[i] === '--output') args.output = argv[++i]; + else if (argv[i] === '--url') args.url = argv[++i]; + else if (argv[i] === '--help' || argv[i] === '-h') { + console.log('Usage: node afdocs_audit.mjs [--output FILE] [--url URL]'); + process.exit(0); + } + } + return args; +} + +function runAfdocsCheck(url) { + try { + const stdout = execSync(`npx afdocs check ${url} --format json`, { + encoding: 'utf8', + maxBuffer: 10 * 1024 * 1024, // 10 MB — the JSON output can be large + timeout: 300_000, // 5 minutes + stdio: ['pipe', 'pipe', 'pipe'], + }); + return JSON.parse(stdout); + } catch (error) { + // npx afdocs exits with code 1 when there are failures, but still + // prints valid JSON to stdout. Try to parse it. + if (error.stdout) { + try { + return JSON.parse(error.stdout); + } catch { + // Fall through to error + } + } + throw new Error(`Failed to run afdocs check: ${error.message}`); + } +} + +function buildReport(raw) { + const { summary, results } = raw; + const score = raw.summary?.score ?? estimateScore(results); + const grade = scoreToGrade(score); + + // Group results by category + const categories = {}; + for (const r of results) { + if (!categories[r.category]) { + categories[r.category] = { checks: [], pass: 0, fail: 0, warn: 0, skip: 0 }; + } + categories[r.category].checks.push(r); + categories[r.category][r.status] = (categories[r.category][r.status] || 0) + 1; + } + + // Extract issues (fail + warn) + const issues = results + .filter((r) => r.status === 'fail' || r.status === 'warn') + .map((r) => ({ + id: r.id, + category: r.category, + status: r.status, + message: r.message, + fix: r.details?.fix || r.fix || null, + })); + + return { + url: raw.url, + timestamp: raw.timestamp || new Date().toISOString(), + score, + grade, + total_checks: summary.total, + summary: { + pass: summary.pass, + fail: summary.fail, + warn: summary.warn, + skip: summary.skip, + }, + categories: Object.fromEntries( + Object.entries(categories).map(([name, cat]) => [ + name, + { pass: cat.pass, fail: cat.fail, warn: cat.warn, skip: cat.skip }, + ]) + ), + issues, + all_results: results.map((r) => ({ id: r.id, category: r.category, status: r.status, message: r.message })), + }; +} + +/** + * Estimate score from results when the raw JSON doesn't include a score field. + * Uses a simple formula: (pass / (total - skip)) * 100. + */ +function estimateScore(results) { + const scored = results.filter((r) => r.status !== 'skip'); + if (scored.length === 0) return 100; + const passing = scored.filter((r) => r.status === 'pass').length; + // Warnings count as half-pass + const warnings = scored.filter((r) => r.status === 'warn').length; + return Math.round(((passing + warnings * 0.5) / scored.length) * 100); +} + +function printSummary(report) { + console.log(`\nAFDocs Audit — ${report.url}`); + console.log(`Score: ${report.score}/100 (${report.grade})`); + console.log( + `Checks: ${report.total_checks} total | ${report.summary.pass} pass, ${report.summary.fail} fail, ${report.summary.warn} warn, ${report.summary.skip} skip` + ); + + if (report.issues.length === 0) { + console.log('\n✅ All checks passed!'); + return; + } + + const failures = report.issues.filter((i) => i.status === 'fail'); + const warnings = report.issues.filter((i) => i.status === 'warn'); + + if (failures.length > 0) { + console.log(`\nFailures (${failures.length}):`); + for (const f of failures) { + console.log(` ✗ ${f.id}: ${f.message}`); + if (f.fix) console.log(` Fix: ${f.fix}`); + } + } + + if (warnings.length > 0) { + console.log(`\nWarnings (${warnings.length}):`); + for (const w of warnings) { + console.log(` ⚠ ${w.id}: ${w.message}`); + } + } +} + +// Main +const args = parseArgs(process.argv.slice(2)); +console.log(`Running AFDocs check on ${args.url}...`); + +const raw = runAfdocsCheck(args.url); +const report = buildReport(raw); + +printSummary(report); + +if (args.output) { + const outputPath = resolve(args.output); + writeFileSync(outputPath, JSON.stringify(report, null, 2)); + console.log(`\nReport written to ${outputPath}`); +} From b0066fd04099d90a7515d8578bac798d7350141c Mon Sep 17 00:00:00 2001 From: Rachael Rose Renk <91027132+rachaelrenk@users.noreply.github.com> Date: Tue, 5 May 2026 11:54:46 -1000 Subject: [PATCH 2/7] docs: add afdocs-fix remediation skill (Part B) Add the companion skill that reads the afdocs-audit output and applies automated fixes for each failing check. Includes: - Fix procedures for each AFDocs check (llms.txt directive, content negotiation middleware, llms.txt coverage, MCP discovery, etc.) - Code snippets and file paths for each fix - Diagnostic steps for coverage mismatches - Clear separation of automatable vs. non-automatable checks - PR conventions (title prefix, labels, co-author) This is the 'fix layer' that a scheduled Oz agent runs after the audit skill to automatically remediate regressions. Co-Authored-By: Oz --- .agents/skills/afdocs-fix/SKILL.md | 166 +++++++++++++++++++++++++++++ 1 file changed, 166 insertions(+) create mode 100644 .agents/skills/afdocs-fix/SKILL.md diff --git a/.agents/skills/afdocs-fix/SKILL.md b/.agents/skills/afdocs-fix/SKILL.md new file mode 100644 index 0000000..3e75387 --- /dev/null +++ b/.agents/skills/afdocs-fix/SKILL.md @@ -0,0 +1,166 @@ +--- +name: afdocs-fix +description: >- + Fix agent-friendly documentation issues found by the afdocs-audit skill. + Reads the AFDocs audit report and applies automated fixes for failing checks. + Use after running the afdocs-audit skill, or when asked to fix AFDocs issues, + improve the agent-ready score, or remediate llms.txt / markdown availability + problems. +--- + +# AFDocs Fix + +Apply automated fixes for issues found by the `afdocs-audit` skill. This skill reads an AFDocs audit report and remediates fixable issues. + +## Prerequisites + +1. Run the audit skill first to produce a report: + ```bash + node .agents/skills/afdocs-audit/scripts/afdocs_audit.mjs --output /tmp/afdocs-report.json + ``` +2. Read the report and cross-reference against `afdocs-audit/references/known-exceptions.md` to identify which issues are genuine (vs. allowlisted). + +## Fix procedures by check + +For each failing or warning check, follow the procedure below. Skip checks listed in `known-exceptions.md`. + +### llms-txt-directive-html (Content Discoverability) + +**What's wrong**: No agent-facing directive pointing to llms.txt in the HTML pages. + +**Fix**: Add a visually-hidden element in `src/components/CustomHeader.astro` before the `.header` div: + +```html + +``` + +The `.sr-only` class is Starlight's built-in screen-reader-only utility. The `sanitizeRoot` function in `docs-markdown-integration.js` strips `.sr-only` elements during HTML → markdown conversion, so this directive will appear only in the HTML version (the markdown version gets its own directive — see below). + +**Files**: `src/components/CustomHeader.astro` + +### llms-txt-directive-md (Content Discoverability) + +**What's wrong**: No llms.txt directive in the generated markdown pages. + +**Fix**: In `src/integrations/docs-markdown-integration.js`, in the `convertHtmlToMarkdown` function, prepend a blockquote before the title: + +```javascript +const llmsDirective = + '> For the complete documentation index, see [llms.txt](/llms.txt).\n' + + '> Markdown versions of each page are available by appending .md to any URL.'; +const sections = [llmsDirective, `# ${normalizeWhitespace(title)}`]; +``` + +**Files**: `src/integrations/docs-markdown-integration.js` + +### content-negotiation (Markdown Availability) + +**What's wrong**: Server ignores `Accept: text/markdown` header and returns HTML. + +**Fix**: Create Astro middleware at `src/middleware.ts` that uses the existing `shouldServeMarkdown()` helper from `src/lib/docs-markdown.js`: + +```typescript +import { defineMiddleware } from 'astro:middleware'; +import { + shouldServeMarkdown, + isEligibleDocHtmlPath, + getMarkdownPathFromHtmlPath, +} from './lib/docs-markdown.js'; + +export const onRequest = defineMiddleware(async (context, next) => { + const { request, url } = context; + if (!isEligibleDocHtmlPath(url.pathname)) return next(); + if (!shouldServeMarkdown(request)) return next(); + const mdPath = getMarkdownPathFromHtmlPath(url.pathname); + return context.rewrite(mdPath); +}); +``` + +The `shouldServeMarkdown` helper checks both the `Accept` header (for `text/markdown` and `text/plain`) and user-agent tokens (ChatGPT, Claude, Cursor, etc.). + +**Files**: `src/middleware.ts` (new file) + +### llms-txt-coverage (Observability) + +**What's wrong**: llms.txt covers less than 95% of sitemap pages. + +**Fix**: Update the `customSets` paths in `astro.config.mjs` under `starlightLlmsTxt()`: + +1. Check that every content directory under `src/content/docs/` has a matching customSet entry. +2. Common mismatches: + - Directory was renamed but customSet path wasn't updated (e.g., `university/**` → `guides/**`) + - New content directories added without a corresponding customSet + - Sub-paths excluded too aggressively (e.g., `support-and-community/community/` pages) +3. For large pages that cause `hast-util-to-text` stack overflows: the `exclude` option only applies to `llms-small.txt`, NOT `llms-full.txt`. Do not add pages to `exclude` expecting them to be skipped in `llms-full.txt`. + +**Diagnostic steps**: +```bash +# List content directories +ls src/content/docs/ + +# Compare against customSets in astro.config.mjs +grep -A1 "label:" astro.config.mjs | grep "paths:" +``` + +**Files**: `astro.config.mjs` + +### markdown-url-support (Markdown Availability) + +**What's wrong**: Some pages don't return markdown when `.md` is appended to the URL. + +**Fix**: Check that `src/integrations/docs-markdown-integration.js` and `src/pages/[...slug].md.ts` are both present and correctly configured. The integration generates static `.md` files at build time; the page route serves them in dev mode. + +**Files**: `src/integrations/docs-markdown-integration.js`, `src/pages/[...slug].md.ts` + +### http-status-codes (URL Stability) + +**What's wrong**: The site returns 200 for non-existent pages (soft 404). + +**Fix**: Ensure `public/404.html` or an Astro 404 page exists and returns a proper 404 status code. In Vercel, configure `cleanUrls` and ensure the adapter handles 404 responses correctly. + +**Files**: `vercel.json`, `src/pages/404.astro` (if it exists) + +### MCP Server Discoverable + +**What's wrong**: No MCP server discovery endpoint found. + +**Fix**: Add a static discovery file at `public/.well-known/mcp.json`: + +```json +{ + "name": "Warp Documentation", + "description": "Search and retrieve Warp documentation.", + "url": "https://warp.mcp.kapa.ai/sse", + "transport": "sse" +} +``` + +This points to the existing Kapa-hosted MCP server. No custom implementation needed. + +**Files**: `public/.well-known/mcp.json` (new file) + +## Checks with no automated fix + +These checks require infrastructure or design changes that can't be automated: + +- **content-start-position** — Inherent to Starlight's layout. Mitigated by content negotiation and llms.txt directives. See `known-exceptions.md`. +- **page-size-markdown / page-size-html** — Requires editorial decision to split long pages. Flag in the report but do not auto-fix. +- **section-header-quality** — Content-level change requiring human judgment. + +## Applying fixes + +1. Create a branch: `git checkout -b automated/afdocs-fixes origin/main` +2. Apply the fixes for each failing check (skip allowlisted checks). +3. Validate: `npm run build` (the build must succeed). +4. Commit with the prefix: `Automated AFDocs fixes: ` +5. Open a PR with the `afdocs` label: `gh pr create --label afdocs` + +## PR conventions + +- Title must be prefixed with `Automated AFDocs fixes:` (e.g., `Automated AFDocs fixes: add llms.txt directive and content negotiation middleware`) +- Add the `afdocs` label to the PR +- Include the audit score (before/after if known) in the PR description +- Include the co-author line: `Co-Authored-By: Oz ` From fd3d9a9ed67eb341574d2d151f1f62843390e80b Mon Sep 17 00:00:00 2001 From: Rachael Rose Renk <91027132+rachaelrenk@users.noreply.github.com> Date: Tue, 5 May 2026 12:11:30 -1000 Subject: [PATCH 3/7] chore: hardcode #growth-docs channel ID (C09BVK0PL3Y) in afdocs-audit skill Co-Authored-By: Oz --- .agents/skills/afdocs-audit/SKILL.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.agents/skills/afdocs-audit/SKILL.md b/.agents/skills/afdocs-audit/SKILL.md index db0dc39..61aa3f0 100644 --- a/.agents/skills/afdocs-audit/SKILL.md +++ b/.agents/skills/afdocs-audit/SKILL.md @@ -83,12 +83,12 @@ AFDocs audit complete: 23 checks run, score 82/100 (B). After reporting, ask the user which issues they want to address. -## Slack notification (optional) +## Slack notification -If instructed to send a report to Slack, post a summary after the audit completes. +After the audit completes, post a summary to the **#growth-docs** Slack channel (`C09BVK0PL3Y`). 1. Check if `BUZZ_SLACK_TOKEN` environment variable exists. -2. If the token exists, send a summary to the specified channel. +2. If the token exists, send a summary to `#growth-docs`. **Format:** @@ -113,7 +113,7 @@ curl -X POST https://slack.com/api/chat.postMessage \ -H "Authorization: Bearer $BUZZ_SLACK_TOKEN" \ -H "Content-Type: application/json" \ -d '{ - "channel": "", + "channel": "C09BVK0PL3Y", "text": "", "unfurl_links": false, "unfurl_media": false From c81c9c7d6010a8564566bfabeb739167b5c703b0 Mon Sep 17 00:00:00 2001 From: Rachael Rose Renk <91027132+rachaelrenk@users.noreply.github.com> Date: Tue, 5 May 2026 12:18:44 -1000 Subject: [PATCH 4/7] =?UTF-8?q?chore:=20simplify=20PR=20conventions=20?= =?UTF-8?q?=E2=80=94=20shorten=20prefix,=20remove=20nonexistent=20label?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Oz --- .agents/skills/afdocs-fix/SKILL.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/.agents/skills/afdocs-fix/SKILL.md b/.agents/skills/afdocs-fix/SKILL.md index 3e75387..c34c497 100644 --- a/.agents/skills/afdocs-fix/SKILL.md +++ b/.agents/skills/afdocs-fix/SKILL.md @@ -152,15 +152,14 @@ These checks require infrastructure or design changes that can't be automated: ## Applying fixes -1. Create a branch: `git checkout -b automated/afdocs-fixes origin/main` +1. Create a branch: `git checkout -b afdocs-fixes origin/main` 2. Apply the fixes for each failing check (skip allowlisted checks). 3. Validate: `npm run build` (the build must succeed). -4. Commit with the prefix: `Automated AFDocs fixes: ` -5. Open a PR with the `afdocs` label: `gh pr create --label afdocs` +4. Commit with the prefix: `AFDocs fixes: ` +5. Open a PR: `gh pr create` ## PR conventions -- Title must be prefixed with `Automated AFDocs fixes:` (e.g., `Automated AFDocs fixes: add llms.txt directive and content negotiation middleware`) -- Add the `afdocs` label to the PR +- Title must be prefixed with `AFDocs fixes:` (e.g., `AFDocs fixes: add llms.txt directive and content negotiation middleware`) - Include the audit score (before/after if known) in the PR description - Include the co-author line: `Co-Authored-By: Oz ` From c20e878655647b73e1de8c0b05a92413e7853e63 Mon Sep 17 00:00:00 2001 From: Rachael Rose Renk <91027132+rachaelrenk@users.noreply.github.com> Date: Tue, 5 May 2026 12:41:18 -1000 Subject: [PATCH 5/7] =?UTF-8?q?chore:=20review=20fixes=20=E2=80=94=20stale?= =?UTF-8?q?=20MCP=20URL,=20URL=20validation,=20soften=20hardcoded=20counts?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Fix stale /sse path in afdocs-fix MCP example (matches PR #23 fix) - Add URL validation in afdocs_audit.mjs to prevent shell injection - Remove hardcoded baseline score from SKILL.md reporting instructions - Soften hardcoded page counts in known-exceptions.md (varies by run) - Scope page-size exception to /changelog/ only, not all large pages Co-Authored-By: Oz --- .agents/skills/afdocs-audit/SKILL.md | 2 +- .../skills/afdocs-audit/references/known-exceptions.md | 10 +++++----- .agents/skills/afdocs-audit/scripts/afdocs_audit.mjs | 10 ++++++++++ .agents/skills/afdocs-fix/SKILL.md | 5 ++--- 4 files changed, 18 insertions(+), 9 deletions(-) diff --git a/.agents/skills/afdocs-audit/SKILL.md b/.agents/skills/afdocs-audit/SKILL.md index 61aa3f0..8ead2ad 100644 --- a/.agents/skills/afdocs-audit/SKILL.md +++ b/.agents/skills/afdocs-audit/SKILL.md @@ -57,7 +57,7 @@ Only include a section if its count is > 0. Never list allowlisted issues under After running the audit, ALWAYS report the results to the user before taking any action. Include: -1. **Score**: Overall score, grade, and comparison to last known score (82/100 B as of 2026-05-05) +1. **Score**: Overall score and grade 2. **Failures first**: List every fail-severity check with its message and fix guidance. These are the most impactful. 3. **Warnings**: List warning-severity checks with context. 4. **Allowlisted**: Briefly note any known exceptions that were flagged. diff --git a/.agents/skills/afdocs-audit/references/known-exceptions.md b/.agents/skills/afdocs-audit/references/known-exceptions.md index ce3c09a..dee3589 100644 --- a/.agents/skills/afdocs-audit/references/known-exceptions.md +++ b/.agents/skills/afdocs-audit/references/known-exceptions.md @@ -1,17 +1,17 @@ # AFDocs Known Exceptions -This file lists checks that may flag as warnings or failures but are expected and intentional. When reporting audit results, classify these as "Allowlisted" rather than "Remaining." +This file lists checks from the afdocs-audit skill that may flag as warnings or failures but are expected and intentional. When reporting audit results, classify these as "Allowlisted" rather than "Remaining." ## content-start-position **Expected status**: fail or warn -**Reason**: 34/50 sampled pages have content starting past 50% of the HTML output. This is inherent to Starlight's layout — sidebar navigation, header markup, and JavaScript/CSS precede the `
` content area. +**Reason**: Sampled pages may have content starting past 50% of the HTML output. This is inherent to Starlight's layout — sidebar navigation, header markup, and JavaScript/CSS precede the `
` content area. **Mitigation**: The llms.txt directive, `` in ``, and `Accept: text/markdown` content negotiation middleware all steer agents to the clean markdown version, bypassing the HTML boilerplate entirely. **Action**: No fix needed. This is a structural property of Starlight sites. ## markdown-content-parity -**Expected status**: warn (7 pages, ~2% average difference) +**Expected status**: warn (several pages, ~2% average difference) **Reason**: False positive. The "missing" segments are numbered heading text like "2. Tabbed File Viewer" where Turndown correctly escapes the period (`### 2\. Tabbed File Viewer`) to prevent markdown parsers from interpreting it as a list item. The content IS present in the markdown — the AFDocs checker's text comparison doesn't account for markdown escaping. **Affected pages** (as of 2026-05-05): - `/agent-platform/cloud-agents/triggers/scheduled-agents-quickstart/` — step headings @@ -25,9 +25,9 @@ This file lists checks that may flag as warnings or failures but are expected an ## page-size-markdown / page-size-html -**Expected status**: warn (1 page between 50K-100K chars) +**Expected status**: warn — but only allowlist `/changelog/` **Reason**: The changelog page (`/changelog/`) is intentionally a single long page (~4,000 lines of MDX). It is excluded from `llms-full.txt` generation due to a `hast-util-to-text` stack overflow, but is still accessible at its URL and indexed by the sitemap. -**Action**: No fix needed unless the page grows significantly larger. Petra has indicated long pages are lower priority. +**Action**: If the only flagged page is `/changelog/`, classify as allowlisted. If other pages are flagged, treat those as genuine issues that may need splitting. ## section-header-quality diff --git a/.agents/skills/afdocs-audit/scripts/afdocs_audit.mjs b/.agents/skills/afdocs-audit/scripts/afdocs_audit.mjs index 5c01080..b49d369 100755 --- a/.agents/skills/afdocs-audit/scripts/afdocs_audit.mjs +++ b/.agents/skills/afdocs-audit/scripts/afdocs_audit.mjs @@ -52,6 +52,16 @@ function parseArgs(argv) { } function runAfdocsCheck(url) { + // Validate URL to prevent shell injection + try { + const parsed = new URL(url); + if (!['http:', 'https:'].includes(parsed.protocol)) { + throw new Error(`Invalid protocol: ${parsed.protocol}`); + } + } catch (e) { + throw new Error(`Invalid URL "${url}": ${e.message}`); + } + try { const stdout = execSync(`npx afdocs check ${url} --format json`, { encoding: 'utf8', diff --git a/.agents/skills/afdocs-fix/SKILL.md b/.agents/skills/afdocs-fix/SKILL.md index c34c497..e7f72ea 100644 --- a/.agents/skills/afdocs-fix/SKILL.md +++ b/.agents/skills/afdocs-fix/SKILL.md @@ -133,12 +133,11 @@ grep -A1 "label:" astro.config.mjs | grep "paths:" { "name": "Warp Documentation", "description": "Search and retrieve Warp documentation.", - "url": "https://warp.mcp.kapa.ai/sse", - "transport": "sse" + "url": "https://warp.mcp.kapa.ai" } ``` -This points to the existing Kapa-hosted MCP server. No custom implementation needed. +This points to the existing Kapa-hosted MCP server (OAuth-protected). No custom implementation needed. **Files**: `public/.well-known/mcp.json` (new file) From 695eb08198aa61af035176dc83b0ea5bacfeea84 Mon Sep 17 00:00:00 2001 From: Rachael Rose Renk <91027132+rachaelrenk@users.noreply.github.com> Date: Tue, 5 May 2026 12:57:41 -1000 Subject: [PATCH 6/7] fix: deploy middleware as Vercel Edge Function for content negotiation The Astro middleware in src/middleware.ts was not running at request time for pre-rendered (static) pages. The default middlewareMode ('classic') only executes middleware at build time for static pages, so Accept: text/markdown requests were served HTML from Vercel's CDN cache. Setting middlewareMode: 'edge' deploys the middleware as a separate Vercel Edge Function that runs at request time for ALL requests, including static pages. This enables content negotiation so agents sending Accept: text/markdown get clean markdown automatically. Co-Authored-By: Oz --- astro.config.mjs | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/astro.config.mjs b/astro.config.mjs index f66ae03..6b41d7a 100644 --- a/astro.config.mjs +++ b/astro.config.mjs @@ -163,5 +163,9 @@ export default defineConfig({ }), docsMarkdownIntegration(), ], - adapter: vercel(), + // Deploy Astro middleware as a Vercel Edge Function so it runs at + // request time for ALL pages, including pre-rendered static pages. + // Without this, middleware only runs at build time for static pages + // and content negotiation (Accept: text/markdown) doesn't work. + adapter: vercel({ middlewareMode: 'edge' }), }); From b03c411a7faba1f95f2cd67b2b17e47b2652c753 Mon Sep 17 00:00:00 2001 From: Rachael Rose Renk <91027132+rachaelrenk@users.noreply.github.com> Date: Tue, 5 May 2026 13:24:25 -1000 Subject: [PATCH 7/7] chore: revert Slack channel to placeholder per review feedback Reverts the hardcoded #growth-docs channel ID (C09BVK0PL3Y) back to a placeholder, matching the docs-seo-audit pattern. The channel is specified in the scheduled agent's prompt, not in the skill. Co-Authored-By: Oz --- .agents/skills/afdocs-audit/SKILL.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.agents/skills/afdocs-audit/SKILL.md b/.agents/skills/afdocs-audit/SKILL.md index 8ead2ad..9a74871 100644 --- a/.agents/skills/afdocs-audit/SKILL.md +++ b/.agents/skills/afdocs-audit/SKILL.md @@ -83,12 +83,12 @@ AFDocs audit complete: 23 checks run, score 82/100 (B). After reporting, ask the user which issues they want to address. -## Slack notification +## Slack notification (optional) -After the audit completes, post a summary to the **#growth-docs** Slack channel (`C09BVK0PL3Y`). +If instructed to send a report to Slack, post a summary after the audit completes. 1. Check if `BUZZ_SLACK_TOKEN` environment variable exists. -2. If the token exists, send a summary to `#growth-docs`. +2. If the token exists, send a summary to the channel the user specified (or the channel configured in the agent's instructions). **Format:** @@ -113,7 +113,7 @@ curl -X POST https://slack.com/api/chat.postMessage \ -H "Authorization: Bearer $BUZZ_SLACK_TOKEN" \ -H "Content-Type: application/json" \ -d '{ - "channel": "C09BVK0PL3Y", + "channel": "", "text": "", "unfurl_links": false, "unfurl_media": false