From 461408f519f95d9f406d26cc58fd635d1554c133 Mon Sep 17 00:00:00 2001 From: Mark S Date: Mon, 29 Jun 2026 09:15:01 -0400 Subject: [PATCH] chore: update and synchronize docs with source files --- README.md | 502 +++++++++++++++++++-------------- eval/README.md | 1 + reference/ast-plugin/README.md | 4 + 3 files changed, 297 insertions(+), 210 deletions(-) diff --git a/README.md b/README.md index 1dcc4f5..259d5f5 100644 --- a/README.md +++ b/README.md @@ -4,9 +4,10 @@ **4,000+ downloads and counting.** -**Zero dependencies. AST precision. 30+ framework detectors. 13 ORM parsers. 13 MCP tools. One `npx` call.** +**Zero dependencies. AST precision. 30+ framework detectors. 14 ORM parsers. 14 MCP tools. One `npx` call.** -**Works with TypeScript, JavaScript, Python, Go, Ruby, Elixir, Java, Kotlin, Rust, PHP, Dart, Swift, C#, and BrightScript/BrighterScript (Roku).** TypeScript projects get full AST precision. Everything else uses battle-tested regex detection across the same 30+ frameworks. +**Works with TypeScript, JavaScript, Python, Go, Ruby, Elixir, Java, Kotlin, Rust, PHP, Dart, Swift, C#, and BrightScript/BrighterScript ( +Roku).** TypeScript projects get full AST precision. Everything else uses battle-tested regex detection across the same 30+ frameworks. [![npm version](https://img.shields.io/npm/v/codesight?style=for-the-badge&logo=npm&color=CB3837)](https://www.npmjs.com/package/codesight) [![npm downloads](https://img.shields.io/npm/dm/codesight?style=for-the-badge&logo=npm&color=blue&label=Monthly%20Downloads)](https://www.npmjs.com/package/codesight) @@ -21,16 +22,18 @@ [![houseofmvps.com](https://img.shields.io/badge/houseofmvps.com-Website-green?style=for-the-badge&logo=google-chrome&logoColor=white)](https://houseofmvps.com) [![kailxlabs.co](https://img.shields.io/badge/kailxlabs.co-Website-6366F1?style=for-the-badge&logo=google-chrome&logoColor=white)](https://www.kailxlabs.co) -**Built by [Kailesk Khumar](https://www.linkedin.com/in/kailesk-khumar), founder of [HouseofMVPs](https://houseofmvps.com) and [Kailxlabs](https://www.kailxlabs.co)** +**Built by [Kailesk Khumar](https://www.linkedin.com/in/kailesk-khumar), founder of [HouseofMVPs](https://houseofmvps.com) +and [Kailxlabs](https://www.kailxlabs.co)** -*Also: [ultraship](https://github.com/Houseofmvps/ultraship) (39 expert skills for Claude Code) · [claude-rank](https://github.com/Houseofmvps/claude-rank) (SEO/GEO/AEO plugin for Claude Code)* +*Also: [ultraship](https://github.com/Houseofmvps/ultraship) (39 expert skills for Claude +Code) · [claude-rank](https://github.com/Houseofmvps/claude-rank) (SEO/GEO/AEO plugin for Claude Code)* --- ``` -0 dependencies · Node.js >= 18 · 27 tests · 13 MCP tools · MIT · tested on 25+ OSS projects across 14 languages +0 dependencies · Node.js >= 18 · 145 tests · 14 MCP tools · MIT · tested on 25+ OSS projects across 14 languages ``` ## Works With @@ -49,18 +52,19 @@ That's it. Run it in any project root. No config, no setup, no API keys. npx codesight --wiki # Generate wiki knowledge base (.codesight/wiki/) npx codesight --init # Generate CLAUDE.md, .cursorrules, codex.md, AGENTS.md npx codesight --open # Open interactive HTML report in browser -npx codesight --mcp # Start as MCP server (13 tools) for Claude Code / Cursor +npx codesight --mcp # Start as MCP server (14 tools) for Claude Code / Cursor npx codesight --blast src/lib/db.ts # Show blast radius for a file npx codesight --profile claude-code # Generate optimized config for a specific AI tool npx codesight --benchmark # Show detailed token savings breakdown -npx codesight --native-ast # Opt-in: use user-supplied WASM AST plugins (see docs/wasm-plugins.md) +npx codesight --native-ast # Opt-in: AST plugins for more languages (see docs/wasm-plugins.md) npx codesight --mode knowledge # Map knowledge base (.md notes → KNOWLEDGE.md) npx codesight --mode knowledge ~/vault # Map Obsidian vault, ADRs, meeting notes, retros ``` ## Wiki Knowledge Base (v1.6.2) -Inspired by [Karpathy's LLM wiki pattern](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) — but compiled from AST, not an LLM. Zero API calls. 200ms. +Inspired by [Karpathy's LLM wiki pattern](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) — but compiled from AST, not an +LLM. Zero API calls. 200ms. ```bash npx codesight --wiki @@ -84,29 +88,32 @@ Generates `.codesight/wiki/` — a persistent knowledge base of your codebase th Instead of loading the full 5K token context map every conversation, your AI reads one targeted article: -| Question | Without wiki | With wiki | -|---|---|---| -| "How does auth work?" | ~12K tokens (reads 8+ files) | ~300 tokens (`auth.md`) | -| "What models exist?" | ~5K tokens (CODESIGHT.md) | ~400 tokens (`database.md`) | -| New session start | ~5K tokens (full reload) | ~200 tokens (`index.md`) | +| Question | Without wiki | With wiki | +|-----------------------|------------------------------|-----------------------------| +| "How does auth work?" | ~12K tokens (reads 8+ files) | ~300 tokens (`auth.md`) | +| "What models exist?" | ~5K tokens (CODESIGHT.md) | ~400 tokens (`database.md`) | +| New session start | ~5K tokens (full reload) | ~200 tokens (`index.md`) | -**Persistent across sessions.** The wiki lives in `.codesight/wiki/`, committed to git. Every new Claude Code, Cursor, or Codex session starts with full codebase knowledge from the first message. +**Persistent across sessions.** The wiki lives in `.codesight/wiki/`, committed to git. Every new Claude Code, Cursor, or Codex session +starts with full codebase knowledge from the first message. **Auto-regenerates.** Use `--watch` to keep the wiki current as you code. Use `--hook` to regenerate on every commit. **3 new MCP tools** for wiki access: -| Tool | What it does | -|---|---| -| `codesight_get_wiki_index` | Get the wiki catalog (~200 tokens) at session start | -| `codesight_get_wiki_article` | Read one article by name: `auth`, `database`, `payments`, etc. | -| `codesight_lint_wiki` | Health check: orphan articles, missing cross-links, stale content | +| Tool | What it does | +|------------------------------|-------------------------------------------------------------------| +| `codesight_get_wiki_index` | Get the wiki catalog (~200 tokens) at session start | +| `codesight_get_wiki_article` | Read one article by name: `auth`, `database`, `payments`, etc. | +| `codesight_lint_wiki` | Health check: orphan articles, missing cross-links, stale content | -The key difference from general-purpose wiki tools: codesight already knows your routes, schema, blast radius, and middleware from AST — no LLM needed to extract code structure. The wiki is a narrative layer on top of data your codebase already contains. +The key difference from general-purpose wiki tools: codesight already knows your routes, schema, blast radius, and middleware from AST — no +LLM needed to extract code structure. The wiki is a narrative layer on top of data your codebase already contains. ## Knowledge Mode (v1.9.3) -Not just code — your decisions, meeting notes, ADRs, and retrospectives carry as much context as the codebase itself. `--mode knowledge` maps them the same way codesight maps code. +Not just code — your decisions, meeting notes, ADRs, and retrospectives carry as much context as the codebase itself. `--mode knowledge`maps +them the same way codesight maps code. ```bash npx codesight --mode knowledge # Scan current directory for .md files @@ -118,41 +125,49 @@ Outputs `.codesight/KNOWLEDGE.md` — a compact AI context primer: ```markdown # Knowledge Map — my-project + > 47 notes · 12 decisions · 8 open questions · 2025-09-01 → 2026-04-01 ## Key Decisions (12) + - [2026-03-20] Going with Polar.sh over Stripe Connect — simpler global payments - [2026-03-15] Decided to use PostgreSQL — better JSON support and Drizzle compatibility - [2026-02-10] Will use Redis for rate limiting — BullMQ already in stack ## Open Questions (8) + - Should we support PayPal later? - When do we start the Stripe marketplace application? ## Note Index (47) ### Decision Records (8) + - `decisions/adr-002-payments.md` — 2026-03-20 — Going with Polar.sh over Stripe Connect - `decisions/adr-001-database.md` — 2026-03-15 — We need a relational database... ### Meeting Notes (14) + ### Retrospectives (6) + ### Specs & PRDs (5) + ### Research (4) ``` **What it detects automatically:** -| Note type | Signals | -|---|---| +| Note type | Signals | +|------------------|--------------------------------------------------------------------------| | Decision records | ADR format (`## Decision`), "decided to", "going with", "chose X over Y" | -| Meeting notes | `Attendees:`, `Action items:`, filename: `standup`, `sync`, `1on1` | -| Retrospectives | "What went well", "Stop doing", filename: `retro`, `retrospective` | -| Specs / PRDs | `## Goals`, `## Requirements`, filename: `prd`, `spec`, `roadmap` | -| Research | filename: `research`, `analysis`, `benchmark`, `comparison` | -| Session logs | filename: `session`, `daily`, `weekly` | +| Meeting notes | `Attendees:`, `Action items:`, filename: `standup`, `sync`, `1on1` | +| Retrospectives | "What went well", "Stop doing", filename: `retro`, `retrospective` | +| Specs / PRDs | `## Goals`, `## Requirements`, filename: `prd`, `spec`, `roadmap` | +| Research | filename: `research`, `analysis`, `benchmark`, `comparison` | +| Session logs | filename: `session`, `daily`, `weekly` | **Supports:** + - Obsidian vaults (YAML frontmatter, `[[backlinks]]`, `#tags`) - Notion exports (`.md` files with frontmatter) - ADR tooling (`adr-tools`, `Log4brains`, raw markdown) @@ -169,25 +184,31 @@ CI: add `npx codesight --mode knowledge` alongside your existing codesight step. ## Benchmarks (Real Projects) -Every number below comes from running codesight on real production codebases — both small SaaS projects (v1.6.2) and large open-source platforms with 4K–10K+ files (v1.6.4). Output tokens are measured from actual file size (chars / 4). Exploration tokens are estimated from what was extracted — routes × 400, models × 300, components × 250, etc. Route counts and model counts are cross-checked against actual source files. +Every number below comes from running codesight on real production codebases — both small SaaS projects (v1.6.2) and large open-source +platforms with 4K–10K+ files (v1.6.4). Output tokens are measured from actual file size (chars / 4). Exploration tokens are estimated from +what was extracted — routes × 400, models × 300, components × 250, etc. Route counts and model counts are cross-checked against actual +source files. ### Three-Level Token Reduction codesight saves tokens at two distinct layers. The wiki (v1.6.2) adds a second layer on top of the base savings: -| Project | Manual exploration | codesight scan | codesight --wiki (targeted) | **Total reduction** | -|---|---|---|---|---| -| **SaaS A** | 46,020 tokens | 3,936 tokens (11.7x) | ~550 tokens | **83.7x** | -| **SaaS B** | 26,130 tokens | 3,629 tokens (7.2x) | ~440 tokens | **59.4x** | -| **SaaS C** | 47,450 tokens | 4,162 tokens (11.4x) | ~360 tokens | **131.8x** | +| Project | Manual exploration | codesight scan | codesight --wiki (targeted) | **Total reduction** | +|------------|--------------------|----------------------|-----------------------------|---------------------| +| **SaaS A** | 46,020 tokens | 3,936 tokens (11.7x) | ~550 tokens | **83.7x** | +| **SaaS B** | 26,130 tokens | 3,629 tokens (7.2x) | ~440 tokens | **59.4x** | +| **SaaS C** | 47,450 tokens | 4,162 tokens (11.4x) | ~360 tokens | **131.8x** | -**Average combined reduction: 91x.** The wiki's "targeted" number = reading `index.md` at session start (~200 tokens) + one relevant article (~160-350 tokens depending on project). Your AI never loads the full context map for targeted questions. +**Average combined reduction: 91x.** The wiki's "targeted" number = reading `index.md` at session start (~200 tokens) + one relevant +article (~160-350 tokens depending on project). Your AI never loads the full context map for targeted questions. The two savings layers are independent and compound: -**Layer 1 — codesight scan** eliminates manual file exploration. Instead of your AI running glob/grep/read across 40-138 files to understand the project, it reads one pre-compiled map. +**Layer 1 — codesight scan** eliminates manual file exploration. Instead of your AI running glob/grep/read across 40-138 files to understand +the project, it reads one pre-compiled map. -**Layer 2 — `--wiki`** eliminates loading the full map for every question. Instead of loading 3K-5K tokens of full context at session start, your AI reads a 200-token index and pulls the one relevant article (~160-350 tokens) for each question. +**Layer 2 — `--wiki`** eliminates loading the full map for every question. Instead of loading 3K-5K tokens of full context at session start, +your AI reads a 200-token index and pulls the one relevant article (~160-350 tokens) for each question. ``` Without codesight: AI reads 26K-47K tokens per session exploring files @@ -197,73 +218,81 @@ With --wiki: AI reads ~200 tokens at start + ~300 per targeted question ### Base Scan Results -| Project | Stack | Files | Routes | Models | Components | Output Tokens | Exploration Tokens | Savings | Scan Time | -|---|---|---|---|---|---|---|---|---|---| -| **SaaS A** | Hono + Drizzle | 138 | 38 | 12 | 0 | 3,936 | 46,020 | **11.7x** | 186ms | -| **SaaS B** | Hono + Drizzle, 3 workspaces | 53 | 17 | 8 | 10 | 3,629 | 26,130 | **7.2x** | 201ms | -| **SaaS C** | FastAPI + MongoDB | 40 | 56 | 0 | 0 | 4,162 | 47,450 | **11.4x** | 890ms | +| Project | Stack | Files | Routes | Models | Components | Output Tokens | Exploration Tokens | Savings | Scan Time | +|------------|------------------------------|-------|--------|--------|------------|---------------|--------------------|-----------|-----------| +| **SaaS A** | Hono + Drizzle | 138 | 38 | 12 | 0 | 3,936 | 46,020 | **11.7x** | 186ms | +| **SaaS B** | Hono + Drizzle, 3 workspaces | 53 | 17 | 8 | 10 | 3,629 | 26,130 | **7.2x** | 201ms | +| **SaaS C** | FastAPI + MongoDB | 40 | 56 | 0 | 0 | 4,162 | 47,450 | **11.4x** | 890ms | -SaaS C has 0 models because it uses MongoDB — no SQL ORM declarations for codesight to parse. This is correct detection, not a false negative. +SaaS C has 0 models because it uses MongoDB — no SQL ORM declarations for codesight to parse. This is correct detection, not a false +negative. ![Token comparison: Without codesight (46K-66K tokens) vs With codesight (3K-5K tokens)](assets/token-comparison.jpg) ### Multi-Language OSS Benchmark (v1.6.7) -Tested against real open-source codebases spanning every supported language and framework. Output tokens are measured from actual file size. Exploration tokens are estimated (routes×400 + models×300 + components×250 + revisit multiplier). Zero false positives across all tests. - -| Language | Stack | Files | Routes | Models | Components | Output tokens | Est. exploration | Savings | -|---|---|---|---|---|---|---|---|---| -| **TypeScript · Next.js** | Next.js + tRPC + Prisma · 110+ workspaces | 7,509 | 479 | 173 | 1,309 | 158,660 | ~1,485,000 | **~9x** | -| **TypeScript · NestJS** | NestJS + TypeORM + Mongoose | 162 | 19 | 8 | 0 | 5,300 | ~67,500 | **~12.7x** | -| **TypeScript · Hono** | Hono | — | 8 | 0 | 0 | — | — | ✓ | -| **TypeScript · Remix** | Remix + Prisma | 36 | 11 | 0 | 9 | — | — | ✓ | -| **TypeScript · SvelteKit** | SvelteKit | — | 0³ | 0 | 23 | — | — | ✓ | -| **TypeScript · Nuxt** | Nuxt | 141 | 8 | 0 | 64 | — | — | ✓ | -| **JavaScript · Express** | Express + Mongoose | 51 | 10 | 5 | 0 | 1,241 | ~20,800 | **~17x** | -| **Ruby · Rails** | Rails + ActiveRecord | 4,172 | 607 | 116 | 0 | 21,711 | ~386,100 | **~17.8x** | -| **PHP · Laravel** | Laravel + Eloquent | 3,896 | 652 | 59 | 0 | 30,739 | ~493,285 | **~16x** | -| **Python · Django** | Django + pyproject.toml | 4,232 | 7¹ | 56 | 0 | 83,842 | ~631,020 | **~7.5x** | -| **Python · Flask** | Flask + SQLAlchemy | 30 | 12 | 5 | 0 | 1,148 | ~16,705 | **~14.5x** | -| **Python · FastAPI** | FastAPI + SQLModel (monorepo) | 143 | 21 | 2 | 36 | 2,487 | ~38,090 | **~15.3x** | -| **Elixir · Phoenix** | Phoenix + Ecto | 1,406 | 198 | 54 | 0 | 9,589 | ~152,100 | **~15.9x** | -| **Go · Gin** | Gin + GORM (enterprise app) | 388 | 202 | 169 | 0 | 15,266 | ~262,730 | **~17.2x** | -| **Go · Echo** | Echo | — | 7 | 0 | 0 | — | — | ✓ | -| **Go · Fiber** | Fiber | — | 5 | 0 | 0 | — | — | ✓ | -| **Rust · Actix** | actix-web | 528 | 30 | 0 | 0 | 1,355 | ~27,170 | **~20x** | -| **Rust · Axum** | Axum | — | 6 | 0 | 0 | — | — | ✓ | -| **C# · ASP.NET** | ASP.NET Core + Entity Framework Core | 256 | 13 | 7 | 0 | 5,126 | ~63,570 | **~12.4x** | -| **Java · Spring** | Spring Boot + Java (Maven) | 47 | 16 | 0 | 0 | 319 | ~13,208 | **~41x**² | -| **Swift · SwiftUI** | SwiftUI | 388 | 0 | 0 | 62 | 7,499 | ~76,830 | **~10.2x** | -| **Swift · Vapor** | Vapor backend | 294 | 81 | 0 | 0 | 6,146 | ~95,160 | **~15.5x** | -| **Dart · Flutter** | Flutter + go_router | 204 | 10 | 0 | 89 | 8,500 | ~86,125 | **~10.1x** | +Tested against real open-source codebases spanning every supported language and framework. Output tokens are measured from actual file size. +Exploration tokens are estimated (routes×400 + models×300 + components×250 + revisit multiplier). Zero false positives across all tests. + +| Language | Stack | Files | Routes | Models | Components | Output tokens | Est. exploration | Savings | +|----------------------------|-------------------------------------------|-------|--------|--------|------------|---------------|------------------|------------| +| **TypeScript · Next.js** | Next.js + tRPC + Prisma · 110+ workspaces | 7,509 | 479 | 173 | 1,309 | 158,660 | ~1,485,000 | **~9x** | +| **TypeScript · NestJS** | NestJS + TypeORM + Mongoose | 162 | 19 | 8 | 0 | 5,300 | ~67,500 | **~12.7x** | +| **TypeScript · Hono** | Hono | — | 8 | 0 | 0 | — | — | ✓ | +| **TypeScript · Remix** | Remix + Prisma | 36 | 11 | 0 | 9 | — | — | ✓ | +| **TypeScript · SvelteKit** | SvelteKit | — | 0³ | 0 | 23 | — | — | ✓ | +| **TypeScript · Nuxt** | Nuxt | 141 | 8 | 0 | 64 | — | — | ✓ | +| **JavaScript · Express** | Express + Mongoose | 51 | 10 | 5 | 0 | 1,241 | ~20,800 | **~17x** | +| **Ruby · Rails** | Rails + ActiveRecord | 4,172 | 607 | 116 | 0 | 21,711 | ~386,100 | **~17.8x** | +| **PHP · Laravel** | Laravel + Eloquent | 3,896 | 652 | 59 | 0 | 30,739 | ~493,285 | **~16x** | +| **Python · Django** | Django + pyproject.toml | 4,232 | 7¹ | 56 | 0 | 83,842 | ~631,020 | **~7.5x** | +| **Python · Flask** | Flask + SQLAlchemy | 30 | 12 | 5 | 0 | 1,148 | ~16,705 | **~14.5x** | +| **Python · FastAPI** | FastAPI + SQLModel (monorepo) | 143 | 21 | 2 | 36 | 2,487 | ~38,090 | **~15.3x** | +| **Elixir · Phoenix** | Phoenix + Ecto | 1,406 | 198 | 54 | 0 | 9,589 | ~152,100 | **~15.9x** | +| **Go · Gin** | Gin + GORM (enterprise app) | 388 | 202 | 169 | 0 | 15,266 | ~262,730 | **~17.2x** | +| **Go · Echo** | Echo | — | 7 | 0 | 0 | — | — | ✓ | +| **Go · Fiber** | Fiber | — | 5 | 0 | 0 | — | — | ✓ | +| **Rust · Actix** | actix-web | 528 | 30 | 0 | 0 | 1,355 | ~27,170 | **~20x** | +| **Rust · Axum** | Axum | — | 6 | 0 | 0 | — | — | ✓ | +| **C# · ASP.NET** | ASP.NET Core + Entity Framework Core | 256 | 13 | 7 | 0 | 5,126 | ~63,570 | **~12.4x** | +| **Java · Spring** | Spring Boot + Java (Maven) | 47 | 16 | 0 | 0 | 319 | ~13,208 | **~41x**² | +| **Swift · SwiftUI** | SwiftUI | 388 | 0 | 0 | 62 | 7,499 | ~76,830 | **~10.2x** | +| **Swift · Vapor** | Vapor backend | 294 | 81 | 0 | 0 | 6,146 | ~95,160 | **~15.5x** | +| **Dart · Flutter** | Flutter + go_router | 204 | 10 | 0 | 89 | 8,500 | ~86,125 | **~10.1x** | ¹ Django project is GraphQL-first — 7 REST utility endpoints detected accurately, 0 false positives. ² High ratio on small boilerplate: Spring Boot route metadata compresses very well. ³ SvelteKit RealWorld app uses page routes (`+page.svelte`), not JSON API endpoints (`+server.ts`). 0 routes is correct. -**How exploration tokens are estimated:** `routes×400 + models×300 + components×250 + hot_files×150 + env_vars×30`, times a 1.3 revisit multiplier, minus the output size. This approximates what an AI would spend asking "what routes exist?", "show me the schema", etc. in a manual exploration session. Output token count is the actual measured file size. +**How exploration tokens are estimated:** `routes×400 + models×300 + components×250 + hot_files×150 + env_vars×30`, times a 1.3 revisit +multiplier, minus the output size. This approximates what an AI would spend asking "what routes exist?", "show me the schema", etc. in a +manual exploration session. Output token count is the actual measured file size. ### Wiki Breakdown (v1.6.2) -| Project | Full CODESIGHT.md | Wiki index only | Index + 1 article | Wiki articles generated | -|---|---|---|---|---| -| **SaaS A** | 3,936 tokens | ~200 tokens | ~550 tokens | 9 | -| **SaaS B** | 3,629 tokens | ~200 tokens | ~440 tokens | 11 | -| **SaaS C** | 4,162 tokens | ~200 tokens | ~360 tokens | 17 | +| Project | Full CODESIGHT.md | Wiki index only | Index + 1 article | Wiki articles generated | +|------------|-------------------|-----------------|-------------------|-------------------------| +| **SaaS A** | 3,936 tokens | ~200 tokens | ~550 tokens | 9 | +| **SaaS B** | 3,629 tokens | ~200 tokens | ~440 tokens | 11 | +| **SaaS C** | 4,162 tokens | ~200 tokens | ~360 tokens | 17 | -"How does auth work?" — without wiki: loads 3,945 tokens. With wiki: reads `auth.md` (~350 tokens). **11x improvement per targeted question, 84x total vs manual.** +"How does auth work?" — without wiki: loads 3,945 tokens. With wiki: reads `auth.md` (~350 tokens). **11x improvement per targeted question, +84x total vs manual.** ### Detection Accuracy -Verified against actual source files. Route counts cross-checked against route definitions; schema models cross-checked against ORM table declarations. +Verified against actual source files. Route counts cross-checked against route definitions; schema models cross-checked against ORM table +declarations. -| Project | Route Recall | Schema Recall | False Positives | Detection Method | -|---|---|---|---|---| -| **SaaS A** | 38/43 (88%) | 12/12 (100%) | 0 | Schema: AST (Drizzle), Routes: AST (Hono) | -| **SaaS B** | 17/17 (100%) | 8/8 (100%) | 0 | Full AST (Hono + Drizzle + React) | -| **SaaS C** | 56/59 (~95%) | 0/0 (correct) | 0 | AST (FastAPI + MongoDB) | +| Project | Route Recall | Schema Recall | False Positives | Detection Method | +|------------|--------------|---------------|-----------------|-------------------------------------------| +| **SaaS A** | 38/43 (88%) | 12/12 (100%) | 0 | Schema: AST (Drizzle), Routes: AST (Hono) | +| **SaaS B** | 17/17 (100%) | 8/8 (100%) | 0 | Full AST (Hono + Drizzle + React) | +| **SaaS C** | 56/59 (~95%) | 0/0 (correct) | 0 | AST (FastAPI + MongoDB) | -SaaS A's 5 missed routes use dynamic `url.match(/pattern/)` inside request handlers — a developer pattern that static analysis cannot resolve at scan time. This is an inherent limit of static analysis, not a framework gap. SaaS C missed an estimated 3 of 59 FastAPI routes. Zero false positives across all three projects. +SaaS A's 5 missed routes use dynamic `url.match(/pattern/)` inside request handlers — a developer pattern that static analysis cannot +resolve at scan time. This is an inherent limit of static analysis, not a framework gap. SaaS C missed an estimated 3 of 59 FastAPI routes. +Zero false positives across all three projects. ### Blast Radius Accuracy @@ -278,13 +307,13 @@ Tested on a production SaaS: changing the database module correctly identified: Measured across the three benchmark projects: -| Detector | SaaS A (138 files) | SaaS B (53 files) | SaaS C (40 files) | -|---|---|---|---| -| **Routes** | 38 | 17 | 56 | -| **Schema models** | 12 | 8 | 0 | -| **Components** | 0 | 10 | 0 | -| **Env vars** | 12 | 7 | 15 | -| **Hot files** | 20 | 20 | 20 | +| Detector | SaaS A (138 files) | SaaS B (53 files) | SaaS C (40 files) | +|-------------------|--------------------|-------------------|-------------------| +| **Routes** | 38 | 17 | 56 | +| **Schema models** | 12 | 8 | 0 | +| **Components** | 0 | 10 | 0 | +| **Env vars** | 12 | 7 | 15 | +| **Hot files** | 20 | 20 | 20 | --- @@ -294,7 +323,8 @@ Measured across the three benchmark projects: ![8 parallel detectors: Routes, Schema, Components, Dep Graph, Middleware, Config, Libraries, Contracts](assets/detectors.jpg) -codesight runs all 8 detectors in parallel, then writes the results as structured markdown. The output is designed to be read by an AI in a single file load. +codesight runs all 8 detectors in parallel, then writes the results as structured markdown. The output is designed to be read by an AI in a +single file load. ## What It Generates @@ -321,19 +351,20 @@ section when there's something to report. ## AST Precision -When TypeScript is installed in the project being scanned, codesight uses the actual TypeScript compiler API to parse your code structurally. No regex guessing. +When TypeScript is installed in the project being scanned, codesight uses the actual TypeScript compiler API to parse your code +structurally. No regex guessing. ![AST precision: TypeScript available → AST Parse, otherwise Regex fallback](assets/ast-precision.jpg) -| What AST enables | Regex alone | -|---|---| -| Follows `router.use('/prefix', subRouter)` chains | Misses nested routers | -| Combines `@Controller('users')` + `@Get(':id')` into `/users/:id` | May miss prefix | -| Parses `router({ users: userRouter })` tRPC nesting | Line-by-line matching | -| Extracts exact Drizzle field types from `.primaryKey().notNull()` chains | Pattern matching | -| Gets React props from TypeScript interfaces and destructuring | Regex on `{ prop }` | -| Detects middleware in route chains: `app.get('/path', auth, handler)` | Not captured | -| Filters out non-route calls like `c.get('userId')` | May false-positive | +| What AST enables | Regex alone | +|--------------------------------------------------------------------------|-----------------------| +| Follows `router.use('/prefix', subRouter)` chains | Misses nested routers | +| Combines `@Controller('users')` + `@Get(':id')` into `/users/:id` | May miss prefix | +| Parses `router({ users: userRouter })` tRPC nesting | Line-by-line matching | +| Extracts exact Drizzle field types from `.primaryKey().notNull()` chains | Pattern matching | +| Gets React props from TypeScript interfaces and destructuring | Regex on `{ prop }` | +| Detects middleware in route chains: `app.get('/path', auth, handler)` | Not captured | +| Filters out non-route calls like `c.get('userId')` | May false-positive | AST detection is reported in the output: @@ -341,18 +372,36 @@ AST detection is reported in the output: Analyzing... done (AST: 60 routes, 18 models, 16 components) ``` -No configuration needed. If TypeScript is in your `node_modules`, AST kicks in automatically. Works with npm, yarn, and pnpm (including strict mode). Falls back to regex for non-TypeScript projects or frameworks without AST support. +No configuration needed. If TypeScript is in your `node_modules`, AST kicks in automatically. Works with npm, yarn, and pnpm (including +strict mode). Falls back to regex for non-TypeScript projects or frameworks without AST support. -**AST-supported frameworks:** Express, Hono, Fastify, Koa, Elysia (route chains + middleware), NestJS (decorator combining + guards), tRPC (router nesting + procedure types), Drizzle (field chains + relations), TypeORM (entity decorators), React (props from interfaces + destructuring + forwardRef/memo). +**AST-supported frameworks:** Express, Hono, Fastify, Koa, Elysia (route chains + middleware), NestJS (decorator combining + guards), tRPC ( +router nesting + procedure types), Drizzle (field chains + relations), TypeORM (entity decorators), React (props from interfaces + +destructuring + forwardRef/memo). ### Native-AST WASM plugins (opt-in) -codesight can hand extraction off to an optional, user-provided WebAssembly -plugin — configured with `--native-ast` (or `--native-ast-strict`). codesight -does not currently ship any WASM-based plugins itself; it only supports the -possibility of user-supplied plugins, and falls back to its built-in extractors -when none are found. To build a conforming plugin, see the contract -reference: [`docs/wasm-plugins.md`](docs/wasm-plugins.md). +By default codesight uses its built-in extractors (AST for TypeScript, regex for everything else). Opt in with `--native-ast` to specify +that **WebAssembly plugins** should be used to acquire full-AST precision fir non-TypeScript source files: + +```bash +npx codesight --native-ast # use every discovered plugin (additive) +npx codesight --native-ast=rust,go # only these languages (authoritative for their files) +npx codesight --native-ast=none # force off (overrides config) +npx codesight --native-ast-strict # like --native-ast, but fail if a named plugin is missing +npx codesight --plugin-dir ./wasm # extra directory to search for plugins +``` + +Dispatch to WASM plugins is **language-driven**: each plugin self-describes (via a `describe()` export) the file extensions it handles, so +*any* language works — not just the ones with built-in detectors. Where no plugin handles a file, built-in extraction stands; with +`--native-ast` unset, behavior is byte-identical to existing `codesight` behavior. + +**codesight's npm package ships no plugins** — they are separate, opt-in artifacts. The project publishes prebuilt reference plugins (Rust/ +`syn`, Python/`ruff`, Go/`go/parser`) as checksummed GitHub release assets; drop the ones you want to use into `~/.codesight/plugins/` (or +point `--plugin-dir` at them), then enable them with `--native-ast`. Alternately, you may build your own plugin for *any language* against +the contract. + +→ Full contract, discovery rules, and download/install steps: [`docs/wasm-plugins.md`](docs/wasm-plugins.md). ## Built-in plugins @@ -360,17 +409,17 @@ Three first-party plugins run on every scan and surface context the main pass misses because it lives in skipped dotfile directories. Each is inert until its target files exist, so they cost nothing on projects that don't use them: -| Plugin | Reads | Output | -|---|---|---| -| `cicd` | `.github/workflows/`, `.circleci/` | Pipeline triggers, jobs, secrets, deploy targets | -| `githooks` | lefthook / husky config, `.git/hooks/` | Which commands run on which git lifecycle | -| `skills` | `.claude/commands/`, `.claude/skills/` | Available slash commands / agent skills + descriptions | +| Plugin | Reads | Output | +|------------|----------------------------------------|--------------------------------------------------------| +| `cicd` | `.github/workflows/`, `.circleci/` | Pipeline triggers, jobs, secrets, deploy targets | +| `githooks` | lefthook / husky config, `.git/hooks/` | Which commands run on which git lifecycle | +| `skills` | `.claude/commands/`, `.claude/skills/` | Available slash commands / agent skills + descriptions | Opt out of any of them per project: ```js // codesight.config.js -export default { disableDetectors: ["cicd", "githooks", "skills"] }; +export default {disableDetectors: ["cicd", "githooks", "skills"]}; ``` **Terraform is opt-in**, not auto-loaded — it deliberately reaches outside the @@ -379,13 +428,15 @@ explicit service name, so it stays off until you ask for it: ```js // codesight.config.js -import { createTerraformPlugin } from "codesight/plugins/terraform"; -export default { plugins: [createTerraformPlugin({ infraPath: "../infra" })] }; +import {createTerraformPlugin} from "codesight/plugins/terraform"; + +export default {plugins: [createTerraformPlugin({infraPath: "../infra"})]}; ``` ## Routes -Not just paths. Methods, URL parameters, what each route touches (auth, database, cache, payments, AI, email, queues), and where the handler lives. Detects routes across 25+ frameworks automatically. +Not just paths. Methods, URL parameters, what each route touches (auth, database, cache, payments, AI, email, queues), and where the handler +lives. Detects routes across 30+ frameworks automatically. Example output: @@ -400,12 +451,14 @@ Example output: ## Schema -Models, fields, types, primary keys, foreign keys, unique constraints, relations. Parsed directly from your ORM definitions via AST. No need to open migration files. +Models, fields, types, primary keys, foreign keys, unique constraints, relations. Parsed directly from your ORM definitions via AST. No need +to open migration files. Example output: ```markdown ### user + - id: text (pk) - name: text (required) - email: text (unique, required) @@ -413,6 +466,7 @@ Example output: - stripeCustomerId: text (fk) ### project + - id: uuid (default, pk) - ownerId: text (fk, required) - name: text (required) @@ -428,6 +482,7 @@ Example output: ```markdown ## Most Imported Files (change these carefully) + - `src/types/index.ts` — imported by **20** files - `src/db/index.ts` — imported by **12** files - `src/lib/auth.ts` — imported by **8** files @@ -524,22 +579,25 @@ Example output (SaaS A — 138 files, Hono + Drizzle): Each detector type maps to a measured token cost that an AI would spend to discover the same information manually: -| What codesight finds | Tokens saved per item | Why | -|---|---|---| -| Each route | ~400 tokens | AI reads the handler file, greps for the path, reads middleware | -| Each schema model | ~300 tokens | AI opens migration/ORM files, parses fields manually | -| Each component | ~250 tokens | AI opens component files, reads prop types | -| Each library export | ~200 tokens | AI greps for exports, reads signatures | -| Each env var | ~100 tokens | AI greps for `process.env`, reads .env files | -| Each file scanned | ~80 tokens | AI runs glob/grep operations to find relevant files | +| What codesight finds | Tokens saved per item | Why | +|----------------------|-----------------------|-----------------------------------------------------------------| +| Each route | ~400 tokens | AI reads the handler file, greps for the path, reads middleware | +| Each schema model | ~300 tokens | AI opens migration/ORM files, parses fields manually | +| Each component | ~250 tokens | AI opens component files, reads prop types | +| Each library export | ~200 tokens | AI greps for exports, reads signatures | +| Each env var | ~100 tokens | AI greps for `process.env`, reads .env files | +| Each file scanned | ~80 tokens | AI runs glob/grep operations to find relevant files | -The 1.3x multiplier accounts for AI revisiting files during multi-turn conversations. These estimates are conservative. A developer manually verified that Claude Code spends 40-70K tokens exploring the same projects that codesight summarizes in 3-5K tokens. +The 1.3x multiplier accounts for AI revisiting files during multi-turn conversations. These estimates are conservative. A developer manually +verified that Claude Code spends 40-70K tokens exploring the same projects that codesight summarizes in 3-5K tokens. ## Roku / BrightScript / SceneGraph -codesight treats Roku channels as first-class projects. The `manifest` file at the channel root anchors detection — the same file Roku itself uses to identify a channel, so zero configuration is needed for the common case. +codesight treats Roku channels as first-class projects. The `manifest` file at the channel root anchors detection — the same file Roku +itself uses to identify a channel, so zero configuration is needed for the common case. -**Standard single-channel layout** (about 90% of Roku repos, matches the Roku docs' getting-started template and projects like `rokucommunity/brighterscript-template`): +**Standard single-channel layout** (about 90% of Roku repos, matches the Roku docs' getting-started template and projects like +`rokucommunity/brighterscript-template`): ``` / @@ -549,9 +607,11 @@ codesight treats Roku channels as first-class projects. The `manifest` file at t images/ ``` -codesight also recognizes the `rokucommunity/brighterscript-template` layout where the channel lives under `src/` and the root carries a `bsconfig.json` for BrighterScript tooling. +codesight also recognizes the `rokucommunity/brighterscript-template` layout where the channel lives under `src/` and the root carries a +`bsconfig.json` for BrighterScript tooling. -**Multi-channel monorepo layout** (less common — used by larger codebases that ship several branded channels from one repo with `roku-deploy` + `gulp` to merge a shared `common/` layer with per-channel assets at build time): +**Multi-channel monorepo layout** (less common — used by larger codebases that ship several branded channels from one repo with +`roku-deploy` + `gulp` to merge a shared `common/` layer with per-channel assets at build time): ``` / @@ -565,28 +625,35 @@ codesight also recognizes the `rokucommunity/brighterscript-template` layout whe manifest ``` -This is detected via a strict structural signal: no manifest at root, `roku-deploy` in deps, and a `common/` directory with at least 2 sibling directories that each have their own `manifest`. When the signal matches, each channel (plus `common/`) is registered as a workspace. +This is detected via a strict structural signal: no manifest at root, `roku-deploy` in deps, and a `common/` directory with at least 2 +sibling directories that each have their own `manifest`. When the signal matches, each channel (plus `common/`) is registered as a +workspace. ### Mappings to codesight's data model -| codesight concept | Roku equivalent | -|---|---| -| Routes | Screens — every child element with an `id` declared in the Scene XML's ``. `method = VIEW` by default, upgraded to `MODAL` if a navigation call-site passes a literal `true` as the second argument. | -| Schema | Every SceneGraph component XML whose `` has at least one `` — the typed contract is the model. | -| Components | Every `` XML (views, tasks, scenes, modals). Props = interface fields. | -| Libraries | `.brs` / `.bs` files outside `components/` — top-level `function`/`sub` plus BrighterScript `class` / `namespace` / `enum` / `interface`. | -| Middleware | `observeField` subscriptions, `m.global.AddField` registrations. BugsnagTask / RudderstackTask recognized when present. | -| Dependencies | `