Add eyeball plugin: document analysis with inline source screenshots 🤖🤖🤖

.github/plugin/marketplace.json

@@ -212,6 +212,12 @@
       "description": "Task Researcher and Task Planner for intermediate to expert users and large codebases - Brought to you by microsoft/edge-ai",
       "version": "1.0.0"
     },
+    {
+      "name": "eyeball",
+      "source": "eyeball",
+      "description": "Document analysis with inline source screenshots. When you ask Copilot to analyze a document, Eyeball generates a Word doc where every factual claim includes a highlighted screenshot from the source material so you can verify it with your own eyes.",
+      "version": "1.0.0"
+    },
     {
       "name": "fastah-ip-geo-tools",
       "source": "fastah-ip-geo-tools",

docs/README.plugins.md

@@ -41,6 +41,7 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-plugins) for guidelines on how t
 | [devops-oncall](../plugins/devops-oncall/README.md) | A focused set of prompts, instructions, and a chat mode to help triage incidents and respond quickly with DevOps tools and Azure resources. | 3 items | devops, incident-response, oncall, azure |
 | [doublecheck](../plugins/doublecheck/README.md) | Three-layer verification pipeline for AI output. Extracts claims, finds sources, and flags hallucination risks so humans can verify before acting. | 2 items | verification, hallucination, fact-check, source-citation, trust, safety |
 | [edge-ai-tasks](../plugins/edge-ai-tasks/README.md) | Task Researcher and Task Planner for intermediate to expert users and large codebases - Brought to you by microsoft/edge-ai | 2 items | architecture, planning, research, tasks, implementation |
+| [eyeball](../plugins/eyeball/README.md) | Document analysis with inline source screenshots. When you ask Copilot to analyze a document, Eyeball generates a Word doc where every factual claim includes a highlighted screenshot from the source material so you can verify it with your own eyes. | 1 items | document-analysis, citation-verification, screenshot, contracts, legal, trust, visual-verification |
 | [fastah-ip-geo-tools](../plugins/fastah-ip-geo-tools/README.md) | This plugin is for network operations engineers who wish to tune and publish IP geolocation feeds in RFC 8805 format. It consists of an AI Skill and an associated MCP server that geocodes geolocation place names to real cities for accuracy. | 1 items | geofeed, ip-geolocation, rfc-8805, rfc-9632, network-operations, isp, cloud, hosting, ixp |
 | [flowstudio-power-automate](../plugins/flowstudio-power-automate/README.md) | Complete toolkit for managing Power Automate cloud flows via the FlowStudio MCP server. Includes skills for connecting to the MCP server, debugging failed flow runs, and building/deploying flows from natural language. | 3 items | power-automate, power-platform, flowstudio, mcp, model-context-protocol, cloud-flows, workflow-automation |
 | [frontend-web-dev](../plugins/frontend-web-dev/README.md) | Essential prompts, instructions, and chat modes for modern frontend web development including React, Angular, Vue, TypeScript, and CSS frameworks. | 4 items | frontend, web, react, typescript, javascript, css, html, angular, vue |

docs/README.skills.md

@@ -131,6 +131,7 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-skills) for guidelines on how to
 | [entra-agent-user](../skills/entra-agent-user/SKILL.md) | Create Agent Users in Microsoft Entra ID from Agent Identities, enabling AI agents to act as digital workers with user identity capabilities in Microsoft 365 and Azure environments. | None |
 | [eval-driven-dev](../skills/eval-driven-dev/SKILL.md) | Set up eval-based QA for Python LLM applications: instrument the app, build golden datasets, write and run eval tests, and iterate on failures. ALWAYS USE THIS SKILL when the user asks to set up QA, add tests, add evals, evaluate, benchmark, fix wrong behaviors, improve quality, or do quality assurance for any Python project that calls an LLM model. | `references/dataset-generation.md`<br />`references/eval-tests.md`<br />`references/instrumentation.md`<br />`references/investigation.md`<br />`references/pixie-api.md`<br />`references/run-harness-patterns.md`<br />`references/understanding-app.md` |
 | [excalidraw-diagram-generator](../skills/excalidraw-diagram-generator/SKILL.md) | Generate Excalidraw diagrams from natural language descriptions. Use when asked to "create a diagram", "make a flowchart", "visualize a process", "draw a system architecture", "create a mind map", or "generate an Excalidraw file". Supports flowcharts, relationship diagrams, mind maps, and system architecture diagrams. Outputs .excalidraw JSON files that can be opened directly in Excalidraw. | `references/element-types.md`<br />`references/excalidraw-schema.md`<br />`scripts/.gitignore`<br />`scripts/README.md`<br />`scripts/add-arrow.py`<br />`scripts/add-icon-to-diagram.py`<br />`scripts/split-excalidraw-library.py`<br />`templates` |
+| [eyeball](../skills/eyeball/SKILL.md) | Document analysis with inline source screenshots. When you ask Copilot to analyze a document, Eyeball generates a Word doc where every factual claim includes a highlighted screenshot from the source material so you can verify it with your own eyes. | `tools` |
 | [fabric-lakehouse](../skills/fabric-lakehouse/SKILL.md) | Use this skill to get context about Fabric Lakehouse and its features for software systems and AI-powered functions. It offers descriptions of Lakehouse data components, organization with schemas and shortcuts, access control, and code examples. This skill supports users in designing, building, and optimizing Lakehouse solutions using best practices. | `references/getdata.md`<br />`references/pyspark.md` |
 | [fedora-linux-triage](../skills/fedora-linux-triage/SKILL.md) | Triage and resolve Fedora issues with dnf, systemd, and SELinux-aware guidance. | None |
 | [finalize-agent-prompt](../skills/finalize-agent-prompt/SKILL.md) | Finalize prompt file using the role of an AI agent to polish the prompt for the end user. | None |

plugins/eyeball/.github/plugin/plugin.json

@@ -0,0 +1,22 @@
+{
+  "name": "eyeball",
+  "description": "Document analysis with inline source screenshots. When you ask Copilot to analyze a document, Eyeball generates a Word doc where every factual claim includes a highlighted screenshot from the source material so you can verify it with your own eyes.",
+  "version": "1.0.0",
+  "author": {
+    "name": "Awesome Copilot Community"
+  },
+  "repository": "https://github.com/github/awesome-copilot",
+  "license": "MIT",
+  "keywords": [
+    "document-analysis",
+    "citation-verification",
+    "screenshot",
+    "contracts",
+    "legal",
+    "trust",
+    "visual-verification"
+  ],
+  "skills": [
+    "./skills/eyeball/"
+  ]
+}

plugins/eyeball/README.md

@@ -0,0 +1,88 @@
+# Eyeball
+
+When AI analyzes a document and tells you "Section 10 requires mutual indemnification," how do you know Section 10 actually says that? Eyeball lets you see for yourself.
+
+Eyeball is a Copilot CLI plugin that generates document analyses as Word files with inline screenshots from the source material. Every factual claim in the analysis includes a highlighted excerpt from the original document, so you can verify each assertion without switching between files or hunting for the right page.
+
+## What it does
+
+You give Copilot a document (Word file, PDF, or web URL) and ask it to analyze something specific. Eyeball reads the source, writes the analysis, and for each claim, captures a screenshot of the relevant section from the original document with the cited text highlighted in yellow. The output is a Word document on your Desktop with analysis text and source screenshots interleaved.
+
+If the analysis says "Section 9.3 allows termination for cause with a 30-day cure period," the screenshot below it shows Section 9.3 from the actual document with that language highlighted. If the screenshot shows something different, the analysis is wrong and you can see it immediately.
+
+## Prerequisites
+
+- [Copilot CLI](https://docs.github.com/copilot/concepts/agents/about-copilot-cli) installed and authenticated
+- Python 3.8 or later
+- One of the following for Word document support (PDFs and web URLs work without these):
+  - Microsoft Word (macOS or Windows)
+  - LibreOffice (any platform)
+
+## Setup
+
+After installing the plugin, install the Python dependencies:
+
+```bash
+pip3 install pymupdf pillow python-docx playwright
+python3 -m playwright install chromium
+```
+
+Verify setup:
+
+```bash
+python3 <plugin-path>/skills/eyeball/tools/eyeball.py setup-check
+```
+
+## Usage
+
+In a Copilot CLI conversation:
+
+```
+use eyeball on ~/Desktop/vendor-agreement.docx -- analyze the indemnification
+and liability provisions and flag anything unusual
+```
+
+```
+run eyeball on https://example.com/terms-of-service -- identify the
+developer-friendly aspects of these terms
+```
+
+```
+use eyeball to analyze this NDA for non-compete provisions
+```
+
+Eyeball reads the source document, writes the analysis with exact section references, and generates a Word document on your Desktop with source screenshots inline.
+
+## Supported source types
+
+| Source type | Requirements |
+|---|---|
+| PDF files | Python + PyMuPDF (included in setup) |
+| Web pages | Python + Playwright + Chromium (included in setup) |
+| Word documents (.docx) | Microsoft Word (macOS/Windows) or LibreOffice |
+
+## How it works
+
+1. Eyeball reads the full text of the source document
+2. It writes analysis with exact section numbers, page references, and verbatim quotes
+3. For each claim, it searches the rendered source for the cited text
+4. It captures a screenshot of the surrounding region with the cited text highlighted in yellow
+5. It assembles a Word document with analysis paragraphs and screenshots interleaved
+6. The output lands on your Desktop
+
+The screenshots are dynamically sized: if a section of analysis references text that spans a large region, the screenshot expands to cover it. If the referenced text appears on multiple pages, the screenshots are stitched together.
+
+## Why screenshots instead of quoted text?
+
+Quoted text is easy to fabricate. A model can generate a plausible-sounding quote that doesn't actually appear in the source, and without checking, you'd never know. Screenshots from the rendered source are harder to fake -- they show the actual formatting, layout, and surrounding context of the original document. You can see at a glance whether the highlighted text matches the claim, and the surrounding text provides context that a cherry-picked quote might omit.
+
+## Limitations
+
+- Word document conversion requires Microsoft Word or LibreOffice. Without one of these, you can still use Eyeball with PDFs and web URLs.
+- Text search is string-matching. If the source document uses unusual encoding, ligatures, or non-standard characters, some searches may not match.
+- Web page rendering depends on Playwright and may not perfectly capture all dynamic content (e.g., content loaded by JavaScript after page load, content behind login walls).
+- Screenshot quality depends on the source formatting. Dense multi-column layouts or very small text may produce less readable screenshots.
+
+## License
+
+MIT

skills/eyeball/SKILL.md

@@ -0,0 +1,165 @@
+---
+name: eyeball
+description: 'Document analysis with inline source screenshots. When you ask Copilot to analyze a document, Eyeball generates a Word doc where every factual claim includes a highlighted screenshot from the source material so you can verify it with your own eyes.'
+---
+
+# Eyeball
+
+Analyze documents with visual proof. When activated, Eyeball produces a Word document on the user's Desktop where every factual assertion includes an inline screenshot from the source material with the cited text highlighted in yellow.
+
+## Activation
+
+When the user invokes this skill (e.g., "use eyeball", "run eyeball on this", "eyeball this document"), respond with:
+
+> **Eyeball is active.** I'll analyze the document and produce a Word doc with inline source screenshots so you can verify every claim with your own eyes.
+
+Then follow the workflow below.
+
+## Supported Sources
+
+- **Local files:** Word documents (.docx, .doc), PDFs (.pdf), RTF files
+- **Web URLs:** Any publicly accessible web page
+
+## Tool Location
+
+The Eyeball Python utility is located at:
+```
+<plugin_dir>/skills/eyeball/tools/eyeball.py
+```
+
+To find the actual path, run:
+```bash
+find ~/.copilot/installed-plugins -name "eyeball.py" -path "*/eyeball/*" 2>/dev/null
+```
+
+If not found there, check the project directory or the user's home directory for the eyeball repo.
+
+## First-Run Setup
+
+Before first use, check that dependencies are installed:
+
+```bash
+python3 <path-to>/eyeball.py setup-check
+```
+
+If anything is missing, install the required dependencies:
+```bash
+pip3 install pymupdf pillow python-docx playwright
+python3 -m playwright install chromium
+```
+
+## Workflow
+
+Follow these steps exactly. The order matters.
+
+### Step 1: Read the source text
+
+Before writing any analysis, extract and read the full text of the source document:
+
+```bash
+python3 <path-to>/eyeball.py extract-text --source "<path-or-url>"
+```
+
+Read the output carefully. Identify actual section numbers, headings, page numbers, and key language.
+
+**CRITICAL:** Do not skip this step. Do not write analysis based on assumptions about how the document is structured. Read the actual text.
+
+### Step 2: Write analysis with exact citations
+
+For each point in your analysis, you must:
+
+1. **Reference the correct section number as it appears in the document** (e.g., "Section 9" not "Section 8" because you assumed the numbering).
+2. **Reference the correct page number** where the section appears in the extracted text.
+3. **Select anchors that are verbatim phrases from the source** that directly support your claim.
+
+### Step 3: Select anchors correctly
+
+This is the most important step. Anchors determine what gets highlighted in the screenshots.
+
+**DO:**
+- Use verbatim phrases from the source text that directly support your assertion
+- Use multiple anchors to span the full range of text the reader should see
+- Use specific, uncommon phrases that appear only where you intend
+
+**DO NOT:**
+- Use generic topic labels (e.g., "Confidentiality") that appear throughout the document
+- Use section titles alone when they appear as cross-references elsewhere
+- Use single common words that match in many places
+
+**Examples:**
+
+WRONG -- uses a generic topic label that matches everywhere:
+```json
+{"anchors": ["User-Generated Content"], "target_page": 8}
+```
+
+RIGHT -- uses the specific language that supports the claim:
+```json
+{"anchors": ["retain ownership", "Ownership of Content, Right to Post"], "target_page": 8}
+```
+
+WRONG -- section title appears as a cross-reference on earlier pages:
+```json
+{"anchors": ["LIMITATION OF LIABILITY"]}
+```
+
+RIGHT -- includes the section number for precision, targets the correct page:
+```json
+{"anchors": ["12. LIMITATION OF LIABILITY", "INDIRECT", "CONSEQUENTIAL"], "target_page": 13}
+```
+
+### Step 4: Build the analysis document
+
+Construct a JSON array of sections and call the build command:
+
+```bash
+python3 <path-to>/eyeball.py build \
+  --source "<path-or-url>" \
+  --output ~/Desktop/<title>.docx \
+  --title "Analysis Title" \
+  --subtitle "Source description" \
+  --sections '[
+    {
+      "heading": "1. Section Title",
+      "analysis": "Your analysis text here. Reference Section X on page Y...",
+      "anchors": ["verbatim phrase 1", "verbatim phrase 2"],
+      "target_page": 5,
+      "context_padding": 40
+    },
+    {
+      "heading": "2. Another Section",
+      "analysis": "More analysis...",
+      "anchors": ["exact quote from source"],
+      "target_pages": [10, 11],
+      "context_padding": 50
+    }
+  ]'
+```
+
+Section object fields:
+- `heading` (required): Section heading in the output document
+- `analysis` (required): Your analysis text
+- `anchors` (required): List of verbatim phrases from the source to search for and highlight
+- `target_page` (optional): Single page number (1-indexed) to search on
+- `target_pages` (optional): List of page numbers to search across (screenshots stitched vertically)
+- `context_padding` (optional): Padding in PDF points above/below the anchor region (default: 40). Increase for more context.
+
+### Step 5: Deliver the output
+
+Save the output to the user's Desktop. Tell the user the filename and that they can open it to verify each claim against the highlighted source screenshots.
+
+## Self-Check Before Delivery
+
+Before saving the final document, mentally verify:
+
+1. Does each section's analysis text reference the correct section number from the source?
+2. Are the anchors verbatim phrases that appear on the target page?
+3. Does each anchor directly support the claim in the analysis, not just relate to the same topic?
+4. If the screenshot doesn't match the analysis, is the analysis wrong or is the anchor wrong? Fix whichever is incorrect.
+
+## Notes
+
+- The output document includes highlighted screenshots that are dynamically sized. If you provide multiple anchors, the screenshot expands to cover all of them.
+- When a search term is not found, the output document will note this. If this happens, the anchor was likely not verbatim enough. Adjust and rebuild.
+- For web pages, Playwright renders the page to PDF first. The resulting page numbers may differ from what you see in a browser. Use the extracted text output (step 1) to determine correct page numbers.
+- If the user has already provided the source text or you have already read it in the current conversation, you can skip step 1. But always verify section numbers and page references against the actual text before writing analysis.