Improve documention

README.md

@@ -1,31 +1,106 @@
-# Outcome Tracker
+<div align="center">
 
-Source code for the frontend of the [Build Canada Outcome Tracker](https://www.buildcanada.com/tracker). The API repo is [OutcomeTrackerAPI](https://github.com/BuildCanada/OutcomeTrackerAPI).
+# 🏗️ Build Canada 🇨🇦
 
-## Getting Started
+### Outcome Tracker
+*Transparency in Government Promises*
 
-- Fork the repo, clone it, and install dependencies:
-  ```bash
-  git clone https://github.com/BuildCanada/OutcomeTracker.git
-  cd OutcomeTracker
-  pnpm install
-  ```
+<br/>
 
-- Copy .env.example to .env
-  - If you're using the production API, set NEXT_PUBLIC_API_URL=https://www.buildcanada.com/tracker/api/v1
-  - If you're running the API locally, set NEXT_PUBLIC_API_URL=http://localhost:3000/
+[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+[![Next.js](https://img.shields.io/badge/Next.js-15.2.4-black)](https://nextjs.org)
+[![TypeScript](https://img.shields.io/badge/TypeScript-5.x-blue)](https://www.typescriptlang.org)
 
-- Run the Frontend
-  ```bash
-  pnpm turbo
-  ```
+</div>
 
-- 🎉 **Time to explore!** Head over to [http://localhost:4444/tracker](http://localhost:4444/tracker) to see your local instance in action!
+---
 
-## Metrics
+A non-partisan platform for tracking the progress of key commitments made during the 45th Parliament of Canada. This application provides transparency and accountability by monitoring government promises across all departments.
 
-Metrics are scraped using github actions which automatically updates the repo with up to date data. 
+## 🎯 Overview
 
-## Contributing
+The Outcome Tracker monitors promises from:
+- 2021 Mandate Letters
+- 2025 Liberal Party Platform
+- Throne speeches
+- Major policy announcements
 
-We would love to have your help! Please fill in our vounteer intake form 
+Each promise is tracked with:
+- Progress scores and summaries
+- Evidence of actions taken
+- Timeline of key events
+- Impact assessments
+- Department assignments
+
+## 🚀 Quick Start
+
+```bash
+# Clone the repository
+git clone https://github.com/BuildCanada/OutcomeTracker.git
+cd OutcomeTracker
+
+# Install dependencies
+pnpm install
+
+# Set up environment variables
+# Create a .env file in the root directory with:
+echo "NEXT_PUBLIC_API_URL=https://www.buildcanada.com/tracker/api/v1" > .env
+
+# Run development server
+pnpm turbo
+```
+
+Visit [http://localhost:4444/tracker](http://localhost:4444/tracker) to see your local instance.
+
+## 📚 Documentation
+
+Detailed documentation is available in the [`/docs`](./docs) directory:
+
+- [**Architecture Overview**](./docs/ARCHITECTURE.md) - System design and technical stack
+- [**Development Guide**](./docs/DEVELOPMENT.md) - Setup, coding standards, and workflows
+- [**Data & Metrics**](./docs/DATA_METRICS.md) - Data sources and metric calculations
+- [**Components Guide**](./docs/COMPONENTS.md) - UI component documentation
+- [**API Integration**](./docs/API_INTEGRATION.md) - Backend API usage
+- [**Deployment**](./docs/DEPLOYMENT.md) - Production deployment guide
+
+## 🏗️ Tech Stack
+
+- **Frontend**: Next.js 15.2.4 with TypeScript
+- **Styling**: Tailwind CSS with shadcn/ui components
+- **Charts**: Chart.js with react-chartjs-2
+- **State Management**: SWR for data fetching
+- **Testing**: Vitest with Storybook
+- **Analytics**: Simple Analytics
+
+## 📊 Key Features
+
+- **Department Dashboard**: Track promises by government department
+- **Progress Visualization**: Real-time progress scores and timelines
+- **Economic Metrics**: Interactive charts for GDP, housing, population, and more
+- **Evidence Tracking**: Links to bills, news, and official documents
+- **Responsive Design**: Mobile-first approach with modern UI
+
+## 🤝 Contributing
+
+We welcome contributions! Please see our [Contributing Guide](./docs/CONTRIBUTING.md) for details.
+
+### Getting Involved
+1. Fill out our [volunteer intake form](https://5nneq7.share-na3.hsforms.com/2l9iIH2gFSomphjDe-ci5OQ)
+2. Join our community discussions
+3. Check open issues for ways to help
+
+## 📄 License
+
+This project is open source and available under the [MIT License](LICENSE).
+
+## 🔗 Related Resources
+
+- **Production Site**: [buildcanada.com/tracker](https://www.buildcanada.com/tracker)
+- **API Repository**: [OutcomeTrackerAPI](https://github.com/BuildCanada/OutcomeTrackerAPI)
+- **Main Website**: [buildcanada.com](https://www.buildcanada.com)
+
+---
+
+<div align="center">
+Built with ❤️ by Build Canada volunteers 🏗️🇨🇦
+</div>

docs/AI_PROMPTS.md

@@ -0,0 +1,240 @@
+# AI Prompts & Data Processing
+
+## Overview
+
+The Outcome Tracker uses AI-powered analysis to process government documents, extract evidence, and score promise progress. This document explains the prompt system and how it supports transparent tracking of government commitments.
+
+## Prompt Categories
+
+### 1. Evidence Extraction
+
+These prompts identify and extract relevant evidence from various government sources:
+
+#### Bill Evidence (`prompt_bill_evidence.md`)
+- Extracts evidence from parliamentary bills
+- Identifies which promises are addressed
+- Assesses impact and implementation timeline
+
+#### News Evidence (`prompt_news_evidence.md`)
+- Processes government news releases
+- Extracts announcements and policy changes
+- Links news to specific promises
+
+#### Order in Council Evidence (`prompt_oic_evidence.md`)
+- Analyzes Orders in Council (OIC)
+- Identifies regulatory changes
+- Maps OICs to promise implementation
+
+#### Gazette Evidence (`prompt_gazette2_evidence.md`)
+- Processes Canada Gazette publications
+- Extracts regulatory implementations
+- Identifies formal government actions
+
+### 2. Promise Analysis
+
+#### Progress Scoring (`prompt_progress_scoring.md`)
+- Evaluates promise completion percentage
+- Considers multiple evidence sources
+- Provides consistent scoring methodology
+
+**Scoring Scale:**
+- 0-20%: Not started or minimal action
+- 21-40%: Initial steps taken
+- 41-60%: Substantial progress
+- 61-80%: Mostly complete
+- 81-100%: Fully implemented
+
+#### Evidence-Promise Linking (`prompt_link_evidence_to_promise.md`)
+- Matches evidence to specific promises
+- Assesses relevance and impact
+- Maintains audit trail
+
+#### Promise Validation (`prompt_evidence_promise_validation.md`)
+- Validates promise interpretations
+- Ensures consistent categorization
+- Checks for duplicate or overlapping promises
+
+### 3. Content Generation
+
+#### What It Means for Canadians (`prompt_generate_whatitmeans.md`)
+- Translates policy language to plain English
+- Explains practical impacts
+- Provides context for citizens
+
+### 4. Rating & Evaluation
+
+#### Detailed Rating Instructions (`detailed_rating_instructions.md`)
+- Comprehensive scoring methodology
+- Evidence weighting criteria
+- Quality assurance guidelines
+
+## Build Canada Tenets
+
+The project follows core principles outlined in `build_canada_tenets.txt`:
+- Non-partisan analysis
+- Evidence-based tracking
+- Transparent methodology
+- Citizen accessibility
+- Continuous improvement
+
+## Economic Context
+
+The `economic_contexts/` directory contains:
+- **2021 Mandate Letters**: Original government commitments
+- **2025 Platform**: Updated policy positions
+- **Historical Context**: Background for understanding promises
+
+## Prompt Engineering Guidelines
+
+### 1. Objectivity
+```markdown
+# Good Practice
+"Identify factual actions taken by the government related to this promise"
+
+# Avoid
+"Determine if the government has successfully fulfilled this promise"
+```
+
+### 2. Evidence Requirements
+All prompts require:
+- Source citations
+- Date of evidence
+- Direct quotes when relevant
+- Clear impact assessment
+
+### 3. Consistency
+Prompts use standardized:
+- Scoring scales
+- Impact categories
+- Evidence types
+- Output formats
+
+## Data Processing Pipeline
+
+```mermaid
+graph TD
+    A[Government Documents] --> B[Evidence Extraction]
+    B --> C[Promise Matching]
+    C --> D[Progress Scoring]
+    D --> E[Validation]
+    E --> F[Public Display]
+
+    G[News/Announcements] --> B
+    H[Bills/Legislation] --> B
+    I[Orders in Council] --> B
+```
+
+## Adding New Prompt Types
+
+### 1. Create Prompt File
+```markdown
+# prompt_new_source_evidence.md
+
+## Objective
+Extract evidence from [NEW SOURCE TYPE] that demonstrates government action on promises.
+
+## Input Format
+- Source document text
+- Document metadata (date, type, url)
+
+## Analysis Requirements
+1. Identify specific government actions
+2. Match to promise IDs
+3. Assess implementation impact
+4. Extract relevant quotes
+
+## Output Format
+```
+
+### 2. Define Evidence Structure
+```json
+{
+  "source_type": "new_source",
+  "evidence_items": [
+    {
+      "promise_id": 12345,
+      "action_taken": "Description of action",
+      "impact": "positive|neutral|negative",
+      "impact_magnitude": "minor|moderate|significant",
+      "quote": "Relevant quote from source",
+      "date": "2024-01-15"
+    }
+  ]
+}
+```
+
+### 3. Integrate with Pipeline
+- Add to evidence processing workflow
+- Update validation rules
+- Test with sample documents
+
+## Quality Assurance
+
+### Validation Checks
+1. **Source Verification**: All evidence linked to official sources
+2. **Date Consistency**: Evidence dates match source documents
+3. **Promise Alignment**: Evidence correctly matched to promises
+4. **Scoring Logic**: Progress scores align with evidence
+
+### Human Review
+Critical decisions require human validation:
+- Major progress updates (>25% change)
+- Controversial interpretations
+- New evidence types
+- Promise completion status
+
+## Ethical Considerations
+
+### Bias Prevention
+- Multiple evidence sources required
+- Both positive and negative evidence tracked
+- Regular methodology reviews
+- Community feedback integration
+
+### Transparency
+- All prompts publicly available
+- Scoring methodology documented
+- Evidence trails maintained
+- Changes logged and justified
+
+## Prompt Maintenance
+
+### Regular Updates
+- Review prompts quarterly
+- Incorporate feedback
+- Adjust for new document types
+- Refine scoring accuracy
+
+### Version Control
+- Track prompt changes in git
+- Document modification reasons
+- Test impacts before deployment
+- Maintain backwards compatibility
+
+## Performance Metrics
+
+Track prompt effectiveness:
+- Evidence extraction accuracy
+- Promise matching precision
+- Scoring consistency
+- Processing speed
+- User trust ratings
+
+## Future Enhancements
+
+### Planned Improvements
+1. Multi-language support
+2. Provincial promise tracking
+3. Real-time evidence updates
+4. Advanced NLP models
+5. Citizen evidence submission
+
+### Research Areas
+- Automated fact-checking
+- Sentiment analysis
+- Promise interdependency mapping
+- Predictive completion modeling
+
+---
+
+For technical implementation details, see the [API Integration Guide](./API_INTEGRATION.md).