+
+
+ 
+
+
+ 
+
++ +
+ AI-powered semantic code search for your repositories +
+ + + ++ Quick Start β’ + Deployment β’ + MCP Integration β’ + Contributing +
-**MCP server for AI-powered codebase intelligence.** Semantic search, dependency analysis, and impact prediction for your repositories. - -## The Problem - -AI coding assistants are powerful, but they're flying blind in large codebases: -- Can't semantically search across thousands of files -- Don't understand dependency relationships -- Can't predict what breaks when you change a file -- Have no context on team coding patterns - -## The Solution - -CodeIntel is an MCP (Model Context Protocol) server that gives AI agents deep codebase understanding: - -```typescript -// Ask Claude (via MCP): -"Find authentication middleware in this repo" - -// CodeIntel semantically searches 10,000+ functions -// Returns exact implementations, not keyword matches -``` - -**Built for production. Not a demo.** - -## Key Features - -### π Semantic Code Search -Search by meaning, not keywords. Find `"error handling logic"` even if functions are named `processFailure()`. - -### π Dependency Analysis -Visualize your entire codebase architecture. See which files are critical, which are isolated, and how everything connects. +--- -### β‘ Impact Prediction -Before changing a file, know exactly what breaks: -``` -src/auth/middleware.py -ββ 15 files affected (HIGH RISK) - ββ src/api/routes.py - ββ src/services/user.py - ββ ... + 12 more -``` + + -### π¨ Code Style Analysis -Understand team patterns: naming conventions (camelCase vs snake_case), async adoption %, type hint usage. +## What is OpenCodeIntel? -### π Performance That Scales +OpenCodeIntel gives AI coding assistants deep understanding of your codebase. It's an MCP server that provides semantic code search, dependency analysis, and impact prediction. -**Batch Processing:** 100x faster indexing -- Before: 40+ min for 1,000 functions (individual API calls) -- After: 22.9 sec (batch embedding requests) +**Search by meaning, not keywords.** Find "error handling logic" even when functions are named `processFailure()`. -**Incremental Indexing:** 700x faster re-indexing -- Full re-index: 51.4s -- Incremental (git diff): 0.07s -- Perfect for active development +## Features -**Supabase Caching:** 5x search speedup -- Cold search: 800ms -- Cached: 150ms +- **Semantic Search** - Vector-based code search that understands intent +- **Dependency Graph** - Visualize how your codebase connects +- **Impact Analysis** - Know what breaks before you change a file +- **Code Style Analysis** - Understand team patterns and conventions +- **MCP Integration** - Works directly with Claude Desktop ## Quick Start -### π³ Docker (Recommended) - -**Fastest way to get started:** +### Using Docker (Recommended) ```bash -# 1. Clone repo git clone https://github.com/OpenCodeIntel/opencodeintel.git cd opencodeintel -# 2. Configure environment cp .env.example .env -# Edit .env with your API keys +# Add your API keys to .env -# 3. Start everything docker compose up -d - -# Frontend: http://localhost:3000 -# Backend: http://localhost:8000 -# Docs: http://localhost:8000/docs ``` -**Full guide:** [`DOCKER_QUICKSTART.md`](./DOCKER_QUICKSTART.md) -**Troubleshooting:** [`DOCKER_TROUBLESHOOTING.md`](./DOCKER_TROUBLESHOOTING.md) +- Frontend: http://localhost:3000 +- Backend: http://localhost:8000 +- API Docs: http://localhost:8000/docs ---- - -### π¦ Manual Setup +### Manual Setup -### Prerequisites -- Python 3.11+ -- Node.js 20+ -- OpenAI API key -- Pinecone account -- Supabase project - -### 1. Clone & Setup Backend +**Requirements:** Python 3.11+, Node.js 20+ ```bash +# Backend cd backend python -m venv venv -source venv/bin/activate # Windows: venv\Scripts\activate +source venv/bin/activate pip install -r requirements.txt - -# Configure .env cp .env.example .env -# Add your API keys to .env -``` - -### 2. Run Backend - -```bash python main.py -# Server runs on http://localhost:8000 -``` -### 3. Setup Frontend - -```bash +# Frontend (new terminal) cd frontend npm install npm run dev -# UI at http://localhost:5173 -``` - -### 4. Add a Repository - -```bash -# Via API -curl -X POST http://localhost:8000/api/repos \ - -H "Authorization: Bearer dev-secret-key" \ - -H "Content-Type: application/json" \ - -d '{"name": "zustand", "git_url": "https://github.com/pmndrs/zustand"}' - -# Or use the web UI ``` ## MCP Integration -CodeIntel works as an MCP server with Claude Desktop. **[π Full MCP Setup Guide](./docs/MCP_SETUP.md)** +Connect OpenCodeIntel to Claude Desktop for AI-powered code assistance. -**Quick Setup:** +Add to your Claude Desktop config: ```json -// Add to Claude Desktop config { "mcpServers": { - "codeintel": { + "opencodeintel": { "command": "python", "args": ["/path/to/opencodeintel/mcp-server/server.py"], "env": { @@ -160,142 +105,41 @@ CodeIntel works as an MCP server with Claude Desktop. **[π Full MCP Setup Gui } ``` -**Available MCP Tools:** -| Tool | Description | -|------|-------------| -| `search_code` | Semantic code search - finds code by meaning | -| `list_repositories` | View all indexed repos | -| `get_dependency_graph` | Visualize architecture and file connections | -| `analyze_code_style` | Team conventions and patterns | -| `analyze_impact` | Know what breaks before you change it | -| `get_repository_insights` | High-level codebase overview | - -Now ask Claude: *"What's the authentication logic in the user service?"* and it searches your actual codebase. +**Available tools:** `search_code`, `list_repositories`, `get_dependency_graph`, `analyze_code_style`, `analyze_impact`, `get_repository_insights` -**[β Complete setup guide with troubleshooting](./docs/MCP_SETUP.md)** +See [MCP Setup Guide](./docs/mcp-setup.md) for detailed instructions. ## Architecture ``` -βββββββββββββββ -β Frontend β React + TypeScript + Tailwind -β (Vite app) β Dependency graphs, search UI -ββββββββ¬βββββββ - β -ββββββββΌβββββββ -β FastAPI β Python backend -β Backend β /api/search, /api/repos/{id}/dependencies -ββββββββ¬βββββββ - β - βββββββΊ Pinecone (vector search) - βββββββΊ OpenAI (embeddings) - βββββββΊ Supabase (persistence) - βββββββΊ Redis (caching) -``` - -**Tech Stack:** -- **Backend:** FastAPI, tree-sitter (AST parsing), OpenAI embeddings -- **Vector DB:** Pinecone for semantic search -- **Database:** Supabase (PostgreSQL) for metadata + caching -- **Cache:** Redis for 5x search speedup -- **Frontend:** React, TypeScript, Tailwind CSS, shadcn/ui, ReactFlow - -## Performance Benchmarks - -Real numbers from indexing the Zustand repository (1,174 functions): - -| Metric | Value | -|--------|-------| -| Full indexing | 29.5s (39.7 functions/sec) | -| Incremental re-index | 0.07s (700x faster) | -| Batch embedding | 22.9s for 1,174 functions | -| Search (cold) | 800ms | -| Search (cached) | 150ms | - -## Use Cases - -**For AI Agents (via MCP):** -- Semantic code search during pair programming -- Understanding unfamiliar codebases -- Finding implementation patterns -- Impact analysis before refactoring - -**For Development Teams:** -- Onboarding new engineers (visualize architecture) -- Code review prep (see change blast radius) -- Tech debt identification (find highly coupled files) -- Pattern enforcement (analyze style consistency) - -## What Makes This Different - -**Most code search tools:** Keyword matching (grep, GitHub search) -**CodeIntel:** Understands *meaning* - finds `error handling` even if the function is called `processFailure()` - -**Most dependency tools:** Static analysis only -**CodeIntel:** Combines AST parsing + semantic understanding + impact prediction - -**Most demos:** In-memory, doesn't scale -**CodeIntel:** Production-grade with Supabase persistence, Redis caching, incremental indexing - -## Deployment - -### π³ Local Development (Docker) -```bash -# Start all services -make dev - -# Or using docker compose -docker compose -f docker-compose.dev.yml up -d - -# Services available at: -# - Backend: http://localhost:8000 -# - Frontend: http://localhost:3000 -# - API Docs: http://localhost:8000/docs -``` - -### βοΈ Production Deployment - -**Backend + Redis β Railway** -```bash -# Automated deployment -./scripts/deploy-railway.sh - -# Or manually: -railway login -railway init -railway up +Frontend (React + TypeScript) + β +Backend (FastAPI + Python) + β +ββββββ΄βββββ¬βββββββββββββ +Pinecone Supabase Redis +(vectors) (database) (cache) ``` -**Frontend β Vercel** -```bash -# Automated deployment -./scripts/deploy-vercel.sh +**Stack:** FastAPI, React, TypeScript, Pinecone, Supabase, Redis, tree-sitter -# Or manually: -cd frontend -vercel --prod -``` +## Documentation -**π Full deployment guide:** See [DEPLOYMENT.md](DEPLOYMENT.md) for complete instructions, environment variables, and troubleshooting. +| Guide | Description | +|-------|-------------| +| [Docker Quickstart](./docs/docker-quickstart.md) | Get running in 5 minutes | +| [Deployment](./docs/deployment.md) | Production deployment guide | +| [MCP Setup](./docs/mcp-setup.md) | Claude Desktop integration | +| [Docker Troubleshooting](./docs/docker-troubleshooting.md) | Common issues and fixes | ## Contributing -Built in a focused 2-week sprint to demonstrate production-grade AI development tooling. +We welcome contributions! See [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines. -Contributions welcome! Areas for improvement: -- Support for more languages (currently: Python, JS/TS) -- Advanced graph algorithms (find circular dependencies, suggest refactorings) -- GitHub integration (PR impact analysis) -- Team analytics (who writes what patterns) +**Quick links:** +- [Open Issues](https://github.com/OpenCodeIntel/opencodeintel/issues) +- [Good First Issues](https://github.com/OpenCodeIntel/opencodeintel/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) ## License -MIT License - use it, fork it, build on it. - -## Built With - -Commitment to shipping production-grade AI tools. Not a side project. Not a demo. Real infrastructure that scales. - ---- - -**Questions?** Open an issue or reach out. +MIT License - see [LICENSE](./LICENSE) for details. diff --git a/SETUP_COMPLETE.md b/SETUP_COMPLETE.md deleted file mode 100644 index 80ffe6a..0000000 --- a/SETUP_COMPLETE.md +++ /dev/null @@ -1,279 +0,0 @@ -# π CodeIntel Docker & Deployment Setup Complete! - -## β What's Ready - -### 1. Docker Configuration -- β `docker-compose.yml` - Production setup -- β `docker-compose.dev.yml` - Development with hot reload -- β Backend `Dockerfile` - Multi-stage, optimized -- β Frontend `Dockerfile` - Nginx production build -- β Root `.env` file - All API keys configured -- β `.gitignore` updated - API keys won't leak - -### 2. Deployment Files -- β `DEPLOYMENT.md` - Complete deployment guide (337 lines) -- β `DOCKER_QUICKSTART.md` - 5-minute quick start (197 lines) -- β `DOCKER_TROUBLESHOOTING.md` - Common issues & fixes (284 lines) -- β `railway.json` - Railway config -- β Deployment scripts (executable): - - `scripts/deploy-railway.sh` - Backend to Railway - - `scripts/deploy-vercel.sh` - Frontend to Vercel - - `scripts/verify-setup.sh` - Pre-deployment checks - -### 3. Developer Experience -- β `Makefile` - 20+ commands for dev workflow -- β README updated - Docker section added -- β Health checks - All services monitored -- β Graceful restarts - No data loss -- β Redis persistence - AOF enabled - -## π Quick Start Commands - -### Local Development -```bash -# Verify setup -./scripts/verify-setup.sh - -# Start everything -make dev -# OR -docker compose up -d - -# View logs -make logs - -# Stop -make stop -``` - -**Access at:** -- Frontend: http://localhost:3000 -- Backend: http://localhost:8000 -- API Docs: http://localhost:8000/docs -- Redis: localhost:6379 - -### Production Deployment - -**Option 1: Automated Scripts** -```bash -# Deploy backend to Railway -./scripts/deploy-railway.sh - -# Deploy frontend to Vercel -./scripts/deploy-vercel.sh -``` - -**Option 2: Makefile** -```bash -make deploy-backend -make deploy-frontend -# OR -make deploy-all -``` - -**Option 3: Manual** -See `DEPLOYMENT.md` for step-by-step guide - -## π Pre-Deployment Checklist - -Before deploying to production, make sure: - -- [ ] Docker Desktop is running -- [ ] All API keys are set in `.env` -- [ ] Tests passing: `make test` -- [ ] Local Docker works: `make dev` -- [ ] Health check passes: `make health` -- [ ] Railway CLI installed: `npm i -g @railway/cli` -- [ ] Vercel CLI installed: `npm i -g vercel` -- [ ] Changed `API_KEY` from default value -- [ ] Supabase RLS policies configured -- [ ] Read through `DEPLOYMENT.md` - -## π― Next Steps - -### 1. Test Locally -```bash -# Start services -make dev - -# In another terminal, run tests -make test - -# Check everything is healthy -make health -``` - -### 2. Deploy Backend (Railway) -```bash -# Automated -./scripts/deploy-railway.sh - -# Follow prompts to: -# - Login to Railway -# - Create/link project -# - Add Redis service -# - Set environment variables -# - Deploy -``` - -### 3. Deploy Frontend (Vercel) -```bash -# Get your Railway backend URL first -railway domain - -# Then deploy frontend -./scripts/deploy-vercel.sh - -# Enter Railway URL when prompted -``` - -### 4. Configure Production -After deployment: -1. Update CORS in `backend/main.py` with Vercel URL -2. Test all endpoints work -3. Monitor logs: `railway logs -f` -4. Set up custom domains (optional) - -## π Documentation Reference - -| Document | Purpose | -|----------|---------| -| `README.md` | Project overview, features, quick start | -| `DOCKER_QUICKSTART.md` | Get running in 5 minutes | -| `DOCKER_TROUBLESHOOTING.md` | Fix common Docker issues | -| `DEPLOYMENT.md` | Complete deployment guide | -| `SECURITY.md` | Security practices & vulnerability reporting | -| `CONTRIBUTING.md` | How to contribute | - -## π§ Useful Commands - -### Docker -```bash -make dev # Start dev environment -make prod # Start production environment -make logs # View all logs -make stop # Stop services -make clean # Nuclear option - remove everything -make health # Check service health -make restart-backend # Quick backend restart -``` - -### Testing -```bash -make test # Run tests -make test-watch # Watch mode -make coverage # Coverage report -``` - -### Deployment -```bash -make deploy-backend # Deploy to Railway -make deploy-frontend # Deploy to Vercel -make deploy-all # Deploy everything -``` - -### Debugging -```bash -make shell-backend # Bash into backend container -make shell-redis # Redis CLI -make redis-stats # View Redis info -docker compose ps # Check container status -docker compose logs -f backend # Follow backend logs -``` - -## π Common Issues - -| Issue | Quick Fix | -|-------|-----------| -| Docker daemon not running | Open Docker Desktop | -| Port already in use | `lsof -i :8000` and kill process | -| Env vars not found | Make sure `.env` exists in project root | -| Build fails | `make clean && make build` | -| Services keep restarting | Check logs: `make logs` | - -**Full troubleshooting:** See `DOCKER_TROUBLESHOOTING.md` - -## π What Got Built - -### Architecture -``` -βββββββββββββββ βββββββββββββββ βββββββββββββββ -β Frontend βββββββΆβ Backend βββββββΆβ Redis β -β Vite+React β β FastAPI β β Cache β -β Port 3000 β β Port 8000 β β Port 6379 β -βββββββββββββββ βββββββββββββββ βββββββββββββββ - β - ββββββΆ Supabase (Postgres) - ββββββΆ Pinecone (Vectors) -``` - -### Files Created/Updated -- β `.env` - Root environment variables -- β `docker-compose.yml` - Production services (removed obsolete `version`) -- β `docker-compose.dev.yml` - Dev services (removed obsolete `version`) -- β `DOCKER_QUICKSTART.md` - Quick start guide -- β `DOCKER_TROUBLESHOOTING.md` - Troubleshooting guide -- β `scripts/verify-setup.sh` - Pre-deployment verification (made executable) -- β `README.md` - Added Docker quick start section - -### Already Existing (Verified Working) -- β `backend/Dockerfile` - Production-ready -- β `frontend/Dockerfile` - Multi-stage build with nginx -- β `railway.json` - Railway configuration -- β `DEPLOYMENT.md` - Comprehensive deployment guide -- β `Makefile` - Developer commands -- β `scripts/deploy-railway.sh` - Railway deployment -- β `scripts/deploy-vercel.sh` - Vercel deployment - -## π What You Learned - -This setup demonstrates: -1. **Production-grade Docker Compose** - Multi-service orchestration -2. **Multi-stage builds** - Optimized image sizes -3. **Health checks** - Service monitoring -4. **Environment management** - Secrets handling -5. **Deployment automation** - Scripts for Railway/Vercel -6. **Developer experience** - Makefile commands, hot reload -7. **Documentation** - Comprehensive guides for users - -## π° Expected Costs - -**Hobby/Free Tier:** -- Railway: $5/month credit (backend + Redis) -- Vercel: Free for personal projects -- **Total: $0-5/month** - -**Production:** -- Railway Pro: $20/month -- Vercel Pro: $20/month -- OpenAI API: ~$10-50/month -- Pinecone Starter: $70/month -- **Total: ~$120-160/month** - -## π You're Ready! - -Your CodeIntel project is now: -- β Docker Compose ready for local dev -- β Production-ready Dockerfiles -- β Deployment scripts for Railway + Vercel -- β Comprehensive documentation -- β Developer-friendly tooling - -**Start building:** -```bash -make dev -open http://localhost:3000 -``` - -**Deploy to production:** -```bash -./scripts/verify-setup.sh # Verify first -./scripts/deploy-railway.sh # Deploy backend -./scripts/deploy-vercel.sh # Deploy frontend -``` - ---- - -**Questions?** Check `DOCKER_TROUBLESHOOTING.md` or open an issue on GitHub. - -**Ready to ship!** π diff --git a/docs/HANDOFF-114.md b/docs/HANDOFF-114.md deleted file mode 100644 index a938fc6..0000000 --- a/docs/HANDOFF-114.md +++ /dev/null @@ -1,60 +0,0 @@ -# Handoff: Anonymous Indexing (#114) - -## TL;DR -Let users index their own GitHub repos without signup. 5 backend endpoints needed. - -## GitHub Issues (Full Specs) -- **#124** - Validate GitHub URL -- **#125** - Start anonymous indexing -- **#126** - Get indexing status -- **#127** - Extend session management -- **#128** - Update search for user repos - -**Read these first.** Each has request/response schemas, implementation notes, acceptance criteria. - -## Order of Work -``` -#127 + #124 (parallel) β #125 β #126 β #128 -``` - -## Key Files to Understand - -| File | What It Does | -|------|--------------| -| `backend/config/api.py` | API versioning (`/api/v1/*`) | -| `backend/routes/playground.py` | Existing playground endpoints | -| `backend/services/playground_limiter.py` | Session + rate limiting | -| `backend/services/repo_validator.py` | File counting, extensions | -| `backend/dependencies.py` | Indexer, cache, redis_client | - -## Constraints (Anonymous Users) -- 200 files max -- 1 repo per session -- 50 searches per session -- 24hr TTL - -## Workflow -See `CONTRIBUTING.md` for full guide. - -**Quick version:** -```bash -# Create branch -git checkout -b feat/124-validate-repo - -# Make changes, test -pytest tests/ -v - -# Commit -git add . -git commit -m "feat(playground): add validate-repo endpoint" - -# Push to YOUR fork -git push origin feat/124-validate-repo - -# Create PR on OpenCodeIntel/opencodeintel -# Reference issue: "Closes #124" -``` - -## Questions? -- Check GitHub issues first -- Ping Devanshu for blockers diff --git a/docs/TIER_SYSTEM_DESIGN.md b/docs/TIER_SYSTEM_DESIGN.md deleted file mode 100644 index d5b093f..0000000 --- a/docs/TIER_SYSTEM_DESIGN.md +++ /dev/null @@ -1,387 +0,0 @@ -# User Tier & Limits System - Design Document - -> **Issues**: #93, #94, #95, #96, #97 -> **Author**: Devanshu -> **Status**: Implemented -> **Last Updated**: 2025-12-13 - ---- - -## 1. Problem Statement - -CodeIntel needs a tiered system to: -1. **Protect costs** - Indexing is expensive ($0.02-$50/repo depending on size) -2. **Enable growth** - Freemium model with upgrade path -3. **Prevent abuse** - Rate limit anonymous playground users - -**Key Insight**: Searching is nearly free ($0.000001/query). Indexing is the real cost driver. - ---- - -## 2. Tier Definitions - -| Tier | Max Repos | Files/Repo | Functions/Repo | Playground/Day | -|------|-----------|------------|----------------|----------------| -| **Free** | 3 | 500 | 2,000 | 50 | -| **Pro** | 20 | 5,000 | 20,000 | Unlimited | -| **Enterprise** | Unlimited | 50,000 | 200,000 | Unlimited | - -**Rationale**: -- Free tier: Enough for personal projects, not enterprise codebases -- Playground limit: 50/day is generous (anti-abuse, not business gate) -- File/function limits: Prevent expensive indexing jobs - ---- - -## 3. Current API Endpoints - -### 3.1 Authentication (`/api/v1/auth`) -| Method | Endpoint | Auth | Description | -|--------|----------|------|-------------| -| POST | `/signup` | None | Create account | -| POST | `/login` | None | Get JWT | -| POST | `/refresh` | JWT | Refresh token | -| POST | `/logout` | JWT | Invalidate session | -| GET | `/me` | JWT | Get current user | - -### 3.2 Repositories (`/api/v1/repos`) -| Method | Endpoint | Auth | Description | **Limits Check** | -|--------|----------|------|-------------|------------------| -| GET | `/` | JWT | List user repos | - | -| POST | `/` | JWT | Add repo | **#95: Check repo count** | -| POST | `/{id}/index` | JWT | Index repo | **#94: Check file/function count** | - -### 3.3 Search (`/api/v1/search`) -| Method | Endpoint | Auth | Description | **Limits Check** | -|--------|----------|------|-------------|------------------| -| POST | `/search` | JWT | Search code | - | -| POST | `/explain` | JWT | Explain code | - | - -### 3.4 Playground (`/api/v1/playground`) - **Anonymous** -| Method | Endpoint | Auth | Description | **Limits Check** | -|--------|----------|------|-------------|------------------| -| GET | `/repos` | None | List demo repos | - | -| POST | `/search` | None | Search demo repos | **#93: Rate limit 50/day** | - -### 3.5 Analysis (`/api/v1/analysis`) -| Method | Endpoint | Auth | Description | -|--------|----------|------|-------------| -| GET | `/{id}/dependencies` | JWT | Dependency graph | -| POST | `/{id}/impact` | JWT | Impact analysis | -| GET | `/{id}/insights` | JWT | Repo insights | -| GET | `/{id}/style-analysis` | JWT | Code style | - -### 3.6 Users (`/api/v1/users`) - **NEW** -| Method | Endpoint | Auth | Description | -|--------|----------|------|-------------| -| GET | `/usage` | JWT | Get tier, limits, current usage | -| GET | `/limits/check-repo-add` | JWT | Pre-check before adding repo | - ---- - -## 4. Implementation Plan by Issue - -### Issue #96: User Tier System (Foundation) β DONE -**Files Created**: -- `backend/services/user_limits.py` - Core service -- `backend/routes/users.py` - API endpoints -- `supabase/migrations/001_user_profiles.sql` - DB schema - -**Service Methods**: -```python -class UserLimitsService: - def get_user_tier(user_id) -> UserTier - def get_user_limits(user_id) -> TierLimits - def get_user_repo_count(user_id) -> int - def check_repo_count(user_id) -> LimitCheckResult - def check_repo_size(user_id, file_count, func_count) -> LimitCheckResult - def get_usage_summary(user_id) -> dict - def invalidate_tier_cache(user_id) -> None # Call after tier upgrade -``` - -### Issue #95: Repo Count Limits -**Where**: `POST /api/v1/repos` - -**Changes to `routes/repos.py`**: -```python -@router.post("") -def add_repository(request, auth): - # NEW: Check repo count limit - result = user_limits.check_repo_count(auth.user_id) - if not result.allowed: - raise HTTPException( - status_code=403, - detail=result.to_dict() - ) - # ... existing code -``` - -**Frontend Integration**: -- Call `GET /users/limits/check-repo-add` before showing Add Repo button -- Show "2/3 repos used" in sidebar -- Show upgrade prompt when limit reached - -### Issue #94: Repo Size Limits -**Where**: `POST /api/v1/repos/{id}/index` - -**Changes to `routes/repos.py`**: -```python -@router.post("/{repo_id}/index") -def index_repository(repo_id, auth): - repo = get_repo_or_404(repo_id, auth.user_id) - - # Count files and estimate functions BEFORE indexing - file_count = count_code_files(repo["local_path"]) - estimated_functions = file_count * 25 # Conservative estimate - - # NEW: Check size limits - result = user_limits.check_repo_size( - auth.user_id, file_count, estimated_functions - ) - if not result.allowed: - raise HTTPException( - status_code=400, - detail=result.to_dict() - ) - # ... existing indexing code -``` - -### Issue #93: Playground Rate Limiting -**Where**: `POST /api/v1/playground/search` - -**New File**: `backend/services/playground_rate_limiter.py` -```python -class PlaygroundRateLimiter: - def __init__(self, redis_client): - self.redis = redis_client - self.daily_limit = 50 - - def check_and_increment(self, ip: str) -> tuple[bool, dict]: - """Returns (allowed, headers_dict)""" - key = f"playground:rate:{ip}" - - # Atomic increment - count = self.redis.incr(key) - if count == 1: - self.redis.expire(key, 86400) # 24h TTL - - ttl = self.redis.ttl(key) - reset_time = int(time.time()) + ttl - - headers = { - "X-RateLimit-Limit": str(self.daily_limit), - "X-RateLimit-Remaining": str(max(0, self.daily_limit - count)), - "X-RateLimit-Reset": str(reset_time) - } - - if count > self.daily_limit: - headers["Retry-After"] = str(ttl) - return False, headers - - return True, headers -``` - -**Changes to `routes/playground.py`**: -```python -from fastapi import Request, Response - -@router.post("/search") -def playground_search(request: Request, response: Response, body: SearchRequest): - # Get client IP - ip = request.client.host - forwarded = request.headers.get("X-Forwarded-For") - if forwarded: - ip = forwarded.split(",")[0].strip() - - # Check rate limit - allowed, headers = playground_rate_limiter.check_and_increment(ip) - - # Always add headers - for key, value in headers.items(): - response.headers[key] = value - - if not allowed: - raise HTTPException( - status_code=429, - detail={ - "error": "RATE_LIMIT_EXCEEDED", - "message": "Daily search limit reached. Sign up for unlimited searches!", - "limit": 50, - "reset": headers["X-RateLimit-Reset"] - } - ) - - # ... existing search code -``` - -### Issue #97: Progressive Signup CTAs -**Where**: Frontend only - -**Implementation**: -```typescript -// hooks/usePlaygroundUsage.ts -const usePlaygroundUsage = () => { - const [searchCount, setSearchCount] = useState(0); - - // Read from response headers after each search - const trackSearch = (response: Response) => { - const remaining = response.headers.get('X-RateLimit-Remaining'); - const limit = response.headers.get('X-RateLimit-Limit'); - if (remaining && limit) { - setSearchCount(parseInt(limit) - parseInt(remaining)); - } - }; - - return { searchCount, trackSearch }; -}; - -// Show CTAs at thresholds -// 10 searches: Subtle "Want to search YOUR codebase?" -// 25 searches: More prominent with feature list -// 40 searches: Final "You clearly love this" -``` - ---- - -## 5. Error Response Format - -All limit-related errors use `LimitCheckResult.to_dict()`: - -```json -{ - "detail": { - "allowed": false, - "current": 3, - "limit": 3, - "limit_display": "3", - "message": "Repository limit reached (3/3). Upgrade to add more repositories.", - "tier": "free", - "error_code": "REPO_LIMIT_REACHED" - } -} -``` - -**Error Codes**: -| Code | HTTP Status | Description | -|------|-------------|-------------| -| `REPO_LIMIT_REACHED` | 403 | Max repos for tier | -| `REPO_TOO_LARGE` | 400 | File/function count exceeds tier | -| `RATE_LIMIT_EXCEEDED` | 429 | Playground daily limit | -| `INVALID_USER` | 400 | Invalid or missing user_id | -| `SYSTEM_ERROR` | 500 | Database/system failure | - ---- - -## 6. Database Schema - -### user_profiles (NEW) -```sql -CREATE TABLE user_profiles ( - id UUID PRIMARY KEY, - user_id UUID REFERENCES auth.users(id), - tier TEXT DEFAULT 'free', -- 'free', 'pro', 'enterprise' - created_at TIMESTAMPTZ, - updated_at TIMESTAMPTZ -); -``` - -**Security Notes:** -- RLS enabled with SELECT/INSERT for authenticated users -- NO UPDATE policy for users (prevents self-upgrade) -- Tier updates only via service role key (payment webhooks) - -### repositories (existing, no changes needed) -Already has `user_id` column for ownership. - ---- - -## 7. Fail-Safe Behavior - -| Scenario | Behavior | Reason | -|----------|----------|--------| -| DB down during `check_repo_count` | **DENY** (fail-closed) | Prevent unlimited repos | -| DB down during `get_usage_summary` | Return defaults | Read-only, safe to fail-open | -| Redis cache miss | Query DB | Graceful degradation | -| Redis down | Continue without cache | Non-critical | -| Invalid user_id | Return FREE limits | Safe default | - ---- - -## 8. Redis Keys - -| Key Pattern | TTL | Description | -|-------------|-----|-------------| -| `playground:rate:{ip}` | 24h | Playground search count | -| `user:tier:{user_id}` | 5min | Cached user tier | - ---- - -## 9. Frontend Integration Points - -### Dashboard -- Show usage bar: "2/3 repositories" -- Show tier badge: "Free Tier" -- Upgrade CTA when near limits - -### Add Repository Flow -1. Call `GET /users/limits/check-repo-add` -2. If `allowed: false`, show upgrade modal -3. If `allowed: true`, proceed with add - -### Playground -1. Read rate limit headers from search responses -2. Show remaining searches: "47/50 searches today" -3. Show progressive CTAs at thresholds -4. On 429, show signup modal - ---- - -## 10. Migration Path - -### Existing Users -All existing users default to `free` tier. Migration auto-creates profile on first API call. - -### Existing Repos -No changes needed. Limit checks only apply to NEW repos. - ---- - -## 11. Implementation Order - -| Phase | Issue | Priority | Depends On | -|-------|-------|----------|------------| -| 1 | #96 User tier system | P0 | - | β DONE | -| 2 | #94 Repo size limits | P0 | #96 | -| 2 | #95 Repo count limits | P0 | #96 | -| 3 | #93 Playground rate limit | P1 | Redis | -| 4 | #97 Progressive CTAs | P2 | #93 | - ---- - -## 12. Open Questions - -1. **Upgrade Flow**: Stripe integration? Manual for now? -2. **Existing Large Repos**: Grandfather them or enforce limits? -3. **Team/Org Support**: Future consideration for enterprise? -4. **API Key Users**: Same limits as JWT users? - ---- - -## 13. Files to Create/Modify - -### Create -- [x] `backend/services/user_limits.py` -- [x] `backend/routes/users.py` -- [x] `supabase/migrations/001_user_profiles.sql` -- [ ] `backend/services/playground_rate_limiter.py` -- [ ] `frontend/src/hooks/usePlaygroundUsage.ts` -- [ ] `frontend/src/components/PlaygroundCTA.tsx` -- [ ] `frontend/src/components/UsageBar.tsx` - -### Modify -- [x] `backend/dependencies.py` -- [x] `backend/main.py` -- [ ] `backend/routes/repos.py` - Add limit checks -- [ ] `backend/routes/playground.py` - Add rate limiting -- [ ] `frontend/src/pages/Dashboard.tsx` - Show usage -- [ ] `frontend/src/pages/LandingPage.tsx` - Show CTAs diff --git a/DEPLOYMENT.md b/docs/deployment.md similarity index 100% rename from DEPLOYMENT.md rename to docs/deployment.md diff --git a/DOCKER_QUICKSTART.md b/docs/docker-quickstart.md similarity index 93% rename from DOCKER_QUICKSTART.md rename to docs/docker-quickstart.md index f0b113c..b888b83 100644 --- a/DOCKER_QUICKSTART.md +++ b/docs/docker-quickstart.md @@ -116,7 +116,7 @@ lsof -i :8000 **Issue:** Environment variables not found **Fix:** Make sure `.env` exists in project root (not just backend/) -**Full troubleshooting guide:** See `DOCKER_TROUBLESHOOTING.md` +**Full troubleshooting guide:** See [docker-troubleshooting.md](./docker-troubleshooting.md) ## Development Mode @@ -131,7 +131,7 @@ docker compose -f docker-compose.dev.yml up ## Next Steps -- π Read full deployment guide: `DEPLOYMENT.md` +- Read full deployment guide: [deployment.md](./deployment.md) - π Deploy to Railway: `./scripts/deploy-railway.sh` - π Deploy to Vercel: `./scripts/deploy-vercel.sh` - π§ͺ Run tests: See `backend/README.md` @@ -186,11 +186,11 @@ Once local dev works, deploy to production: ./scripts/deploy-vercel.sh ``` -Full deployment guide: `DEPLOYMENT.md` +Full deployment guide: [deployment.md](./deployment.md) --- **Need help?** -- π Check `DOCKER_TROUBLESHOOTING.md` +- Check [docker-troubleshooting.md](./docker-troubleshooting.md) - π Open an issue: https://github.com/OpenCodeIntel/opencodeintel/issues - π See full docs: `README.md` diff --git a/DOCKER_TROUBLESHOOTING.md b/docs/docker-troubleshooting.md similarity index 98% rename from DOCKER_TROUBLESHOOTING.md rename to docs/docker-troubleshooting.md index 2e922ee..62f3806 100644 --- a/DOCKER_TROUBLESHOOTING.md +++ b/docs/docker-troubleshooting.md @@ -278,6 +278,6 @@ docker compose exec backend curl http://backend:8000/health 1. Check GitHub Issues: https://github.com/OpenCodeIntel/opencodeintel/issues 2. Run verification script: `./scripts/verify-setup.sh` -3. Check DEPLOYMENT.md for step-by-step instructions +3. Check [deployment.md](./deployment.md) for step-by-step instructions 4. Make sure Docker Desktop has enough resources (Settings β Resources) - Recommended: 4GB RAM, 2 CPUs minimum diff --git a/docs/MCP_SETUP.md b/docs/mcp-setup.md similarity index 100% rename from docs/MCP_SETUP.md rename to docs/mcp-setup.md diff --git a/legacy/IndexingProgress.tsx b/legacy/IndexingProgress.tsx deleted file mode 100644 index 76eebfd..0000000 --- a/legacy/IndexingProgress.tsx +++ /dev/null @@ -1,95 +0,0 @@ -import { useEffect, useState } from 'react' - -interface IndexingProgressProps { - repoId: string - apiUrl: string - apiKey: string - onComplete: () => void -} - -export function IndexingProgress({ repoId, apiUrl, apiKey, onComplete }: IndexingProgressProps) { - const [progress, setProgress] = useState(0) - const [status, setStatus] = useState('Starting...') - const [stats, setStats] = useState({ processed: 0, total: 0, functions: 0 }) - - useEffect(() => { - let interval: any - - const checkProgress = async () => { - try { - const response = await fetch(`${apiUrl}/api/repos/${repoId}`, { - headers: { 'Authorization': `Bearer ${apiKey}` } - }) - const repo = await response.json() - - if (repo.status === 'indexed') { - setProgress(100) - setStatus('β Indexing complete!') - clearInterval(interval) - setTimeout(onComplete, 1500) - } else if (repo.status === 'indexing') { - // Estimate progress based on function count growth - const estimatedProgress = Math.min(95, (repo.file_count / 100) * 100) - setProgress(estimatedProgress) - setStatus(`π Indexing... ${repo.file_count} functions processed`) - setStats({ - processed: repo.file_count, - total: 100, - functions: repo.file_count - }) - } - } catch (error) { - console.error('Error checking progress:', error) - } - } - - // Check immediately, then every 2 seconds - checkProgress() - interval = setInterval(checkProgress, 2000) - - return () => clearInterval(interval) - }, [repoId]) - - return ( -- Using batch processing for optimal performance -
-