Skip to content

feat(backend): WebSocket endpoint for real-time indexing progress (#149)#150

Merged
DevanshuNEU merged 4 commits into
OpenCodeIntel:mainfrom
DevanshuNEU:feature/149-websocket-indexing-progress
Jan 5, 2026
Merged

feat(backend): WebSocket endpoint for real-time indexing progress (#149)#150
DevanshuNEU merged 4 commits into
OpenCodeIntel:mainfrom
DevanshuNEU:feature/149-websocket-indexing-progress

Conversation

@DevanshuNEU

Copy link
Copy Markdown
Collaborator

Summary

Adds WebSocket support for real-time indexing progress in the playground. Instead of polling every 2 seconds, the frontend can now subscribe to a WebSocket and receive instant updates as each file is processed.

Changes

New Files

  • backend/routes/ws_playground.py - WebSocket endpoint handler
  • backend/tests/test_ws_playground.py - Test suite (7 tests)

Modified Files

  • backend/main.py - Register new WS route
  • backend/services/anonymous_indexer.py - Add Redis Pub/Sub publishing

Endpoint

WS /api/v1/ws/playground/{job_id}

Event Types

{"type": "connected", "job_id": "...", "message": "Listening for indexing events"}
{"type": "cloning", "repo_name": "flask", "message": "Cloning repository..."}
{"type": "progress", "files_processed": 25, "files_total": 100, "current_file": "src/app.py", "functions_found": 150, "percent": 25}
{"type": "completed", "repo_id": "anon_xxx", "stats": {...}, "message": "Indexing complete!"}
{"type": "error", "error": "clone_failed", "message": "...", "recoverable": false}

Architecture

Background Job → Redis Pub/Sub → WebSocket → Browser
                job:{id}:events

Tests

All 7 tests passing:

  • WebSocket rejects invalid job IDs
  • WebSocket handles missing Redis gracefully
  • Pub/Sub events published on status changes
  • Progress events include current file info
  • Completed events include stats
  • No duplicate PROCESSING events
  • Event types are strings (not enum objects)

Why WebSocket over Polling?

The killer feature is the streaming file list - showing each file as it's processed in real-time. Polling every 2 seconds can't deliver that smooth experience. For the HN launch demo, this creates the "watching the AI work" moment.

Closes #149

…enCodeIntel#149)

- Add /api/v1/ws/playground/{job_id} WebSocket endpoint
- Implement Redis Pub/Sub for real-time event streaming
- Publish events: connected, cloning, progress, completed, error
- Progress events include current_file for streaming file list
- Skip duplicate PROCESSING events (handled by update_progress)
- Add comprehensive test suite (7 tests)

Closes OpenCodeIntel#149
@vercel

vercel Bot commented Jan 5, 2026

Copy link
Copy Markdown

@DevanshuNEU is attempting to deploy a commit to the Dev's projects Team on Vercel.

A member of the Team first needs to authorize it.

- Fix: Must accept() before close() - was causing runtime errors
- Fix: Check job exists before subscribing (return 4404)
- Fix: Handle race condition - job may complete before WS connects
- Fix: Remove unused INITIAL_TIMEOUT_SECONDS constant
- Add: Idle timeout (120s) to prevent zombie connections
- Add: 6 new tests for edge cases (13 total)

Review by CTO caught these before production 🙏
- Test creates real indexing job and connects via WebSocket
- Verifies full event flow: connected → progress → completed
- Tests against zustand repo (47 files, ~1182 functions)
- Confirms real-time streaming works end-to-end

Tested manually - all events received correctly
The WebSocket E2E test requires:
- Running server
- Redis
- aiohttp (not in CI deps)

Moving to scripts/manual_ws_test.py so pytest doesn't collect it.
Unit tests in test_ws_playground.py still run in CI (13 tests).
@vercel

vercel Bot commented Jan 5, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Review Updated (UTC)
opencodeintel Ignored Ignored Preview Jan 5, 2026 8:53pm

@DevanshuNEU DevanshuNEU merged commit d0f2033 into OpenCodeIntel:main Jan 5, 2026
6 checks passed
DevanshuNEU added a commit to DevanshuNEU/opencodeintel-fork-new that referenced this pull request Jan 5, 2026
)

🚀 Features:
- useIndexingWebSocket hook with auto-reconnect & polling fallback
- Streaming file list - shows files appearing in real-time
- Enhanced IndexingProgress with phase indicators & glow effects
- IndexingComplete celebration screen with confetti & animated stats
- Integrated WebSocket into HeroPlayground component

📁 Files:
- NEW: hooks/useIndexingWebSocket.ts (WebSocket state machine)
- NEW: components/playground/IndexingComplete.tsx (celebration UI)
- UPDATED: components/playground/IndexingProgress.tsx (streaming file list)
- UPDATED: components/playground/HeroPlayground.tsx (WS integration)
- UPDATED: components/playground/index.ts (exports)

🎨 UX Highlights:
- Files stream in real-time as they're processed
- Phase indicator: Clone → Index → Done
- Glowing progress bar
- Confetti on completion
- Animated stat counters

Connects to backend WebSocket endpoint from PR OpenCodeIntel#150
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(backend): WebSocket endpoint for real-time indexing progress

1 participant