feat: AI-first scraping with MCP integration and modern development environment #73
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Fix progressive image blur logic to show crisp images when no low-quality version exists - Consolidate DEVOTEAM_LOCATION coordinates to single source of truth - Update restaurant sorting to prioritize distance over dish count with alphabetical tiebreaker - Strengthen AI menu extraction boundaries to prevent bar section inclusion - Add priority-based extraction rules for "Veckans" weekly specials - Enhance extraction prompt with hard stop rules for better section filtering - Fix TypeScript import extensions and boolean type coercion 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add explicit anti-hallucination rules to prevent AI from adding common dishes like Caesar salad or Moo Tod that don't exist on menus - Remove problematic examples from extraction prompts that were causing false suggestions - Set temperature to 0.0 for maximum determinism and reduced creativity - Add PDF text cleaning function to fix common OCR errors (0läsk -> fläsk, con0iterad -> confiterad) - Update system prompts to emphasize extracting only dishes that actually appear in content - Clean up tsconfig files across packages for better module resolution - Simplify geolocation handling in Sort component with graceful fallbacks - Remove file extensions from server imports for cleaner code 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Update all major dependencies to latest versions (Vite 7, Vitest 3, TypeScript 5.7) - Fix SVG imports for vite-plugin-svgr v4 compatibility (?react syntax) - Replace deprecated Puppeteer waitForTimeout with Promise-based approach - Update server module resolution for ESM compatibility with tsx - Fix ESLint configuration to ignore Puppeteer cache files - Resolve all dependency security vulnerabilities - Add proper TypeScript declarations for SVG React components 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add .editorconfig for consistent coding standards across IDEs - Add .nvmrc to pin Node.js 22.14.0 for team consistency - Update Docker to Node.js 22 LTS and pnpm 10 for performance - Enhance .dockerignore with modern exclusion patterns - Add engine requirements (Node.js >=22, pnpm >=10) to package.json - Upgrade Vitest configuration with coverage reporting and v8 provider - Add comprehensive testing dependencies (@testing-library, jsdom) - Create test setup file with automated cleanup - Modernize GitHub Actions to latest versions (checkout@v4, setup-node@v4, pnpm@v4) - Add coverage reporting and security audit to CI pipeline - Update Turbo configuration for better test caching 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Changes moduleResolution from "node" to "bundler" to match base config - Adds setTimeout and clearTimeout to browser globals in ESLint config 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
…upport Improved the notify-slack function with better formatting, threading support, and comprehensive restaurant coverage including multi-location venues. Key improvements: - Implemented threaded messaging to reduce message size (8 restaurants per thread) - Added support for multi-location restaurants (now shows all 39 restaurants) - Enhanced markdown formatting with proper Slack syntax - Added visual hierarchy with dish type emojis (🥩🐟🥗🌱) - Fixed clickable link formatting for malmolunch.se - Improved TypeScript configuration for ESNext modules - Removed external dependency on @devolunch/shared package - Added rate limiting protection between API calls 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Removed unused getTodayNiceFormat function - Fixed setTimeout reference using globalThis for Node.js compatibility 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
The TypeScript build outputs to dist/apps/functions/scraper/src/ directory, so update the scrape script to use the correct path. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Added a 'dev' script to the scraper that pipes output through pino-pretty for better readability during development, replacing raw JSON logs with formatted, human-readable output. Changes: - Added 'dev' script to scraper package.json with pino-pretty formatting - Updated root 'scrape:dev' command to use the new dev script - Maintains structured JSON logging for production while improving DX 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Extract PromptPanel (1100+ lines) into 5 focused components: * ConversationArea: message display and rendering * MenuModal: restaurant menu popup with card-style design * ScrollToSection: navigation chevron component * SearchInput: search field with validation * SuggestionList: AI recommendation cards - Improve MenuModal design: * Restaurant name in header opposite close button * Distance and opening times below name * Website/Directions links under name * Dishes styled like restaurant cards with dots and borders * Better space utilization and left-aligned text - Remove auto-scrolling behavior from conversation results - Add missing DOM types to ESLint config - Move scroll-to-section out of PromptPanel for better separation 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR represents a complete modernization of DevoLunch, introducing AI-powered menu scraping, real-time communication capabilities, and a significantly improved development experience. The changes transform the application from a traditional web scraper to an intelligent, AI-driven system that adapts to website changes automatically.
🤖 AI-First Scraping Architecture
🔌 MCP (Model Context Protocol) Integration
packages/mcp-server/): Exposes DevoLunch data via MCP protocolget_restaurants: Returns current scrape datasearch_dishes: Searches dishes across all restaurantssuggest_meal: AI-powered meal suggestions with reasoning🎨 Frontend Improvements
New Components:
PromptPanel: Interactive AI conversation interfaceMenuModal: Enhanced menu viewing experienceSearchInput: Improved search with suggestionsSuggestionList: AI-powered meal suggestionsErrorBoundary: Comprehensive error handlingMcpStatus: Real-time MCP connection statusConversationArea: Chat-like interface for AI interactionsMulti-Location Support: Restaurants with multiple locations now properly supported with individual coordinates and maps
WebSocket Communication: Real-time updates between client and server
Better Error Handling: Component-level error boundaries with recovery
🏗️ Development Environment Modernization
Testing Migration: Replaced Jest with Vitest for faster, more modern testing
ESLint Upgrade: Migrated to ESLint flat config (v9+)
.eslintrc.jsonfilesLogging Improvements:
@devolunch/shared)TypeScript Enhancements:
tsconfigpackage in favor of root configBuild System:
📚 Documentation Overhaul
New Documentation Structure (
docs/):architecture.md: System architecture overviewdevelopment.md: Development workflows and guidelinesinfra.md: Infrastructure and deploymentDEPENDENCY_MANAGEMENT.md: Renovate configuration and automated updatesImproved README: Better organized with links to detailed docs
Scraper Documentation: Comprehensive guide for AI-powered scraping
Contributing Guide: Moved to
.github/CONTRIBUTING.mdwith better instructions🔧 Infrastructure & DevOps
Renovate Integration: Automated dependency management
GitHub Workflows:
Git Hooks:
Editor Configuration: Added
.editorconfigfor consistent code style🗄️ Backend Enhancements
New API Routes:
/api/v1/ai: AI interaction endpointsMCP Client Service: Server-side MCP client for browser automation
Storage Service Improvements: Better error handling and retry logic
Environment Configuration: Cleaner
.envhandling🔄 Refactoring & Code Quality
Scraper Refactoring:
Shared Packages:
Removed Deprecated Code:
📦 Package Management
🎯 Restaurant Coverage
Maintained and improved coverage for 20+ Malmö restaurants with simplified configuration:
Each restaurant now only requires metadata - AI handles the rest!
🧹 Breaking Changes
packages/eslint(use root config)packages/tsconfig(use root config)Technical Highlights
Migration Path
For developers working on this codebase:
pnpm install(enforced by preinstall script)docs/folder for architecture and development guidespnpm test(Vitest) instead of Jest@devolunch/sharedFuture Work
Future developers will need to address:
🔌 AI API Integration: The AI scraping system requires connection to an AI service (Claude API or similar) for dish extraction. The infrastructure is in place but needs:
💬 Slack Integration: The Slack notification service needs updates to work with the new data structure:
These integrations are required for full production functionality.
🤖 Generated with Claude Code
Co-Authored-By: Claude Sonnet 4.5 noreply@anthropic.com