diff --git a/README.md b/README.md index 4fd8dc0..bd8d64e 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,6 @@ - # Project Overview -RouterArena is the first open platform enabling comprehensive evaluation and comparison of LLM routers. The platform features: - -- **Principled Dataset**: 8,400 queries across 9 domains and 44 categories using Dewey Decimal Classification and Bloom's taxonomy -- **Multi-Dimensional Metrics**: Arena Score, Cost Ratio, Optimality, Latency, and Robustness evaluation -- **Automated Framework**: Support for both academic and commercial routers with real-time leaderboard updates -- **Fair Comparison**: Unified evaluation protocol for transparent and reproducible results +RouterArena is the first open platform enabling comprehensive evaluation and comparison of LLM routers. Check out the [website](https://routeworks.github.io) for more details. ## Research Team @@ -20,42 +14,7 @@ RouterArena is the first open platform enabling comprehensive evaluation and com - Hongyi Liu (hl87@rice.edu) - Jiarong Xing (jxing@rice.edu) -## Website Features - -### 🏠 Home Page -- Project overview and key features -- Dataset statistics and difficulty levels -- Contact information and collaboration details -- Interactive leaderboard preview - -### 🏆 Leaderboard Page -- Comprehensive router rankings across multiple metrics -- Search and filter functionality -- Detailed performance metrics for each router -- Links to papers and source code - -### 📝 Submit Page -- Router submission form for evaluation -- Clear guidelines for academic and commercial routers -- Evaluation process explanation -- Contact information for collaboration - -### â„šī¸ About Page -- Detailed project methodology -- Research team information -- Citation information -- Contact and collaboration details - -## Technology Stack - -- **Frontend**: React 19.2.0 with TypeScript -- **Routing**: React Router DOM 6.8.1 -- **Icons**: Lucide React 0.263.1 -- **Charts**: Recharts 2.8.0 -- **Styling**: CSS3 with modern features -- **Build Tool**: Create React App - -## Getting Started +## For Developers ### Prerequisites @@ -90,7 +49,37 @@ npm run build This builds the app for production to the `build` folder. -## Project Structure +### Coding/PR Style Check + +We use pre-commit to ensure a consistent coding style. You can set it up by + +``` +pip install pre-commit +pre-commit install +``` + +Before pushing your code, please run the following check and make sure your code passes all checks. + +``` +pre-commit run --all-files +``` + +We have set up Google Gemini to review your code. You can use it by running the following command: + +``` +/gemini review +``` + +### Web Stack + +- **Frontend**: React 19.2.0 with TypeScript +- **Routing**: React Router DOM 6.8.1 +- **Icons**: Lucide React 0.263.1 +- **Charts**: Recharts 2.8.0 +- **Styling**: CSS3 with modern features +- **Build Tool**: Create React App + +### Project Structure ``` src/ @@ -114,8 +103,6 @@ src/ └── index.tsx # Application entry point ``` -## Key Components - ### Router Data Structure ```typescript interface Router { @@ -137,14 +124,6 @@ interface Router { } ``` -### Evaluation Metrics - -1. **Arena Score**: Overall performance combining accuracy and cost efficiency -2. **Cost Ratio Score**: Efficiency in cost optimization relative to optimal routing -3. **Optimality Score**: Frequency of selecting the most efficient model for each query -4. **Latency Score**: Router overhead and response time performance -5. **Robustness Score**: Stability against query perturbations and noise - ## Contributing We welcome contributions to the RouterArena platform! Please see our submission guidelines on the website or contact us directly. @@ -167,7 +146,7 @@ If you use RouterArena in your research, please cite our paper: ## Contact -- **Email**: yifan.lu@rice.edu +- **Email**: yifan.lu@rice.edu, jxing@rice.edu - **Paper**: https://www.arxiv.org/abs/2510.00202 ## Acknowledgments