Feature/router comparison

[shenrunzhang]

Summary

• load the metadata/metrics from routers.json and routerMetrics/category_scores.json and update routerData.ts to with data loaders to load real data for use in router compare modal
• add compare router feature that allows comparison of multiple router metrics at domain and category level of stratification and cross difficulty levels.
• added spider chart, grouped bar charts for difficulty levels, deferral chart for cost efficiency view.
• add the Hugging Face shortcut to the nav bar

Compare / Visualizations

• add cost-aware axis scaling for both the spider chart and the difficulty bars so tiny cost values remain legible.
• force the compare view into a grouped bar chart when there are fewer than three axes, or when a category has no subcategories.
• pin “Computer science, information, and general works” to the top angle so labels stop rotating between selections.
• disable the spider chart entirely when a scope has no subcategories, and add inline router labels + hover fixes to the deferral and difficulty charts.
• restyle the selection chips to share the router colors shown in the charts.

Leaderboard / Model Cards

• remove the old heat map, the “Type” column, and the explicit compare column to simplify the table.
• highlight the best numeric value in each column, and add a snap target at the recommended beta value on the slider.
• redesign the model card so the “Add to compare” action now opens the compare modal preloaded with just that router.

[shenrunzhang]

/gemini summary
/gemini review

[gemini-code-assist]

Hello @shenrunzhang,

It looks like you've invoked the /gemini summary and /gemini review commands. These commands are processed by the Gemini Code Assist bot to provide a pull request summary and a code review.

As a comment responder, I am unable to execute these commands directly. The Gemini Code Assist bot should process your request for a summary and review shortly.

If you have any specific questions about the code or would like me to provide feedback on a particular aspect, please ask directly in a comment, and I'll do my best to assist!

[shenrunzhang]

/gemini summary

[gemini-code-assist]

Summary of Changes

This pull request significantly enhances the router analysis capabilities by introducing a dedicated comparison modal. Users can now delve into detailed metric comparisons across different categories and difficulty levels using a suite of new, interactive visualizations. The underlying data infrastructure has been updated to power these features, and the overall user experience on the leaderboard and model cards has been refined for better clarity and functionality.

Highlights

• Router Comparison Feature: Introduced a new, comprehensive router comparison modal that allows users to select and compare multiple routers based on various metrics, categories, and difficulty levels.
• Interactive Visualizations: Added interactive spider charts, grouped bar charts for difficulty breakdowns, and deferral curves to visually represent router performance and cost efficiency. These charts include cost-aware axis scaling and dynamic rendering based on selected data.
• Data Management and Structure: Refactored data loading to use routers.json and routerMetrics/category_scores.json, introducing new data structures and helper functions to support the detailed comparison metrics and categories.
• Leaderboard and Model Card Enhancements: Improved the main leaderboard table by removing obsolete columns, highlighting the best numeric values in each metric, and adding a snap target for the beta value slider. The model card has been redesigned, and its 'Add to compare' action now directly opens the comparison modal.
• Navigation Bar Update: A new 'Hugging Face' shortcut has been added to the main navigation bar for quick access.

Changelog

• src/components/CompareDeferralChart.tsx
  • Added new React component for rendering deferral curves, including log-scaled X-axis for cost and custom tooltip/label formatting.
• src/components/CompareModal.tsx
  • Added new React modal component to facilitate router comparison, managing state for selected metrics, difficulties, categories, and chart views.
  • Implemented dynamic rendering of spider charts, grouped bar charts, and deferral curves within the modal.
  • Included search functionality for routers and a dropdown for selecting comparison scope (overall or specific categories).
• src/components/DifficultyBarPanel.tsx
  • Added new React component for displaying grouped bar charts, specifically for visualizing metric performance across different difficulty levels, with cost-aware axis scaling.
• src/components/Header.css
  • Adjusted nav-desktop gap from 2rem to 0.5rem.
  • Modified nav-item padding from 0.75rem 1rem to 0.4rem 1rem.
  • Updated nav-mobile-toggle padding from 0.5rem to 0.1rem.
• src/components/Header.tsx
  • Updated imports to use Smile icon from lucide-react.
  • Added a 'Hugging Face' link to the navigation items, using the Smile icon and contactInfo.huggingface.
• src/components/RouterModelCard.tsx
  • Added new React component to display detailed information for a single router, including metrics, model pool, and external links.
  • Integrated 'Compare' button that triggers the comparison modal with the selected router.
• src/data/routerData.ts
  • Removed YAML and raw.macro dependencies.
  • Imported routersMetadataJson directly.
  • Introduced toRouterId helper function for consistent ID formatting.
  • Defined new types: DifficultyLevel, CompareMetric, DifficultyMetricMap, CompareSubcategory, CompareCategory, RouterCompareEntry.
  • Added computeCostScore function for normalizing cost metrics.
  • Implemented createDifficultyMetrics and buildCompareEntry functions to generate synthetic category and difficulty scores for comparison.
  • Normalized existing routerCategoryScores and exported new data structures: compareMetrics, compareDifficulties, routerMetricsById, compareRouterNames, routerIdToName, and compareRouterOptions.
• src/data/routers.json
  • Added new JSON file containing detailed metadata for various routers, including names, types, descriptions, affiliations, model pools, and URLs for paper, website, GitHub, and Hugging Face.
• src/pages/LeaderboardPage.css
  • Added new CSS classes for full-bleed leaderboard layout (.leaderboard-full-bleed, .leaderboard-full-bleed .leaderboard-container).
  • Introduced extensive styling for the new comparison modal (.compare-modal-overlay, .compare-modal, etc.), including responsive adjustments.
  • Added styles for scope dropdowns, axis labels, view toggles, and control sections within the comparison modal.
  • Defined styles for router model cards (.router-model-card, .router-card-header, etc.) and their elements.
  • Updated leaderboard table grid layout (--lb-grid) to include a 'Select' column and removed the 'Type' column.
  • Added styling for best metric values (.score--best, .score--best--invert).
  • Introduced styles for router type indicators (.router-type-indicator) and router name links (.router-name-link).
  • Added styles for difficulty bar panels and deferral chart panels.
• src/pages/LeaderboardPage.tsx
  • Updated imports to include new icons (Layers, LinkIcon, Unlock, Lock) and new components (RouterModelCard, CompareModal).
  • Introduced state for selectedCompareIds, isCompareModalOpen, and modelCardRouter to manage router comparison and model card display.
  • Defined SNAP_TARGET and SNAP_THRESHOLD for the beta slider to enable snapping to a recommended value.
  • Added MetricBestMap and isBestValue helper functions to identify and highlight the best metric values in the leaderboard table.
  • Implemented toggleCompareSelection, handleSoloCompare, and handleRemoveFromCompare functions for managing router selections for comparison.
  • Modified the leaderboard table to include a checkbox for comparison selection and updated router name links to open model cards.
  • Integrated the RouterModelCard and CompareModal components, conditionally rendering them based on user interaction.

Activity

• shenrunzhang requested a summary and review using /gemini summary and /gemini review.
• gemini-code-assist[bot] acknowledged the request but stated it cannot execute the commands directly.
• shenrunzhang re-requested a summary using /gemini summary.

[shenrunzhang]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive router comparison feature, including a new comparison modal with multiple chart visualizations. The implementation is extensive, adding several new components and significantly modifying the data handling and leaderboard page. My review focuses on improving maintainability and code clarity. I've identified some areas for improvement, such as refactoring a large component, removing unused code, and replacing magic numbers with named constants. Overall, this is a great feature addition with many well-thought-out UI/UX improvements.