ADR-0020: Remove llmfit Dependency

Acharnite · 2026-06-27T12:11:26Z

Replaces the llmfit Rust binary (~5.8 MB) with three independent, simpler solutions:

Phase 1 — Add (New Functionality Alongside Old)

Python Hardware Detection (hardware.py) — tiered detection via psutil, nvidia-smi, rocm-smi, torch
llama-server -hf Serving — POST /api/local-backends/llamacpp/serve-hf with SSE progress streaming
Frontend UI — HF model repo input, quantization selector, guidance card, HF badge in llama.cpp card
Full test coverage — 21 hardware + serve-hf tests

Phase 2 — Remove (Old llmfit Code)

Removed llmfit install/uninstall endpoints (-445 lines from backends.py)
Removed /api/tools/recommendations endpoint (-78 lines from models.py)
Removed llmfit download endpoint (-269 lines)
Removed frontend: install/uninstall buttons, recommendations card, download UI
Simplified hardware detection to use Python detection only

Documentation

ADR-0020 promoted: Proposed → Accepted
ADR-0005: Added superseded note for llmfit sections
ADR-0018: Resolved deferred -hf decision to Accepted
VERSION: 1.6.0 → 1.7.0
Design doc: v1.8 → v1.9

Stats

+1,175 lines (new code: hardware.py, serve-hf, HF UI, tests)
-1,084 lines (removed: llmfit endpoints, recommendations, download)
Net: +91 lines — cleaner, no external binary dependency
616 tests passing (all existing + 21 new)

Closes issues #101, #102, #110 (llmfit bugs addressed by replacement)

…f serving Phase 1 (Add, don't break): - Add psutil dependency to pyproject.toml - Create hardware.py with tiered detection (platform + psutil + nvidia-smi/rocm-smi + torch) - Add POST /api/local-backends/llamacpp/serve-hf endpoint with SSE progress streaming - Add /api/hardware endpoint with llmfit fallback (backward compat) - Update /api/local-backends/llamacpp/status with hf_supported flag - Add serveHfFromHF() API client with SSE stream reader - Add HF serve form + guidance card in dashboard.html - Add serveHfAction() in settings.js with Python-first hardware detection - Add HF badge (🤗 HF Ready) in llama.cpp card - Full test coverage: 21 tests for hardware detection + serve-hf endpoint Part of ADR-0020: Remove llmfit Dependency

…ead code Reviewers found 2 blocking bugs + 1 cleanup: - Bug 1 (CRITICAL): Frontend hardware field paths were wrong — platform, cpu_model, gpu, ram_total_gb, etc. all read at wrong nesting or with wrong names. Entire Python hardware detection branch was dead code as a result. - Bug 2 (CRITICAL): SSE event name mismatch — backend sends 'hf_serve_complete' but frontend checked for 'install_complete'. Post-success actions never fired. - Minor: Removed dead-code 'if False' block in test_llamacpp.py

Phase 2 (Remove old code, keep new): Backend: - Removed llmfit install/uninstall endpoints from backends.py (-445 lines) - Removed /api/tools/recommendations endpoint from models.py (-78 lines) - Removed /api/local-backends/models/download and /download/progress endpoints - Updated /api/tools/status — no longer reports llmfit status - Kept llmfit fallback in /api/hardware for backward compatibility Frontend: - Removed installLlmfit, uninstallLlmfit, fetchModelRecommendations, getDownloadModelURL, getLlmfitInstallURL from api.js - Removed loadModelRecommendations(), install/uninstall/download actions from settings.js - Simplified loadHardwareInfo() to use Python detection only - Removed llmfit from backend listing in settings - Removed llmfit actions and recommendations card from dashboard.html Tests: - Removed llmfit endpoint tests from test_local_backends.py and test_web.py - Replaced llmfit cache paths with generic gguf paths in test_llamacpp.py Net: +32 lines, -1075 lines across 8 files

…gn doc - ADR-0020: Proposed → Accepted (v1.1→v1.2) - ADR-0005: Added Superseded by ADR-0020 deprecation note - ADR-0018: Resolved -hf deferred decision to Accepted per ADR-0020 - VERSION: 1.6.0 → 1.7.0 - CHANGES.md: Added v1.7.0 section - Design doc: v1.8→v1.9, ADR index updated - ADR README index: ADR-0020 status updated

…es 1-4) - #116: Empty/incomplete session dirs auto-cleaned on clear_completed() - Added deepresearch cleanup output [--dry-run] CLI command - 17 new tests for output cleanup logic (685 total) - ADR-0019 Phases 1-4: Alpine.js reactive DOM, Alpine.store() state, SSE-to-Alpine bridge, removed ~250 LOC of manual DOM manipulation - Bugs #103 (model lists refresh), #104 (transparent picker), #110 (API cleanup) - VERSION 1.9.0, CHANGES.md updated

…olish Closes #52 (Q&A interaction graph) — the last v0.15.0 blocker Closes #105 (Epic v0.15.0) — all groups complete Closes #106 (Lifecycle controls moved to Local Backends tab) Closes #107 (Streamlined Serve & Connect UX) Phase A: Scribe model prefix fix (full model ID preserved) Phase B: ADR-0018 Phases 2-3 — llama.cpp frontend integration - Lifecycle controls in Local Backends tab - Serve & Connect auto-refresh model dropdown - LiteLLM integration for model visibility Phase C: 24 new pipeline tests (CLI + Dashboard + Error handling) Phase E: CI polish — ESLint, coverage badges, Docker verify 685 + 24 + 4 = 709 tests passing

- 37 provider tests: OpenRouter, OpenAI, OpenCode Go/Zen routing - deepresearch run --benchmark flag for performance timing - scripts/benchmark-pipeline.sh for automated benchmarks - Log file monitoring and memory isolation tests - Closes #52 (Q&A graph), #106 (lifecycle controls), #107 (serve UX) - 709 + 37 = 746 total tests passing

… work Fixes documentation gaps found in final audit: - TODO.md: 9 unchecked items now marked done - README badge: 486 -> 746 tests - Design doc v2.1 with full changelog for v1.9.0 release - ADR-0018 v1.4 with Phases 2-3 frontend completion - ADR-0020 Phase 2 already reflected as complete (verified)

Acharnite added 10 commits June 27, 2026 13:42

fix(lint): remove unused import in test_serve_hf_builds_correct_command

a1ec2d0

style: ruff format — fix formatting in test files

c43d1ac

Acharnite merged commit bc70534 into master Jun 30, 2026
1 of 4 checks passed

Navigation Menu

ADR-0020: Remove llmfit Dependency — Python HW Detection + llama-server -hf - #115