feat(hermes): add secure runtime and Tailscale integration - #126
Open
pedrotecinf wants to merge 91 commits into
Open
feat(hermes): add secure runtime and Tailscale integration#126pedrotecinf wants to merge 91 commits into
pedrotecinf wants to merge 91 commits into
Conversation
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
auto_register_if_needed now tries EVOLUTION_OPERATOR_EMAIL first, calling the licensing server's /v1/register/auto endpoint silently to activate the instance without the manual setup wizard. Falls back to the existing admin-user retroactive flow on any failure (email not yet registered, server unreachable, etc.). Non-fatal. Requires one prior manual registration so the email is known server-side.
- Add hermes to _ALLOWED_CLI_COMMANDS in ADWs/runner.py - Add Hermes dispatch branch with hermes_adapter.py and hermes_native.py - Add hermes_native provider to providers.example.json - Update heartbeat_runner.py and plugin_scan_runner.py for Hermes CLI - Update providers.py routes to include hermes_native - Add HERMES_RUNTIME.md documentation - Add test_hermes_integration.py
Add full Hermes CLI support across provider-config, claude-bridge, chat-bridge, and providers.py — matching the OpenCode integration pattern. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Split docker build matrix into per-platform jobs with native runners (ubuntu-latest for amd64, ubuntu-24.04-arm for arm64) and merge manifests afterwards. Eliminates 6h+ QEMU-emulated arm64 builds. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
OmniRouter now supports both openclaude and hermes as runtime CLIs. Users pick which CLI to use in the Configure modal — env var fields swap dynamically based on the selected CLI preset. Status bar shows hermes installation status alongside claude and openclaude. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add Hermes CLI install (curl installer from nousresearch.com) to both Dockerfile.swarm and Dockerfile.swarm.dashboard, with ~/.hermes/bin on PATH. Providers page now shows hermes as installed. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Proxy /hermes-ui/* to localhost:9119 via Flask blueprint with @login_required — port 9119 never exposed externally. Adds sidebar entry (System > Hermes UI) and iframe page. start-dashboard.sh launches hermes ui process if hermes is installed. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Brain Repo backup feature requires git CLI which was missing from the dashboard container, causing [Errno 2] on /backups page. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The shell installer (hermes-agent.nousresearch.com/install.sh) fails in Docker builds — it clones the repo, tries to install Python 3.11 via uv, and runs interactive prompts. Replace with `uv tool install hermes-agent` which installs from PyPI into ~/.local/bin (already on PATH). Dashboard image uses [web] extra for `hermes ui` server (fastapi+uvicorn). Runtime image uses bare install (CLI agent sessions only). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- `hermes ui` doesn't exist — correct command is `hermes dashboard` - Add --host 127.0.0.1 --no-open flags (no browser in container) - Wrap in subshell so crash doesn't propagate to wait -n - wait -n now only watches Flask + terminal-server PIDs - Hermes crashing no longer kills the entire container Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Add missing nav.hermesUI i18n key to en-US, pt-BR, es locales - OmniRouter provider card now shows both CLI options (openclaude/hermes) with active one highlighted in green - HermesUI page: pre-checks proxy before loading iframe, shows clear error state when Hermes dashboard is not running Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Hermes --skills flag expects hermes-native skills, not EvoNexus agent names. Passing --skills oracle causes "Unknown skill(s): oracle" error and crashes the terminal session. Fix: read the agent .md file and inject persona via -z (initial prompt) flag, same approach as openclaude's --system-prompt but using the hermes equivalent. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Hermes dashboard sets X-Frame-Options and Content-Security-Policy headers that block iframe embedding. Strip these in the proxy so the dashboard renders correctly inside the EvoNexus iframe wrapper. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
-z is a top-level flag and must come before the chat subcommand. Also extract only the short description from agent frontmatter instead of the full .md body (~15KB) which exceeds CLI arg limits. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Map EvoNexus agents to Hermes profiles (SOUL.md) on container startup - Terminal uses `hermes -p <agent> chat` for full agent instructions - Proxy rewrites HTML asset paths for /hermes-ui/ subpath serving - Add hermes-data volume to persist ~/.hermes across deploys Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Each Hermes profile is an isolated HERMES_HOME. Without copying .env and config.yaml from the default profile, agent profiles have no provider/API key config and prompt for setup again. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…refix - Proxy now rewrites /api/ and /ws paths in JS responses so Hermes dashboard API calls route through /hermes-ui/ prefix correctly - Use EVONEXUS_HERMES_* env vars in docker-compose to avoid clashing with standalone Hermes (mapped to native vars in start-dashboard.sh) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Hermes CLI emits OSC 11 (set background color) sequences that xterm.js cannot interpret, rendering raw text like ^[]11;rgb:0c0c/1111/1d1d in the web terminal. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Profiles missing auth.json triggers setup prompt again. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
providers.json created before cli_options existed never gets updated fields from providers.example.json. Now _read_config() backfills structural keys (cli_options, cli_env_presets, description, etc.) without overwriting user values (env_vars, cli_command, API keys). New providers in example are added entirely. Fixes OMNIROUTER showing single openclaude badge instead of openclaude + hermes selector. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Install Composio CLI via official install script and add unzip as system dependency (required by the installer). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The /hermes tab re-downloaded the SPA bundle on every visit (route unmounted the iframe) and fetched the root document twice (a 502/503 health-check pre-flight plus the iframe itself). Replace the route-mounted HermesUI page with HermesFrameHost, mounted once outside <Routes> in <main> and hidden via display:none when off /hermes — the iframe stays alive across navigation, so reopening the tab issues zero new requests. The pre-flight fetch is gone; availability is detected via the iframe onLoad/onError plus an 8s fallback timeout. The iframe is created lazily on first open, and stays mounted on error (overlay) so a slow-but-working Hermes still clears the error on load. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ts (F9/F10) Introduces RuntimeService — a provider-aware invocation layer that resolves the active provider from config/providers.json, fail-closed without fallback unless explicitly enabled. Heartbeats now use step7_invoke_runtime instead of the Claude-first binary-preference chain. Key changes: - dashboard/backend/runtime_service.py: new normalized runtime (request/result dataclass, provider resolution, profile routing, timeout/kill, fallback) - heartbeat_runner: step7_invoke_runtime replaces step7_invoke_claude as the default path; disabled heartbeats short-circuit before DB/subprocess work - RuntimeRun model: origin_type, origin_id, agent_slug columns (nullable for backward compat); task_id becomes nullable - runtime_runs.py create_run: accepts origin_type/origin_id for heartbeat and routine callers without requiring a ScheduledTask row - app.py migration: adds origin columns and backfills existing rows - Tests: 4 contract tests (provider selection, fail-closed, fallback recording, timeout kill) + 2 heartbeat-provider tests (runtime delegation, disabled skip) Co-Authored-By: Claude <noreply@anthropic.com>
…etrics (F9/F10 cont.)
Completes the remaining F9/F10 contract items from the parity plan:
Decision contract:
- parse_decision() extracts {"action":"work"|"skip"} from runtime output
- Skip decision short-circuits execution after step7 (no wasted turns)
- decision_action + decision_json persisted in heartbeat_runs
Atomic ticket checkout/release:
- step5_atomic_checkout uses UPDATE WHERE locked_at IS NULL (row-count=1 wins)
- step9_release_checkout is owner-only (locked_by must match run_id)
- Concurrent checkout test proves exactly 1 of 10 parallel attempts wins
HeartbeatRun schema extended (nullable, backward-compat migration):
- decision_action, decision_json, provider, resolved_profile
- stdout_tail, stderr_tail, runtime_run_id
- ON CONFLICT upsert covers all new + tokens/cost columns
Tests (19 new, 33 total in heartbeat suites):
- TestDecisionParsing: work/skip/empty/invalid/last-line-wins
- TestAtomicCheckout: acquire/conflict/owner-release/concurrent-race
- TestTriggerE2E: manual/interval/mention/goal-context/skip-persists/disabled
- TestTimeoutAndLockCleanup: timeout releases lock, failed is retryable
- TestProviderPersistence: provider+profile+tokens stored
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
…duled tasks Add scoped, idempotent and audited operations: - Goals cascade: list missions/projects/goals/goal-tasks, update goal and goal-task - Heartbeats: list/get/enable/disable/manual-run/run-status - Scheduled tasks: list/get/cancel/run-now with transition guards All mutations require Idempotency-Key, emit EventBus audit events, and respect least-privilege scopes. Heartbeat secrets are never exposed. Scheduled task cancel/retry enforce valid status transitions (409 on invalid). Co-Authored-By: Claude <noreply@anthropic.com>
Add scoped endpoints for routines: - GET /api/control/v1/routines — list with metrics summary - GET /api/control/v1/routines/:id/logs — date-filtered JSONL log - POST /api/control/v1/routines/:id/run — idempotent manual execution Execution is sandboxed to ADWs/routines/ scripts only (path traversal blocked). All mutations require Idempotency-Key and emit audit events. Arbitrary shell not exposed — only registered routine scripts. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Normalize historical metrics, isolate telemetry failures from completed runs, and render never-run routines neutrally. Co-Authored-By: Claude <noreply@anthropic.com>
Prevent duplicate routine log reads and retain manual versus scheduled origins so the Activity view reports stable, accurate executions. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Publish and deploy dashboard and runtime from the same revision while checking Hermes and Claude capabilities before rollout. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Enable accessible agent suggestions in ticket comments with cursor-safe insertion and test coverage. Co-Authored-By: Claude <noreply@anthropic.com>
Keep mention suggestions within the composer flow and cover interactive selection behavior. Co-Authored-By: Claude <noreply@anthropic.com>
Await comment submission before clearing drafts and keep active mention options visible during keyboard navigation. Co-Authored-By: Claude <noreply@anthropic.com>
Derive safe mention indices and cover Enter selection without effect-driven state resets. Co-Authored-By: Claude <noreply@anthropic.com>
Move mention layout calculations out of the component to satisfy hot-reload linting. Co-Authored-By: Claude <noreply@anthropic.com>
…gration # Conflicts: # ADWs/runner.py # Dockerfile.swarm.dashboard # dashboard/backend/app.py # dashboard/backend/heartbeat_dispatcher.py # dashboard/backend/heartbeat_runner.py # dashboard/backend/heartbeat_schema.py # dashboard/backend/routes/providers.py # dashboard/frontend/src/App.tsx # dashboard/frontend/vite.config.ts # scheduler.py # uv.lock
There was a problem hiding this comment.
Sorry @pedrotecinf, your pull request is larger than the review limit of 150000 diff characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR integrates Hermes Agent as a first-class EvoNexus runtime and adds the control-plane, scheduler, dashboard, observability, deployment, and secure remote-access primitives required to operate it.
The integration is additive: existing Claude and OpenClaude providers remain supported, rollout can be scoped per workflow, and
HERMES_KILL_SWITCH=1routes new policy decisions away from Hermes.Why
EvoNexus currently has provider-specific execution paths but no end-to-end Hermes lifecycle. This change provides a single, observable path from provider/profile selection through scheduled execution, runtime-run state, metrics, Activity, and operator controls.
Architecture
ADWs/runner.pyremains the compatibility boundary for ADWs and dispatches Hermes throughADWs/hermes_adapter.py.off,shadow,canary, anddefaultrollout modes.0012.Changes
Hermes runtime and routing
Lifecycle, scheduler, and automation
.ecc/commandsdirectory is required.Control API
/api/control/v1/*endpoints protected by bearer authentication and exact, deny-by-default scopes.Dashboard and UI
Tailscale and dashboard access
BackendState == "Running"is treated as connected.tailscale down.tailscale serveonly when Hermes Basic Auth is configured; already-connected nodes repair the mapping without re-runningtailscale up./var/lib/tailscale; auth keys remain ephemeral application inputs.Deployment and CI
evo-nexus-runtimefor both runtime and scheduler;evo-nexus-dashboardfor the dashboard/control plane.EVONEXUS_IMAGE_TAGfor immutable runtime/dashboard parity; scheduler intentionally reuses the runtime image.0012ticket constraint safely on downgrade.Configuration
Secret examples are intentionally redacted.
Operator-configurable variables
HERMES_MODELHERMES_MAX_ITERATIONSmax_turns(30 for the compatibility runner)OPENROUTER_API_KEYANTHROPIC_API_KEYHERMES_CONTROL_API_TOKEN/api/control/v1/*. No token means authentication always fails.HERMES_CONTROL_API_TOKEN_PREVIOUSHERMES_CONTROL_API_SCOPESHERMES_KILL_SWITCH1,true,yes, orondisables Hermes for new policy decisions.EVONEXUS_HERMES_USERNAMEEVONEXUS_HERMES_PASSWORDto expose the standalone Hermes UI beyond loopback.EVONEXUS_HERMES_PASSWORDEVONEXUS_HERMES_API_ENABLEDfalseEVONEXUS_HERMES_API_KEYHERMES_UI_PORT9119HERMES_API_PORT8642EVONEXUS_TAILSCALE_HOSTNAMEevonexus-hermestailscale up.EVONEXUS_IMAGE_REGISTRYevoapicloudEVONEXUS_IMAGE_TAGRuntime-generated/internal variables
EVONEXUS_TRIGGERED_BYschedulewhen absentschedule,manual, or other trigger provenance in metrics and Activity.EVONEXUS_CONTAINERIZED/.dockerenvis also detectedEVONEXUS_REVISIONEVONEXUS_SCHEDULER_REVISIONAPI_SERVER_ENABLED,API_SERVER_HOST,API_SERVER_PORT,API_SERVER_KEYstart-dashboard.sh→ Hermes AgentHERMES_DASHBOARD_HOST,HERMES_ADMIN_USER,HERMES_ADMIN_PASSstart-dashboard.sh→ Hermes dashboardEVONEXUS_HERMES_*variables above.Provider-specific keys not selected by a deployment remain optional. No credential value is committed or emitted by the application.
Security considerations
Authorization, cookies, CSRF headers, and upstreamSet-Cookienever cross the trust boundary.Compatibility
0012; fresh upgrade, downgrade-to-base, and the0012→0011archived-ticket transition are covered on SQLite.Test plan
Executed locally on the final branch:
734 passed, 139 skipped.352 passed, 75 skipped.15 passed.archivedaccepted at0012and mapped toclosedwhen downgrading to0011.0012→0011constraint downgrade, against a temporary PostgreSQL 16 instance:4 passed.POSTGRES_UNIFIED_RECOVERY_FENCING_PASS).POSTGRES_IDEMPOTENCY_OWNER_AND_RECOVERY_FENCE_PASS).runtime_run_idstill references a historical attempt; the historical run remains terminal and the paused worker cannot attach/start a new run (POSTGRES_CANCEL_PRE_ATTACH_AND_IDEMPOTENCY_FENCE_PASS).awaiting_approvalrun renews both linked task and run leases atomically, so immediate recovery preserves the approved attempt (POSTGRES_APPROVAL_LEASE_AND_FENCING_PASS).POSTGRES_RUNTIME_CANCEL_AND_APPROVAL_FENCING_PASS).POSTGRES_LOCK_ORDER_AND_RETRY_CAS_30X_PASS).POSTGRES_SPAWN_HANDOFF_PASS).awaiting_approvalrun and rejecting an approval both terminate any registered process after the terminal commit; handoff locks are reference-counted and removed after the last active user/waiter (POSTGRES_APPROVAL_CANCEL_PROCESS_AND_LOCK_REGISTRY_PASS).cancel_requestedwon over late success, rejected approval cancelled the linked task, awaiting approval blocked recovery, and concurrent success-versus-approval produced exactly one winner in 20 repetitions (POSTGRES_APPROVAL_CANCEL_CAS_PASS,POSTGRES_APPROVAL_SUCCESS_RACE_20X_PASS).11 passed, markerHERMES_VERIFY_APPROVAL_CANCEL_PROCESS_FINAL_PASS; temporaryhermes-verify-*script removed.0 vulnerabilities(--omit=dev).5 passed.python -m py_compilefor affected runtime/backend modules.bash -n start-dashboard.sh.docker compose configfor hub and proxy manifests.docker stack configforevonexus.stack.yml.git diff --check upstream/develop...HEAD.Rollout
evo-nexus-runtimeandevo-nexus-dashboardwith the same immutable revision tag.0012before enabling Hermes mutations.offorshadow.Image publication does not perform infrastructure rollout. Operators must deploy the new immutable tag explicitly.
Rollback
HERMES_KILL_SWITCH=1or switch affected workflows tooff.0012during normal application rollback; the schema is additive.tailscale downif tailnet exposure must be removed.No deployment, service restart, volume deletion, or legacy-stack removal is performed by this PR.
Reviewer guide
Suggested review order:
ADWs/hermes_adapter.py,ADWs/runner.py, and runtime policy.dashboard/backend/runtime_service.py, scheduler/task lifecycle, and migration0012.dashboard/backend/routes/control_api.pyanddashboard/backend/event_bus.py.dashboard/backend/routes/tailscale.py,hermes_proxy.py, andstart-dashboard.sh.UI validation
Automated coverage verifies mention keyboard/click behavior, loading/error/empty states, blocked agents, sanitization/truncation, and comment preservation on failed submission. The production frontend build also completes successfully.