v0.8.56: Ship DigitalOcean + Telegram remote-workbench setup

[Hmbown]

Goal

Ship a repeatable, documented "remote workbench in ~15 minutes" setup for users outside mainland China: a cheap US VPS running the CodeWhale runtime privately on 127.0.0.1 plus the Telegram long-polling bridge, controlled from a phone DM. This is the narrow implementation slice of #1990 (which stays the broad US-lane evaluation issue).

Cloud decision: DigitalOcean, not AWS Lightsail. #1990's acceptance criteria allow "a clearly documented better alternative" — DO won on auth friction (one API token + doctl auth init vs interactive IAM/credential setup, which actively blocked the first provisioning attempt). The Lightsail scripts are kept as a working alternative lane in the same directory.

Working smoke evidence (2026-06-09, updating as the smoke completes)

Local pre-flight — all green:

• Telegram bridge unit tests: 23/23 pass (Node 22, zero prod npm deps).
• Runtime codewhale serve --http (v0.8.55): binds 127.0.0.1 by default; auth matrix verified — /v1/threads → 401 (no token), 401 (wrong token), 200 (bearer token); /health + /v1/runtime/info deliberately tokenless.
• validate-config.mjs: exit 0 on realistic config, exit 1 on placeholder templates.
• Audit (5 parallel review agents over bridge/deploy/runtime/prior-art): nothing in scripts/tencent-lighthouse/ is Tencent-specific — bootstrap/install/doctor run unmodified on a generic Ubuntu 24.04 VPS.

Cloud smoke — in progress:

• Droplet provisioned via new scripts/remote-smoke/digitalocean/provision.sh (sfo3, s-1vcpu-2gb $12/mo, Ubuntu 24.04, cloud firewall = inbound SSH from operator IP /32 only, price/preflight from the DO API, typed-yes billing gate)
• VM setup (setup-vm.sh): bootstrap + sha256-verified v0.8.55 release binaries + services + validator + doctor
• Phone smoke: /status, /menu, real prompt, /threads + Resume, approval /allow+/deny (buttons and text), /interrupt
• Allowlist enforcement check (unlisted chat refused)
• Service restart + droplet reboot persistence
• Teardown executed with leftover-billable-resources check

This section will be updated with the completed checklist before this issue ships.

Scope

• Land scripts/remote-smoke/ (currently untracked, working):
  • setup-vm.sh — provider-agnostic one-shot VM setup; installs prebuilt release binaries (sha256-verified) instead of a 30-min Rust build, making 1 vCPU / 2 GB plans sufficient
  • digitalocean/{provision,teardown}.sh (active lane), aws-lightsail/{provision,teardown}.sh (alternative)
  • README.md — run order, security model, cost table, China note, audit sharp-edges
• Update docs/REMOTE_VM_US.md: DO quickstart, release-binary path, and the doc/binary corrections below.

Doc/binary mismatches found (must be fixed by this issue)

1. The Rust binary reads only DEEPSEEK_RUNTIME_TOKEN/--auth-token and --port; CODEWHALE_RUNTIME_TOKEN/CODEWHALE_RUNTIME_PORT work only because the systemd unit expands them into flags. docs/REMOTE_VM_US.md implies the env file works standalone — it doesn't for hand-started servers.
2. codewhale-runtime.service fails activation unless /home/codewhale/.codewhale and /home/codewhale/.deepseek exist (ReadWritePaths without - prefix); nothing creates them. setup-vm.sh pre-creates them; the unit or bootstrap should too.
3. No repo script builds/installs the binaries (codewhale delegates to codewhale-tui, so both are required); docs describe a manual rustup/cargo flow that the release artifacts make unnecessary.
4. docs/REMOTE_VM_US.md references stale branch codex/v0.8.53.

Non-goals

• Webhook mode / Cloudflare Tunnel / any public ingress (long polling only)
• Discord / Slack bridges (Remote workbench: evaluate US-first Cloudflare/AWS/Telegram lane #1990 records deferral)
• The codewhale remote-setup interactive wizard — follow-up issue, see below
• CNB deploy-button automation; mainland-China lane changes (Tencent HK + Feishu lane already covers it — Telegram is blocked in mainland China and DO has no China DCs; README documents the split)

Acceptance criteria

• Fresh DO account → working phone control in ≤ ~15 min using only the README run order.
• Full smoke checklist above recorded as passing in this issue.
• Runtime API never publicly reachable (cloud firewall SSH-only + ufw + 127.0.0.1 bind); doctor passes with 0 failures.
• Allowlist enforced: unlisted chat gets a refusal, no command execution.
• Teardown script leaves zero billable resources (droplet, volumes, snapshots, reserved IPs all empty) and is documented in the README.
• Secrets never appear in argv, shell history, logs, or the repo (scp'd chmod-600 file, shredded after landing in /etc/codewhale/*.env 0640).

Security notes

• Telegram long polling is outbound-only; the only inbound port anywhere is SSH (restricted to operator IP by default).
• TELEGRAM_ALLOW_UNLISTED=true exists for first pairing only; setup locks the allowlist when an ID is supplied up front.
• --insecure (auth-off) flag must never be used on a VPS; doctor verifies auth_required=true and localhost bind via /v1/runtime/info.
• Bot tokens are revocable via BotFather; runbook recommends rotation if a token may have been exposed during setup.

Teardown / cost cleanup (requirement)

scripts/remote-smoke/digitalocean/teardown.sh destroys the droplet, firewall, and imported SSH key, then lists droplets/volumes/snapshots/reserved-IPs so an empty result proves billing stopped. Same pattern in the Lightsail variant.

Links

• Evaluation issue: Remote workbench: evaluate US-first Cloudflare/AWS/Telegram lane #1990 (stays in v0.8.57)
• Docs: docs/REMOTE_VM_US.md, docs/REMOTE_SETUP_DESIGN.md, docs/TENCENT_CLOUD_REMOTE_FIRST.md
• Reused scripts: scripts/tencent-lighthouse/{bootstrap-ubuntu,install-services,doctor}.sh, deploy/tencent-lighthouse/systemd/, deploy/tencent-lighthouse/examples/
• Bridge: integrations/telegram-bridge/
• New (to land via PR): scripts/remote-smoke/

[Hmbown]

Smoke progress (2026-06-09, live DO droplet)

Working:

• Provision → setup → doctor: 0 failures on a fresh sfo3 s-1vcpu-2gb Ubuntu 24.04 droplet using scripts/remote-smoke/ (prebuilt v0.8.55 release binaries, sha256-verified — no Rust build; full repo clone of main in /opt/whalebro/codewhale, 2455 commits).
• Runtime: localhost-only bind confirmed on the VM, auth_required=true, bearer-token matrix verified.
• Bridge: long polling clean, /start+/help respond, allowlist locked to a single chat ID from first boot.
• Prompt → turn starts; approval.required surfaces in-chat with Allow once / Allow + remember / Deny buttons.
• Runtime approval API works: POST /v1/approvals/<id> with {"decision":"allow","remember":true} → {"ok":true,"delivered":true} and the turn resumed and completed normally.

Bug found (blocks the approval/interrupt acceptance criteria here): #2966 — the bridge's poll loop serially awaits each update, so a turn's SSE stream blocks the very approval callback the turn is waiting for; structural deadlock until the 15-minute turn timeout, after which the stale callback is dropped anyway. /interrupt, /allow, /deny, and all other chats are equally dead during any active turn. The live tap of "Allow + remember" hit exactly this; the operator delivered the approval via the runtime API as a workaround.

Follow-up filed: #2967 (streaming/resilience hardening — progress-edit, typing, MarkdownV2, chunking, offset/replay safety, backoff curves).

Remaining checklist for this issue: re-run approval + interrupt smoke after #2966 lands, allowlist-rejection observation from an unlisted account, reboot persistence, teardown with the leftover-billing check.