Skip to content

[codex] add WebUI terminal support#712

Open
linyuww wants to merge 12 commits into
TencentCloud:masterfrom
linyuww:issue-643-web-terminal
Open

[codex] add WebUI terminal support#712
linyuww wants to merge 12 commits into
TencentCloud:masterfrom
linyuww:issue-643-web-terminal

Conversation

@linyuww

@linyuww linyuww commented Jul 2, 2026

Copy link
Copy Markdown

Summary

  • add CubeAPI terminal ticket and WebSocket endpoints for sandbox terminal sessions
  • add CubeMaster WebSocket bridge and Cubelet bidirectional terminal stream backed by containerd exec with TTY and resize support
  • add WebUI xterm dialog, running-container selection, reconnect/status controls, i18n, docs, and tests
  • wire CubeShim resize forwarding to the guest agent and validate controlling-terminal setup errors
  • address auto-review feedback for WebSocket origin validation, gRPC stream shutdown handling, bridge error handling, and audit operator identity

Validation

  • git diff --check
  • cd CubeAPI && cargo test --locked
  • cd CubeMaster && go test ./pkg/server ./pkg/service/httpservice/cube -run 'TestRegisterHandlersIncludesSandboxTerminalRoute|TestParseTerminalUint32|TestTerminalOriginAllowed'
  • cd CubeMaster && go test -run '^$' ./pkg/service/httpservice/cube ./pkg/service/sandbox
  • cd Cubelet && go test ./services/cubebox -run 'TestTerminal|TestTerminalIdleExpired|TestTerminalStreamClosedCleanly'
  • cd web && npm run lint && npm run test && npm run build
  • cd CubeShim && cargo check --locked
  • cd agent && make -j1 MUSL=no
  • local dev VM smoke: terminal session opens through CubeAPI/WebSocket, ticket reuse is rejected, concurrent sessions are isolated, test -t 0 succeeds, stty size follows resize to 31x97, and audit logs do not contain command contents

Notes

  • Reconnect intentionally starts a fresh shell session.
  • Some minimal guest images may still make the tty command print not a tty because the PTY slave path cannot be resolved through the container /dev/pts mount; test -t 0, stty size, and full-screen programs verify the PTY behavior. This is documented as a known limitation.

Related to #643

Assisted-by: Codex:GPT-5

Assisted-by: Codex:GPT-5
@cubesandboxbot

cubesandboxbot Bot commented Jul 2, 2026

Copy link
Copy Markdown

PR Review: WebUI Terminal Support (#712)

Well-structured PR with good defense-in-depth (ticket auth, three-hop architecture, audit logging). Below are the issues worth addressing, ordered by severity.


Critical

1. Cubelet: waitIO() called after deleteProcess() in defer cleanup

Cubelet/services/cubebox/terminal.go:326-348

The defer chain runs: closeStdincloseProcessIOkillProcessdeleteProcesswaitIO. After deleteProcess() removes the containerd process object, processIO.Wait() may hang because IO resources are tied to the process lifecycle. On the context-cancellation path where the watcher goroutine hasn't called waitIO yet, this ordering causes waitIO() to block.

Fix: Reorder to: closeStdincloseProcessIOwaitIOkillProcessdeleteProcess.

2. Cubelet: Watcher goroutine can leak on context cancellation

Cubelet/services/cubebox/terminal.go:390-412

process.Wait(ctx) returns a channel that may never receive if the context is cancelled before the process exits. The goroutine blocks on <-statusC without a ctx.Done() select — if the process doesn't exit, neither close(done) nor the goroutine completes.

Fix: Add a select on both statusC and ctx.Done() in the watcher goroutine.


Important

3. CubeAPI: Missing Origin validation on WebSocket upgrade

CubeAPI/src/handlers/terminal.rs:148-164

The CubeAPI's WebSocketUpgrade handler does not validate the Origin header. CorsLayer::permissive() covers HTTP but not WebSocket upgrades. The CubeMaster side (terminalOriginAllowed) has proper validation — this RFC 6455 recommendation is unenforced at the CubeAPI hop.

Fix: Validate the Origin header against the Host header (or configured origin) before calling on_upgrade.

4. Cubelet: terminalMarkActive called before send in terminalOutputWriter.Write

Cubelet/services/cubebox/terminal.go:59-61

If terminalSend fails, lastActive has already been bumped, resetting the idle timer. A failing output stream masquerades as activity, keeping the connection alive.

Fix: Move terminalMarkActive after the successful send check.

5. Cubelet: Context-cancelled path returns nil without error frame

Cubelet/services/cubebox/terminal.go:470-473

When ctx.Done() fires in the main select loop, the function returns nil without sending an ERROR or EXIT frame. The browser sees only a connection close with no explanation. Compare with the idle-timeout path on line 519 which does send an error frame.

Fix: Send an error frame before returning.

6. CubeMaster: bridgeTerminalToWebsocket lacks context parameter

CubeMaster/pkg/service/httpservice/cube/terminal.go:347

bridgeWebsocketToTerminal receives and checks streamCtx on every iteration; bridgeTerminalToWebsocket does not. Under a network partition, gRPC Recv() may not respond to cancellation, causing the bridge to block indefinitely.

Fix: Pass streamCtx and add a ctx.Done() select.

7. CubeMaster: Redundant Close() + CloseSend()

CubeMaster/pkg/service/httpservice/cube/terminal.go:132-133

defer terminalStream.Close() and defer terminalStream.CloseSend() on the same bidi stream. For gRPC client streams, Close() typically internally calls CloseSend() — double-calling can cause undefined behavior.

Fix: Keep only defer terminalStream.CloseSend().

8. Cubelet: Initial resize error silently discarded

Cubelet/services/cubebox/terminal.go:368

If the initial PTY resize fails (process not started yet, terminal not set up), the error is swallowed with _ =. The client gets "connected" status with wrong terminal geometry.

Fix: Log the error at minimum; consider returning it before sending "connected".


Minor

9. Cubelet: Defensive copy in terminalOutputWriter.Write

Cubelet/services/cubebox/terminal.go:60cp := append([]byte(nil), data...)

The data buffer from containerd's FIFO reader is not retained after Write. The protobuf serialization marshals independently. This copy is a per-frame allocation with no aliasing risk. For busy terminals this adds meaningful GC pressure.

Fix: Pass data directly; remove the test assertion that validates the copy (terminal_test.go:83-86).

10. Test coverage gaps

Three critical components have zero tests:

  • proxy_terminal_ws (CubeAPI, ~168 lines) — bidirectional WebSocket proxy with 3 upstream failure paths
  • handleTerminalAction / bridge functions (CubeMaster) — goroutine orchestration and frame translation
  • service.Terminal (Cubelet, ~364 lines) — the most complex function in the PR

Adding unit tests for the bridge frame-translation functions and a lightweight end-to-end test across the pipeline would catch contract mismatches between the three layers.


Security positives

  • Tickets are UUIDv4, single-use (atomically removed via DashMap), tied to sandbox ID, 60-second TTL with a periodic pruner
  • Exec command is hardcoded /bin/sh — no user-controlled args
  • No secrets or credentials in code
  • CubeMaster origin validation correctly handles URL parsing edge cases
  • Session token validation falls through to Bearer/API key, not to silent grant
  • Audit logs record metadata only, not command contents

@linyuww linyuww closed this Jul 2, 2026
Assisted-by: Codex:GPT-5
@linyuww linyuww reopened this Jul 2, 2026
Comment thread CubeAPI/src/routes.rs
Comment thread CubeAPI/src/handlers/terminal.rs
Comment thread CubeAPI/src/handlers/terminal.rs
Comment thread CubeMaster/pkg/service/httpservice/cube/terminal.go Outdated
Comment thread CubeMaster/pkg/service/httpservice/cube/terminal.go Outdated
Comment thread Cubelet/services/cubebox/terminal.go
Comment thread web/src/components/TerminalDialog.tsx Outdated
Comment thread CubeMaster/pkg/service/httpservice/cube/terminal.go
Assisted-by: Codex:GPT-5
Signed-off-by: linyuww <2949203814@qq.com>
Comment thread CubeMaster/pkg/service/httpservice/cube/terminal.go Outdated
Comment thread CubeMaster/pkg/service/httpservice/cube/terminal.go
Comment thread Cubelet/services/cubebox/terminal.go Outdated
Comment thread Cubelet/services/cubebox/terminal.go
Comment thread CubeAPI/src/handlers/terminal.rs
Comment thread web/src/components/TerminalDialog.tsx Outdated
Comment thread CubeMaster/pkg/service/httpservice/cube/terminal.go
linyuww added 2 commits July 2, 2026 20:22
Related to TencentCloud#643

Assisted-by: Codex:GPT-5
Signed-off-by: linyuww <2949203814@qq.com>
Related to TencentCloud#643

Assisted-by: Codex:GPT-5
Signed-off-by: linyuww <2949203814@qq.com>
Comment thread CubeMaster/pkg/service/httpservice/cube/terminal.go
SandboxId: sandboxID,
ContainerId: containerID,
Terminal: true,
Args: []string{"/bin/sh"},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hardcoded /bin/sh: This always execs /bin/sh. Some minimal containers (Alpine → /bin/ash, distroless → no shell) may not have it at this path. Consider making the shell path configurable (e.g., via an API parameter or sandbox annotation) or detecting the available shell from the container image.

return frame.GetType() == cubeboxv1.TerminalFrame_INPUT && len(frame.GetData()) > 0
}

func terminalStreamClosedCleanly(err error) bool {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

codes.Canceled classified as clean close: This treats codes.Canceled as a clean stream closure. However, a network failure on the underlying gRPC transport can also surface as Canceled. The line at 339 (if !terminalStreamClosedCleanly(recvErr)) silently skips error reporting for these cases. Consider logging the error at debug level regardless of classification so transient transport issues remain observable.

The same pattern applies at line 389 where ctx.Done() returns nil on context cancellation — if the context was cancelled due to a network error rather than intentional close, the error is not propagated.

Comment thread web/src/components/TerminalDialog.tsx
linyuww added 2 commits July 3, 2026 13:25
Related to TencentCloud#643

Assisted-by: Codex:GPT-5
Signed-off-by: linyuww <2949203814@qq.com>
Related to TencentCloud#643

Assisted-by: Codex:GPT-5
Signed-off-by: linyuww <2949203814@qq.com>
Comment thread CubeMaster/pkg/service/httpservice/cube/terminal.go
Comment thread CubeAPI/src/handlers/terminal.rs Outdated
Comment thread CubeMaster/pkg/service/httpservice/cube/terminal.go
Comment thread Cubelet/services/cubebox/terminal.go
Comment thread CubeAPI/src/handlers/terminal.rs
Ok(ws
.max_message_size(TERMINAL_WS_MAX_MESSAGE_SIZE)
.on_upgrade(move |socket| proxy_terminal_ws(state, ticket, socket)))
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No Origin header validation on the browser-facing WebSocket endpoint. The CubeAPI WebSocket upgrade does not check the Origin header. While the one-time ticket mechanism mitigates CSRF, this eliminates defense-in-depth if the ticket mechanism were weakened. CorsLayer does not apply to WebSocket upgrades.

Suggestion: Add an Origin check similar to what CubeMaster does at terminal.go:44-54. Extract the Origin header from the request before the upgrade and verify it matches the expected host.

Comment thread CubeAPI/src/handlers/terminal.rs Outdated
linyuww added 3 commits July 3, 2026 19:08
Avoid adopting the CubeAPI browser WebSocket Origin check from the latest bot review. The current WebUI reverse proxy forwards Host without the external port, so matching Origin to Host rejects valid dashboard terminal connections with 401.

Keep the other low-risk review fixes: rate limit terminal WebSocket upgrades, bound CubeAPI terminal WebSocket frame size, avoid inline ticket pruning, track the CubeMaster deadline goroutine, and pin Cubelet delete retry fallback strings.

Related to TencentCloud#643

Assisted-by: Codex:GPT-5
  Terminal dialog can now be dragged by its title bar and resized
  by dragging the bottom-right corner handle. xterm FitAddon
  re-fits after each resize. Includes clampWithinViewport helpers
  and interactive-drag-target guard to prevent interference with
  buttons/inputs.

  Related to TencentCloud#643

Signed-off-by: linyuww <2949203814@qq.com>
Deleting a paused sandbox should not resume the VM from the pause snapshot. The resume path can fail when restoring the virtio-net tap with Device or resource busy, leaving the sandbox undeletable.

Allow only the main sandbox task delete while paused, publish TaskDelete, notify container shutdown, and signal shim exit. Exec deletes remain rejected in paused state.

Related to TencentCloud#643

Assisted-by: Codex:GPT-5
Signed-off-by: linyuww <2949203814@qq.com>
@linyuww

linyuww commented Jul 3, 2026

Copy link
Copy Markdown
Author
result2

@linyuww linyuww marked this pull request as ready for review July 3, 2026 14:19
@linyuww linyuww requested a review from tinklone as a code owner July 3, 2026 14:19
Copilot AI review requested due to automatic review settings July 3, 2026 14:19

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds end-to-end Web Terminal support to CubeSandbox, spanning CubeAPI ticketing + WebSocket proxying, CubeMaster/Cubelet terminal streaming, CubeShim resize forwarding, and a WebUI xterm-based terminal dialog with docs and tests.

Changes:

  • Adds terminal session ticket + WebSocket endpoints in CubeAPI and wires OpenAPI/schema/client updates.
  • Implements CubeMaster ↔ Cubelet bidirectional terminal streaming (TTY, resize, keepalive, lifecycle) over WebSocket + gRPC.
  • Adds WebUI “Open Terminal” entry and xterm dialog (container selection, reconnect/status, i18n) plus frontend tests/docs.

Reviewed changes

Copilot reviewed 42 out of 49 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
web/vitest.config.ts Adds Vitest config (jsdom, aliases, setup file).
web/vite.config.ts Enables WebSocket proxying for /cubeapi in dev server.
web/src/test/setup.ts Adds test setup (ResizeObserver shim).
web/src/pages/SandboxDetail.tsx Adds terminal entry button + dialog wiring; pause disabled during active terminal.
web/src/pages/SandboxDetail.test.tsx Tests terminal entry enable/disable and pause gating behavior.
web/src/locales/zh/sandboxes.json Adds terminal/pause conflict localized error message (zh).
web/src/locales/zh/sandboxDetail.json Adds terminal UI strings + error mapping (zh).
web/src/locales/en/sandboxes.json Adds terminal/pause conflict localized error message (en).
web/src/locales/en/sandboxDetail.json Adds terminal UI strings + error mapping (en).
web/src/lib/sandboxActionError.ts Maps pause conflict messages to a user-facing “close terminal” hint.
web/src/lib/sandboxActionError.test.ts Adds tests for the new pause-conflict error mapping.
web/src/components/TerminalDialog.tsx Implements xterm-based terminal dialog (WS connect, resize, ping, drag/resize UI).
web/src/components/TerminalDialog.test.tsx Adds tests for terminal session creation, I/O bridging, protocol errors, drag/resize.
web/src/api/generated/schema.ts Updates generated OpenAPI types for terminal + container metadata.
web/src/api/client.ts Adds terminal session API client and container DTOs on sandbox detail.
web/package.json Adds Vitest + Testing Library deps; adds npm run test; adds xterm deps.
openapi.yml Adds terminal session + WS routes and terminal-related schemas.
docs/zh/guide/webui.md Documents how to use WebUI terminal (zh) including limitations and security notes.
docs/guide/webui.md Documents how to use WebUI terminal (en) including limitations and security notes.
dev-env/internal/sync_web_to_vm.sh Adds SANDBOX proxy upstream env substitution to the VM sync script.
CubeShim/shim/src/service/task_srv.rs Allows paused delete for sandbox task; forwards resize pty to sandbox/container layer.
CubeShim/shim/src/sandbox/sb.rs Adds paused-container delete path; adds exec PTY resize forwarding to container.
CubeShim/shim/src/container/mod.rs Implements agent call to resize PTY for an exec session.
CubeMaster/pkg/service/httpservice/cube/terminal.go Adds CubeMaster WebSocket terminal endpoint and gRPC bridge to Cubelet.
CubeMaster/pkg/service/httpservice/cube/terminal_test.go Adds unit tests for origin check, parsing, selection, and bridge wait helpers.
CubeMaster/pkg/service/httpservice/cube/cube.go Registers the new /sandbox/terminal action.
CubeMaster/pkg/server/server.go Exposes the new terminal route in the HTTP router.
CubeMaster/pkg/server/server_test.go Tests that the terminal route is registered.
CubeMaster/pkg/cubelet/actions.go Adds a TerminalStream helper and gRPC stream open/timeout behavior.
CubeMaster/api/services/cubebox/v1/cubebox.proto Adds gRPC bidi Terminal RPC and TerminalFrame message.
CubeMaster/api/services/cubebox/v1/cubebox_grpc.pb.go Regenerates gRPC bindings for the new Terminal RPC.
Cubelet/services/cubebox/terminal.go Implements containerd exec-backed interactive terminal stream with idle timeout & cleanup.
Cubelet/services/cubebox/terminal_test.go Adds unit tests for terminal helpers and retry/idle behaviors.
Cubelet/api/services/cubebox/v1/cubebox.proto Adds Terminal RPC and TerminalFrame to Cubelet proto.
Cubelet/api/services/cubebox/v1/cubebox_grpc.pb.go Regenerates Cubelet gRPC bindings for Terminal.
CubeAPI/src/state.rs Adds in-memory one-time terminal ticket store + periodic pruning.
CubeAPI/src/services/sandboxes.rs Surfaces container metadata on sandbox detail for terminal selection.
CubeAPI/src/routes.rs Adds terminal session + WS routes with rate limiting.
CubeAPI/src/openapi.rs Registers terminal endpoints and schemas in API docs.
CubeAPI/src/models/mod.rs Adds terminal request/response models and sandbox container model.
CubeAPI/src/handlers/terminal.rs Implements terminal ticket issuance, WS proxy to CubeMaster, and auth/operator extraction.
CubeAPI/src/handlers/mod.rs Exposes the new terminal handler module.
CubeAPI/src/cubemaster/mod.rs Parses container metadata from CubeMaster response into CubeAPI sandbox detail.
CubeAPI/Cargo.toml Adds tokio-tungstenite dependency for WebSocket proxying.
CubeAPI/Cargo.lock Locks tokio-tungstenite dependency.
agent/rustjail/src/container.rs Fixes controlling terminal ioctl to check return value and report errors.
Files not reviewed (5)
  • CubeMaster/api/services/cubebox/v1/cubebox.pb.go: Generated file
  • CubeMaster/api/services/cubebox/v1/cubebox_grpc.pb.go: Generated file
  • Cubelet/api/services/cubebox/v1/cubebox.pb.go: Generated file
  • Cubelet/api/services/cubebox/v1/cubebox_grpc.pb.go: Generated file
  • web/package-lock.json: Generated file

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread web/src/components/TerminalDialog.tsx
Comment thread web/src/components/TerminalDialog.tsx
Comment thread web/src/components/TerminalDialog.tsx Outdated
Comment thread web/src/components/TerminalDialog.tsx
Comment thread web/src/pages/SandboxDetail.tsx
Handle clipboard permission failures, add accessible labels for terminal icon controls, and disable the
    terminal entry when container metadata has no running container.

Related to TencentCloud#643

Assisted-by: Codex:GPT-5
Signed-off-by: linyuww <2949203814@qq.com>
Comment thread Cubelet/services/cubebox/terminal.go Outdated
Comment thread Cubelet/services/cubebox/terminal.go
Comment thread Cubelet/services/cubebox/terminal.go Outdated
Comment on lines +132 to +133
defer terminalStream.Close()
defer terminalStream.CloseSend()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundant Close/CloseSend: On a gRPC bidi client stream, Close() typically calls CloseSend() internally. Deferring both can produce undefined behavior.

Fix: Remove defer terminalStream.Close() and keep only CloseSend().

Comment thread CubeAPI/src/handlers/terminal.rs
Comment thread Cubelet/services/cubebox/terminal.go Outdated
Signed-off-by: linyuww <2949203814@qq.com>
@fslongjin

Copy link
Copy Markdown
Member

Refs #643

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants