Skip to content

Latest commit

 

History

History
667 lines (579 loc) · 20.7 KB

File metadata and controls

667 lines (579 loc) · 20.7 KB

Configuration Reference

This document lists all configuration options, defaults, environment overrides, and guidance. See docs/architecture.md for the end‑to‑end flow.

Where the CLI reads config from:

  1. CLI flags (e.g., --concurrency, --results-dir, --enable-service-logs). See --help for each command for more details.
  2. Environment variables (prefix TUSK_)
  3. Config file (auto-discovered): .tusk/config.yaml, .tusk/config.yml, tusk.yaml, tusk.yml, or ~/.tusk/config.yaml

✨ Run tusk drift setup in your service root directory to start an agent automatically create a config file based on your service.

Service

Key Type Default Required Description
service.id string Cloud: yes Tusk Drift Cloud service identifier. Required in --cloud mode.
service.name string no Optional display name.
service.port number 3000 no Port where your service listens. CLI will not continue if occupied.
service.start.command string yes Shell command to start your service. Executed via /bin/sh -c. e.g., npm run start.
service.stop.command string no Shell command to stop your service. If omitted, CLI uses process group termination (SIGTERM/SIGKILL). Useful for Docker: docker compose down.
service.communication.type string auto no Communication method between CLI and SDK: auto (detects Docker), unix (Unix socket), or tcp (TCP socket). Auto-detects tcp when start command contains "docker".
service.communication.tcp_port number 9001 no Port for CLI's mock server when using TCP communication (Docker mode). This is separate from service.port.
service.readiness_check.command string no Polling command until it exits 0. If omitted, CLI waits ~10s. Highly recommended if your service has a health check endpoint.
service.readiness_check.timeout duration 10s (effective) no Total time to wait for readiness. Examples: 30s, 2m.
service.readiness_check.interval duration 2s no Poll interval for the readiness command.

Runtime environment variables set by the CLI for your service:

  • TUSK_MOCK_SOCKET: Unix socket path (non-Docker mode)
  • TUSK_MOCK_HOST: Mock server host for TCP mode (Docker)
  • TUSK_MOCK_PORT: Mock server port for TCP mode (Docker)
  • TUSK_DRIFT_MODE=REPLAY: Signals the SDK to run in replay mode
Internal (optional) CLI behavior environment variables:

For internal use - users generally do not need to set these.

  • TUSK_REQUIRE_INBOUND_REPLAY_SPAN=1: Fails a replay test when the SDK does not send the inbound replay span to the CLI. Useful for internal CI/e2e hardening to catch silent telemetry regressions. Truthy values include 1, true, yes, and on.

Docker Support

When using Docker or Docker Compose, the CLI automatically detects Docker commands and switches to TCP communication.

Requirements for Docker

For Docker setups, additional steps are required for the SDK in your containerized app to communicate with the CLI. For Linux environments, you will need to add the add-host flag (if using docker run) or the extra_hosts parameter (if using Docker Compose) to resolve host.docker.internal to the IP address of the host machine.

You should also add a service.stop.command in your config so Tusk knows how to stop the container when tests have complete. See Service parameters above.

See: Docker example.

Docker Compose

While Tusk replays traffic in your app as a standalone service, we do support Docker Compose setups as well.

Create a docker-compose.tusk-override.yml file.

Important: the override filename must be exactly docker-compose.tusk-override.yml

services:
  api:
    environment:
      TUSK_DRIFT_MODE: ${TUSK_DRIFT_MODE:-REPLAY}
      TUSK_MOCK_HOST: ${TUSK_MOCK_HOST:-host.docker.internal}
      TUSK_MOCK_PORT: ${TUSK_MOCK_PORT:-9001}

    # Required on Linux (including CI). No-op on macOS Docker Desktop.
    extra_hosts:
      - "host.docker.internal:host-gateway"

Then, your start command will be something like:

docker compose -f docker-compose.yml -f docker-compose.tusk-override.yml up

If you're recording your service started with Docker Compose, we recommend setting the TUSK_DRIFT_MODE to RECORD as an env parameter in your Docker Compose file (not in the command in your start parameter), so that this can be properly overridden during replay.

See: Docker Compose example.

How it works

  • Your app's port (service.port): Where your API listens for HTTP requests
  • CLI's mock server port (service.communication.tcp_port): Where the SDK connects to get mocks
  • The CLI auto-detects Docker from the start command and uses TCP instead of Unix sockets
  • Your service port is exposed to the host via Docker port mapping (normal -p flag)
  • The SDK inside the container reaches the CLI on the host via host.docker.internal

Traces (local)

Key Type Default Required Env override Description
traces.dir string .tusk/traces no TUSK_TRACES_DIR Directory to load local recorded traces when not in cloud mode. CLI flag --trace-dir overrides. The CLI searches this directory first; if not found, it falls back to traces/, tmp/, and ..

In local recording mode, the SDK will also save trace files to this directory.

Tusk API (Cloud mode)

Key Type Default Required Env override Description
tusk_api.url string Cloud: yes TUSK_API_URL Base URL of Tusk Drift Cloud. The CLI targets /api/drift/test_run_service under this host. This defaults to https://api.usetusk.ai. You generally don't need to override this.

For authentication in cloud mode, either use:

  • Auth0: tusk auth login
  • API key: TUSK_API_KEY

Test execution

Key Type Default Required Notes
test_execution.concurrency number 1 no Max concurrent tests. CLI flag --concurrency overrides. For Node.js applications with CPU-intensive synchronous operations (such as synchronous JWT sign/verify), concurrency of 1 is recommended to avoid test interference, since these operations block the single-threaded event loop and can cause concurrent requests to timeout.
test_execution.timeout duration 30s no Timeout for each trace test (a test usually completes in <1 second).

Comparison (response diffing)

Key Type Default Description
comparison.ignore_fields string[] [] Exact field names (by last path segment) to ignore during JSON comparison.
comparison.ignore_patterns string[] [] Regex patterns for values to ignore when both sides match.
comparison.ignore_uuids boolean true Ignore UUID‑like values when both sides are UUIDs.
comparison.ignore_timestamps boolean true Ignore ISO‑8601 timestamps when both sides are timestamps.
comparison.ignore_dates boolean true Ignore date formats (e.g., YYYY-MM-DD) when both sides are dates.
comparison.ignore_jwt_fields boolean true When both expected and actual values are JWT tokens, decode their payloads and compare claims individually (ignoring the jti claim and applying other dynamic field rules to each claim). This handles tokens that differ only in per‑issuance fields like jti and their resulting signature.

Recording (for SDK)

Key Type Default Description
recording.sampling.mode string fixed Sampling strategy for Drift SDK recording. Supported values: fixed and adaptive.
recording.sampling.base_rate number 0.1 Base sampling fraction when recording traces. In fixed mode this is the effective rate. In adaptive mode the SDK may temporarily reduce below this base rate under pressure.
recording.sampling.min_rate number 0.001 in adaptive mode; unset in fixed mode Lower bound for adaptive sampling after load shedding is applied. This is only defaulted in adaptive mode and remains unset in fixed mode.
recording.sampling.log_transitions boolean true Whether the SDK should emit adaptive sampling transition logs. This is SDK-facing only and can be overridden by TUSK_RECORDING_SAMPLING_LOG_TRANSITIONS.
recording.sampling_rate number 0.1 Deprecated. Legacy alias for recording.sampling.base_rate. Still accepted for backwards compatibility.
recording.export_spans boolean false Whether to export spans to the Tusk Drift Cloud. If false, spans are saved locally in traces.dir
recording.export_env_var_recording boolean false Whether to record and replay environment variables. Recommended for accurate replay behavior if your service's logic depends on environment variables.

This will not affect CLI behavior. See SDK for more details:

Replay

Key Type Default Description
replay.sandbox.mode string unset Replay sandbox strategy: if unset, replay defaults to strict on platforms that support Fence and otherwise runs without sandboxing after a warning. auto starts sandboxed and retries once without sandbox if startup fails, strict requires sandboxing and fails if sandbox cannot be initialized/applied, and off never uses sandboxing.
replay.sandbox.config_path string Optional path to a Fence config file to merge into the built-in replay sandbox. Relative paths are resolved from the repo/service root containing .tusk. Replay-required settings are still enforced after merge. If that Fence config uses extends, those relative paths are resolved relative to the config file itself.

Results

Key Type Default Env override Description
results.dir string .tusk/results TUSK_RESULTS_DIR Directory for saved run outputs when --save-results is used. CLI flag --results-dir takes precedence.

Coverage

Configuration for code coverage collection. See docs/drift/coverage.md for full documentation.

Key Type Default Description
coverage.enabled bool false When true, automatically collect coverage during suite validation runs on the default branch. No CI changes needed.
coverage.include string[] (all files) Only include files matching at least one pattern. Supports ** for recursive matching. Paths are git-relative.
coverage.exclude string[] (none) Exclude files matching any pattern. Applied after include. Supports ** for recursive matching.
coverage.strip_path_prefix string (none) Strip this prefix from coverage file paths. Required for Docker Compose — set to the container mount point (e.g., /app).

Config overrides

Flags that override config

  • --concurrency → overrides test_execution.concurrency
  • --enable-service-logs → enables service log capture (not a config key)
  • --save-results and --results-dir → control result file output (uses results.dir if not provided)
  • --trace-dir → overrides traces.dir
  • --sandbox-mode → overrides replay.sandbox.mode
  • --sandbox-config → overrides replay.sandbox.config_path
  • --cloud and metadata flags (e.g., --trace-test-id, --all-cloud-trace-tests, CI context flags)
  • --agent → writes per-test deviation Markdown files to .tusk/logs/ for coding agent consumption (not a config key)
  • --agent-output-dir → overrides the base output directory for --agent (default: .tusk/logs/)

Environment variables that override config

  • TUSK_TRACES_DIRtraces.dir
  • TUSK_API_URLtusk_api.url
  • TUSK_RESULTS_DIRresults.dir
  • TUSK_RECORDING_SAMPLING_RATErecording.sampling_rate
  • TUSK_RECORDING_SAMPLING_LOG_TRANSITIONSrecording.sampling.log_transitions

Minimal config examples

Local example

service:
  name: my-service
  port: 3000
  start:
    command: npm run dev
  readiness_check:
    command: curl -sf http://localhost:3000/health
    timeout: 30s
    interval: 2s

# Parameters below are optional
traces:
  dir: .tusk/traces

test_execution:
  concurrency: 5

comparison:
  ignore_fields: ["request_id"]
  ignore_uuids: false

results:
  dir: .tusk/results

replay:
  sandbox:
    mode: auto
    # config_path: .tusk/replay.fence.json

Starting your service with docker run

service:
  name: my-service
  port: 9000
  start:
    command: |
      docker run -d \
        --name my-app \
        --add-host=host.docker.internal:host-gateway \
        -p 9000:9000 \
        -e TUSK_MOCK_HOST=host.docker.internal \
        -e TUSK_MOCK_PORT=9001 \
        -e TUSK_DRIFT_MODE=REPLAY \
        my-app-image:latest
  stop:
    command: docker stop my-app && docker rm my-app
  readiness_check:
    command: curl http://localhost:9000/health
    timeout: 45s
    interval: 5s

# Communication auto-detects TCP from "docker" in start command
# Optionally configure explicitly:
# communication:
#   type: tcp
#   tcp_port: 9001

traces:
  dir: .tusk/traces

--add-host is required when running on a Linux machine and is redundant for Mac/Windows.

Starting your service with Docker Compose

service:
  name: my-service
  port: 9000
  start:
    command: docker compose -f docker-compose.yml -f docker-compose.tusk-override.yml up
  stop:
    command: docker compose down
  readiness_check:
    command: curl http://localhost:9000/health
    timeout: 45s
    interval: 5s

# Communication auto-detects TCP from "docker" in start command
# Optionally configure explicitly:
# communication:
#   type: tcp
#   tcp_port: 9001

traces:
  dir: .tusk/traces

As a reminder, you need to create and pass in an override file in your start command (see Docker Support section above). Use the exact filename docker-compose.tusk-override.yml.

Cloud example

service:
  id: 1165f64c-5a5e-4586-a22a-2d7cab42af83
  name: my-service
  port: 9000
  start:
    command: npm run dev
  readiness_check:
    command: curl -sf http://localhost:3000/health
    timeout: 30s
    interval: 2s

To run against traces to Tusk Drift Cloud, your config file must contain service.id and tusk_api.url.

Windows Support

On Windows, Unix sockets are not supported, so you must use TCP communication mode.

Requirements for Windows

  1. Set service.communication.type to tcp in your config
  2. Override TUSK_MOCK_HOST to localhost in your start command (the CLI defaults to host.docker.internal which only works inside Docker)

Example Windows config

service:
  name: my-service
  port: 3000
  start:
    # Note: set TUSK_MOCK_HOST with no space before &&
    command: set TUSK_MOCK_HOST=localhost&& npm run start
  readiness_check:
    # Use curl.exe (not curl) on Windows, or use PowerShell
    command: curl.exe -fsS http://localhost:3000/health
    timeout: 30s
    interval: 1s
  communication:
    type: tcp
    tcp_port: 9001

traces:
  dir: .tusk/traces

Key differences from macOS/Linux

Aspect macOS/Linux Windows
Communication Unix socket (default) TCP (required)
Mock host N/A Must override to localhost
Shell /bin/sh -c cmd.exe /c
curl curl curl.exe

If curl.exe is not available, use PowerShell:

readiness_check:
  command: powershell -Command "try { Invoke-WebRequest -Uri http://localhost:3000/health -UseBasicParsing; exit 0 } catch { exit 1 }"