feat(remote): support texture/RT/depth export in remote replay mode

openspec/changes/2026-06-09-remote-texture-export/proposal.md

@@ -0,0 +1,115 @@
+# Remote texture export via GetTextureData + local decode
+
+## Problem
+
+Issue #236: `tex_export`, `rt_export`, and `rt_depth` all fail in remote mode.
+
+The three export handlers unconditionally call `controller.SaveTexture(ts, path)`, which
+writes a file on the **remote** daemon host's filesystem.  The client receives a path that
+does not exist locally and cannot fetch it.  In the prior codebase all three handlers were
+pre-emptively blocked by the `is_remote` guard (introduced alongside remote replay), so
+callers received a JSON-RPC error before any file I/O is attempted.  The result is that
+remote sessions have no supported path to export rendered targets — a major capability gap
+for headless/cloud workflows where the daemon runs on a GPU server.
+
+## Root Cause
+
+`SaveTexture` is a RenderDoc API that writes to the path it is given, which in a remote
+session is a path on the remote host.  There is no `SaveTexture`-over-network mode in the
+RenderDoc Python API.  The `rt_overlay` handler is blocked for a related but distinct
+reason: rendering overlays (Overdraw, Wireframe, etc.) requires GPU draw submission, which
+is not safe over the remote JSON-RPC channel, so that guard is correct and must stay.
+
+## Solution
+
+Replace the `SaveTexture` call with `controller.GetTextureData(resource_id, sub)`, which
+returns the raw texel bytes over JSON-RPC to the **client** process.  The client decodes the
+bytes locally using numpy and Pillow (both already runtime dependencies) and writes the
+result as a PNG at the caller-supplied path.
+
+Two private helpers live in `handlers/_helpers.py`:
+
+- `_decode_dtype(rd, comp_type, comp_byte_width) -> str | None` — a pure lookup table that
+  maps a `(CompType, compByteWidth)` pair to a numpy dtype name, or `None` to reject pairs
+  with no unambiguous 8-bit display mapping.
+- `_decode_texture_png(rd, tex, raw, mip, *, is_depth) -> bytes | None` — decodes the tightly
+  packed bytes into PNG bytes, or returns `None` when the format cannot be displayed.  It
+  does not touch the filesystem; the caller writes the returned bytes.
+
+A small handler-side helper `_export_remote(...)` in `handlers/texture.py` wraps the wire
+fetch: it calls `GetTextureData`, guards against empty/`None` data, calls
+`_decode_texture_png`, and writes the PNG.  Each export handler routes through it under
+`if state.is_remote:`; the existing `SaveTexture` branch is left untouched for local mode.
+
+### Decode behavior
+
+- Only `ResourceFormatType.Regular` formats are decoded; every non-Regular format returns a
+  clear `-32002` error `"format <Name> not supported for remote decode"`.
+- Mip-level dimensions: `w = max(1, tex.width >> mip)`, `h = max(1, tex.height >> mip)`.
+- Byte layout from `GetTextureData` is tightly packed, top-down (no row padding, no Y-flip).
+- Dtype and 8-bit conversion are selected per `(compType, compByteWidth)` via `_decode_dtype`:
+  - Float16 / Float32 (HDR linear) → tonemap: `clip(x, 0, 1)`, sRGB OETF, `*255`, uint8.
+    Documented as contrast-clamped, not a raw dump.
+  - UNorm8 / UNormSRGB8 → uint8 pass-through (sRGB bytes are display-ready, no extra gamma).
+  - UNorm16 → uint16, scaled to 8-bit via divide-by-257.
+  - SNorm8 / SNorm16 → read as the signed integer width, remapped `[-1, 1] -> [0, 255]`
+    using the signed-int max as the divisor (normal-map friendly).
+  - UInt8 / UInt16 → treated as display values (UInt16 also `/257`).
+  - Depth8 / Depth16 / Depth32 → handled by the depth path (below).
+- `BGRAOrder() == True` with `cc >= 3` → swap to RGBA channel order before save.
+- Channel expansion for the COLOR path: cc=1 → repeat the single channel to RGB and add
+  A=255 (full RGBA, **not** mode `L`); cc=2 → RG0 + A=255; cc=3 → add A=255; cc=4 → RGBA
+  as-is.  Output is always RGBA.
+- Depth path (`is_depth=True`, used only by `rt_depth`): take channel 0, auto-contrast via
+  `(d - d_min) / (d_max - d_min)`, then `*255` uint8, written as mode `L`.  This matches
+  RenderDoc's contrast-stretched depth display; it is not a raw depth dump — use `tex_raw`
+  for exact bytes.
+
+### Cleanly rejected (→ `-32002`, never a garbled image)
+
+- `_decode_dtype` returns `None` for: Typeless, SInt, UScaled, SScaled, and any exotic
+  component width not in the table.
+- Non-`Regular` `ResourceFormatType`: block-compressed (BC1-7, ASTC), packed
+  (R11G11B10, R10G10B10A2, R9G9B9E5, R5G6B5), and combined depth-stencil (D24S8, D32S8).
+- MSAA (`msSamp > 1`).
+- Length mismatch: `len(raw) != h * w * compCount * compByteWidth`.
+- Empty / `None` data: `_decode_texture_png` returns `None` on empty input, and
+  `_export_remote` additionally guards the `GetTextureData` return so a `None`/empty result
+  yields `"no texture data returned"` rather than a `len(None)` crash.
+
+### What stays blocked
+
+`rt_overlay` remote guard is unchanged: overlay rendering requires GPU submission and cannot
+be made safe over remote JSON-RPC.
+
+### Known limitation / follow-up
+
+Local mode (`SaveTexture`) renders depth with a RED colormap, while the remote path renders
+depth as GRAY (mode `L`).  This local(RED)/remote(GRAY) depth-visualization inconsistency is
+an accepted limitation for this change; aligning the two colormaps is deferred to a
+follow-up.
+
+### Scope
+
+- In scope: `tex_export`, `rt_export`, `rt_depth` in remote mode; 3D textures (`depth > 1`)
+  are exported by tiling depth slices vertically into one image.
+- Out of scope: block-compressed formats; packed formats (R11G11B10, R10G10B10A2, R9G9B9E5,
+  R5G6B5, D24S8, D32S8); MSAA (`msSamp > 1`); local mode (unchanged byte-for-byte).
+
+## Risks
+
+| Risk | Mitigation |
+|------|------------|
+| Double sRGB correction | UNormSRGB bytes pass through uint8 unchanged; only linear Float formats get the sRGB OETF. |
+| BGRA channel swap missed | Gate on `BGRAOrder()` (with `cc >= 3`) and reorder to RGBA before `Image.fromarray`. |
+| Signed data read as unsigned | SNorm is read at the signed integer width and remapped `[-1,1] -> [0,255]`; SInt is rejected outright. |
+| Row-padding assumption | Defensive `len(raw) != expected` check; bail to clear `-32002` rather than corrupt reshape. |
+| Y-flip added by mistake | `GetTextureData` is top-down; Pillow `fromarray` is top-down — do NOT flip. |
+| MSAA ambiguity | Reject `msSamp > 1` (mirror `tex_stats` guard) in remote decode path. |
+| Mip dimension off-by-one | Use `max(1, dim >> mip)` per level; `tex.width`/`tex.height` are mip-0 only. |
+| Packed/block format garbage | Guard `tex.format.type == ResourceFormatType.Regular`; emit `-32002` for the rest. |
+| Empty/None texture data crash | `_export_remote` guards the `GetTextureData` return before decode; empty → `-32002`. |
+| rt_export/rt_depth lack TextureDescription | Resolve `Descriptor.resource` → `state.tex_map[int(resource)]` to obtain width/height/format. |
+| Local-mode regression | All new decode logic is strictly under `if state.is_remote:`; existing `SaveTexture` branch is byte-for-byte unchanged. |
+| Depth normalization is relative | Document that remote depth PNG is contrast-stretched for visibility; raw bytes available via `tex_raw`. |
+| Local/remote depth colormap diverge | Accepted limitation (local RED vs remote GRAY); colormap alignment deferred to follow-up. |

openspec/changes/2026-06-09-remote-texture-export/tasks.md

@@ -0,0 +1,68 @@
+# Tasks: remote texture export via GetTextureData
+
+## Phase A: Mock infrastructure
+
+- [x] `tests/mocks/mock_renderdoc.py`: `ResourceFormat` supports a non-Regular `type` and a
+  `Name()` so packed/block/typeless tests can be expressed
+- [x] `tests/mocks/mock_renderdoc.py`: `ResourceFormat.BGRAOrder()` plus
+  `MockReplayController._texture_data` / `GetTextureData`, `MockPipeState`, and `Descriptor`
+  for the remote export paths; existing handler tests still pass
+
+## Phase B: Decode helpers (TDD)
+
+- [x] Write the remote-decode unit tests in `tests/unit/test_tex_stats_handler.py` before
+  implementing the helpers
+- [x] Implement `_decode_dtype(rd, comp_type, comp_byte_width) -> str | None` in
+  `src/rdc/handlers/_helpers.py`: a `(CompType, compByteWidth)` -> numpy-dtype table
+  covering Float16/32, UNorm8/16, UNormSRGB8, SNorm8/16, UInt8/16, Depth8/16/32; returns
+  `None` (reject) for Typeless / SInt / UScaled / SScaled / exotic widths
+- [x] Implement `_decode_texture_png(rd, tex, raw, mip, *, is_depth) -> bytes | None` in
+  `src/rdc/handlers/_helpers.py`:
+  - Empty `raw` -> `None`
+  - Guard: `tex.format.type != rd.ResourceFormatType.Regular` -> `None`
+  - Guard: `tex.msSamp > 1` -> `None`
+  - Mip dimension: `w = max(1, tex.width >> mip)`, `h = max(1, tex.height >> mip)`
+  - Defensive length check: `len(raw) != h * w * compCount * compByteWidth` -> `None`
+  - Dtype/scale by `(compType, compByteWidth)`: Float `clip+sRGB-OETF`, UNorm8/UNormSRGB8/
+    UInt8 pass-through, UNorm16/UInt16 `/257`, SNorm signed-remap `[-1,1]->[0,255]`
+  - `BGRAOrder()` (with `cc >= 3`) channel swap to RGBA
+  - Color channel expand: cc=1 repeat->RGB + A=255, cc=2 RG0+A, cc=3 +A; always RGBA output
+  - Depth path (`is_depth=True`): channel-0 auto-contrast -> mode `L`
+  - Return PNG bytes
+- [x] Verify color/reject decode cases pass
+
+## Phase C: tex_export remote branch
+
+- [x] `src/rdc/handlers/texture.py`: add `_export_remote(...)` helper that calls
+  `GetTextureData`, guards empty/`None` data (`"no texture data returned"`), calls
+  `_decode_texture_png`, emits `-32002` `"format <Name> not supported for remote decode"`
+  on `None`, and writes the PNG returning `{"path", "size"}`
+- [x] `_handle_tex_export`: route through `_export_remote` under `if state.is_remote:`
+- [x] Keep the existing `SaveTexture` branch unchanged for local mode
+- [x] Remove the old remote-rejected guard for `tex_export`
+- [x] Verify `tex_export` remote + local-regression cases pass
+
+## Phase D: rt_export and rt_depth remote branches
+
+- [x] `_handle_rt_export` remote branch: resolve `Descriptor.resource` ->
+  `state.tex_map[int(resource)]`; route through `_export_remote`; error if resource not in
+  `tex_map`
+- [x] `_handle_rt_depth` remote branch: same pattern, passing `is_depth=True` so depth ->
+  mode `L` PNG
+- [x] Keep both `SaveTexture` local branches unchanged
+- [x] Remove the old remote-rejected guards for `rt_export` and `rt_depth`
+- [x] Verify `rt_export` / `rt_depth` remote cases pass
+
+## Phase E: Regression + rt_overlay guard
+
+- [x] Confirm `rt_overlay` remote guard is intact (`test_rt_overlay_remote_still_rejected`)
+- [x] Replace the old `test_*_remote_rejected` tests with cases asserting a PNG is produced
+- [x] Run full unit suite — all green
+- [x] Run `pixi run check` (lint + typecheck + tests)
+
+## Phase F: Code review + merge
+
+- [ ] Code review
+- [ ] Open PR targeting `master`
+- [ ] Manual GPU proxy check (see test-plan manual section) after PR is up
+- [ ] Archive this OpenSpec folder after merge

openspec/changes/2026-06-09-remote-texture-export/test-plan.md

@@ -0,0 +1,115 @@
+# Test Plan: remote texture export via GetTextureData
+
+## Scope
+
+### In scope
+- `_decode_dtype` table + `_decode_texture_png` helper: format coverage, channel fixups,
+  signed remap, length guard, empty-data guard
+- `tex_export` in remote mode: PNG written locally
+- `rt_export` in remote mode: PNG written locally
+- `rt_depth` in remote mode: grayscale PNG written locally
+- Regression: local (non-remote) mode still routes through `SaveTexture` unchanged
+- Regression: `rt_overlay` still returns an error in remote mode
+- `mock_renderdoc.py` extensions required for the above
+
+### Out of scope
+- `rt_overlay` correctness (guard is unchanged)
+- Block-compressed / packed formats (returns -32002; negative tests cover the class)
+- MSAA textures (guarded; one negative test)
+- Live GPU proxy (manual only; see below)
+
+## Test Matrix
+
+| Layer | Type | File |
+|-------|------|------|
+| Unit | `_decode_texture_png` + export handlers (mock) | `tests/unit/test_tex_stats_handler.py` |
+| Unit | mock infrastructure | `tests/mocks/mock_renderdoc.py` |
+
+## Mock infrastructure (prerequisite)
+
+`tests/mocks/mock_renderdoc.py` provides, for the remote-decode tests:
+
+- `ResourceFormat(name, compByteWidth, compCount, compType, type=...)` with `BGRAOrder()`
+  and `Name()`; `type` defaults to Regular and is set non-Regular for packed/block tests.
+- `MockReplayController._texture_data[rid]` pre-populated by each test with the raw bytes;
+  `GetTextureData(rid, sub)` returns it (and can return `b""` to exercise the empty guard).
+- `MockPipeState(output_targets=..., depth_target=...)` and `Descriptor(resource=...)` for
+  the `rt_export` / `rt_depth` paths.
+
+## Unit test cases
+
+All remote cases run with `is_remote=True`; no real GPU, no `SaveTexture` call.
+Test names are the actual functions present in `tests/unit/test_tex_stats_handler.py`.
+
+### Color decode — accepted formats
+
+1. `test_tex_export_remote_rgba8` — R8G8B8A8_UNORM 4x2: valid PNG, size (4,2), mode RGBA.
+2. `test_tex_export_remote_bgra_swaps_channels` — B8G8R8A8_UNORM 1x1: input B,G,R,A bytes
+   decode to RGBA pixel with R/B swapped.
+3. `test_tex_export_remote_r8_grayscale` — R8_UNORM 2x2 (cc=1): single channel repeated to
+   RGB (pixel `(10,10,10)`), output is RGBA.
+4. `test_tex_export_remote_float16_hdr` — R16G16B16A16_FLOAT 1x1: value 2.0 clipped + sRGB
+   encoded to 255; alpha 255.
+5. `test_tex_export_remote_r16_unorm_scales_by_257` — R16_UNORM: 65535/257=255, 32896/257=128.
+6. `test_tex_export_remote_rgba32f_hdr_clip` — R32G32B32A32_FLOAT 1x1: 5.0 clipped to 1.0 ->
+   sRGB -> 255; 0.0 -> 0; alpha 255.
+7. `test_tex_export_remote_snorm_remaps_signed` — R8G8B8A8_SNORM read as int8: -1->0,
+   0->~128, +1->255.
+8. `test_tex_export_remote_uint8_passthrough` — R8G8B8A8_UINT 1x1: bytes pass through to RGBA.
+
+### Color decode — rejected formats (→ -32002)
+
+9. `test_tex_export_remote_length_mismatch_errors` — 4x4 RGBA8 with 4 bytes only -> -32002.
+10. `test_tex_export_remote_special_format_rejected` — BC1_UNORM (non-Regular) ->
+    "not supported".
+11. `test_tex_export_remote_sint_rejected` — R8G8B8A8_SINT (no display mapping) ->
+    "not supported".
+12. `test_tex_export_remote_typeless_rejected` — R8G8B8A8_TYPELESS -> -32002.
+13. `test_tex_export_remote_uscaled_rejected` — R8G8B8A8_USCALED -> -32002.
+14. `test_tex_export_remote_packed_format_rejected` — R11G11B10_FLOAT (packed, non-Regular)
+    -> "not supported".
+15. `test_tex_export_remote_msaa_rejected` — RGBA8_UNORM with `msSamp=4` -> -32002.
+16. `test_tex_export_remote_no_data_rejected` — `GetTextureData` returns `b""` ->
+    "no texture data", no `len(None)` crash.
+
+### `rt_export` remote mode
+
+17. `test_rt_export_remote_decodes_png` — `MockPipeState` output target -> B8G8R8A8_SRGB 2x2
+    `TextureDescription` in `tex_map`; result path is a valid PNG, mode RGBA.
+
+### `rt_depth` remote mode
+
+18. `test_rt_depth_remote_decodes_grayscale` — D32_FLOAT 2x2 depth target: mode `L`, min
+    depth -> 0, max depth -> 255, mid depth ~64.
+19. `test_rt_depth_remote_d16_decodes_grayscale` — D16 (uint16 depth, compType=Depth) 2x2:
+    mode `L`, 0 -> 0, 65535 -> 255, 16384 -> ~64.
+
+### Regression: local mode + rt_overlay
+
+20. `test_rt_overlay_remote_still_rejected` — `rt_overlay` remote -> -32002, "remote mode".
+21. `test_tex_export_local_uses_savetexture` — `is_remote=False`: `SaveTexture` is called
+    exactly once (local path unchanged, not routed through the decode helper).
+
+## Manual / GPU proxy check
+
+The following cannot run in CI (requires a live remote daemon + real GPU):
+
+- Connect client to a remote daemon replaying a vkcube capture; call `tex_export` on a
+  known color target; verify the local PNG opens correctly and matches the texture as
+  displayed in the RenderDoc GUI.
+- Same for `rt_export` and `rt_depth`.
+- Verify a BC7-compressed texture returns `-32002` (not a garbled PNG).
+
+## Run command
+
+```bash
+pytest tests/unit/test_tex_stats_handler.py -q
+```
+
+Optional (explicit interpreter / renderdoc module path used during development):
+
+```bash
+RENDERDOC_PYTHON_PATH=/usr/lib/python3.14/site-packages ./.venv-236/bin/python -m pytest tests/unit/test_tex_stats_handler.py -q
+```
+
+Full gate: `pixi run check` (lint + typecheck + unit tests).

src/rdc/commands/_helpers.py

@@ -22,6 +22,7 @@
     "require_renderdoc",
     "call",
     "call_binary",
+    "call_with_code",
     "try_call",
     "completion_call",
     "fetch_remote_file",
@@ -142,6 +143,29 @@ def try_call(method: str, params: dict[str, Any]) -> dict[str, Any] | None:
     return cast(dict[str, Any], response.get("result", {}))
 
 
+def call_with_code(method: str, params: dict[str, Any]) -> tuple[dict[str, Any] | None, int | None]:
+    """Send a JSON-RPC request, returning (result, error_code).
+
+    Like try_call() but surfaces the JSON-RPC error code so callers can
+    distinguish a missing resource (-32001) from an unsupported operation
+    (-32002). On success returns (result, None); on any failure returns
+    (None, code) where code is the daemon error code or None when the call
+    never reached the daemon.
+    """
+    try:
+        host, port, token = require_session()
+    except SystemExit:
+        return None, None
+    payload = _request(method, 1, {"_token": token, **params}).to_dict()
+    try:
+        response = send_request(host, port, payload)
+    except (OSError, ValueError):
+        return None, None
+    if "error" in response:
+        return None, response["error"].get("code")
+    return cast(dict[str, Any], response.get("result", {})), None
+
+
 def completion_call(method: str, params: dict[str, Any]) -> dict[str, Any] | None:
     """Send a request for shell completion, silently returning None on failure."""
     with contextlib.redirect_stderr(io.StringIO()):

src/rdc/commands/snapshot.py

@@ -8,7 +8,13 @@
 
 import click
 
-from rdc.commands._helpers import call, complete_eid, fetch_remote_file, try_call
+from rdc.commands._helpers import (
+    call,
+    call_with_code,
+    complete_eid,
+    fetch_remote_file,
+    try_call,
+)
 from rdc.formatters.json_fmt import write_json
 
 
@@ -39,21 +45,30 @@ def snapshot_cmd(eid: int, output: str, use_json: bool) -> None:
                 (out_dir / f"shader_{stage}.txt").write_text(disasm_resp["disasm"])
                 files.append(f"shader_{stage}.txt")
 
-    # Color targets (stop on first failure)
+    # Color targets: stop when a target is absent (-32001), but skip-and-warn
+    # on a decode failure (-32002, e.g. an unsupported remote format) so one
+    # bad target does not silently truncate the rest of the bundle.
     for i in range(8):
-        result = try_call("rt_export", {"eid": eid, "target": i})
-        if result is None:
-            break
-        data = fetch_remote_file(result["path"])
-        (out_dir / f"color{i}.png").write_bytes(data)
-        files.append(f"color{i}.png")
+        result, code = call_with_code("rt_export", {"eid": eid, "target": i})
+        if result is not None:
+            data = fetch_remote_file(result["path"])
+            (out_dir / f"color{i}.png").write_bytes(data)
+            files.append(f"color{i}.png")
+            continue
+        if code == -32002:
+            click.echo(f"snapshot: skipped color{i} (decode unsupported)", err=True)
+            continue
+        break
 
-    # Depth target
-    depth_result = try_call("rt_depth", {"eid": eid})
-    if depth_result:
+    # Depth target: surface a warning when remote depth decode is unsupported
+    # (e.g. D24S8 or MSAA) instead of silently omitting depth.png.
+    depth_result, depth_code = call_with_code("rt_depth", {"eid": eid})
+    if depth_result is not None:
         data = fetch_remote_file(depth_result["path"])
         (out_dir / "depth.png").write_bytes(data)
         files.append("depth.png")
+    elif depth_code == -32002:
+        click.echo("snapshot: skipped depth (decode unsupported)", err=True)
 
     # Manifest
     manifest = {