feat(zcash_local_net)!: containerized artifacts with a consumer manifest + preflight pre-check

[dorianvp]

What

Adds the ability to run the whole zcash_local_net setup from containers (docker or podman), replicating all currently supported functionality, plus a consumer-authored artifact manifest (TOML) that doubles as a pre-check and a lockfile, and an escape hatch for artifacts built locally from source.

Container mode is a spawn substitution, not a parallel implementation

Every managed process (zebrad Validator, zainod Indexer, zcash-devtool Wallet) already flowed through one command-construction choke point. That choke point is now an ArtifactSource on each launch config:

• HostProcess { binary: None } — the default: TEST_BINARIES_DIR / PATH resolution, byte-for-byte the historical behavior. Existing callers are unaffected.
• HostProcess { binary: Some(path) } — the escape hatch: run a locally built zebrad/zainod/zcash-devtool directly (ArtifactSource::local_binary(...), or local = "…" in the manifest).
• Container(ContainerImage { runtime, image, entrypoint, pull }) — run the binary from its actual published image (e.g. Zebra's official zfnd/zebra), one image per artifact.

Containers launch in the foreground (run --rm, never --detach), so their stdio streams through the child process the harness already manages; they join the host network namespace; and the harness's temp config/data dirs are bind-mounted at identical paths, so the generated config files are valid verbatim inside. As a result, readiness-indicator scanning, launch-log endpoint discovery, port-collision retry, the front proxies, and Indexer-convergence parsing are shared verbatim between host and container modes — LocalNet, launch_wallet, generate_blocks_converged, chain caching, observers, everything works identically. stop() force-removes the container by name (zcash-local-net-<binary>-<pid>-<n>); wallet operations each run as one-shot containers (with --interactive for the stdin-fed init).

Host networking makes container mode Linux-first (Docker Desktop VM loopback is not host-reachable); documented in crate::container.

The manifest: intent + lockfile in one TOML file

TOML — the configuration language the rest of the ecosystem already speaks (zebrad.toml, zindexer.toml), comments included; zcash-local-net template prints a fully commented starting point:

version = 1
runtime = "docker"

[validator]
image = "zfnd/zebra:v6.0.0"
track = "zfnd/zebra:latest"
pull = "if-missing"

[indexer]
local = "/home/dev/zaino/target/release/zainod"

[wallet]

Image references must be pinned. Untagged and :latest references are rejected at load time (ManifestError::UnpinnedImage) — digest pins (repo@sha256:…) are the strongest form, an explicit non-latest tag the minimum. A registry port (registry:5000/zebra) is not mistaken for a tag. Preflight reports the immutable identity each reference resolved to (repo digest, or local image ID for locally built images), so tag-pinned runs leave comparable evidence.

Pins move only through an explicit, well-defined bump. track records the floating preference a consumer wants to follow (…:latest, a release-channel tag); image records the pin every run uses. Nothing at run time asks the registry what a tag currently means. To move the pins:

zcash-local-net update --manifest ci-artifacts.toml          # all artifacts
zcash-local-net update --manifest ci-artifacts.toml indexer  # a subset

resolves each tracked reference (the track field, or the tag image was pinned from) against the registry and rewrites image to repo:tag@sha256:… in canonical TOML — a reviewable version-control diff per bump, exactly like a lockfile update. A track-only artifact is deliberately not launchable until its first update mints the pin (ManifestError::UnresolvedTrack says so and names the command); digest-only images without track have no floating preference and are never touched. Library entry point: container::update::update_manifest_file.

• In-suite: ArtifactManifest::from_env()?.unwrap_or_default() → .preflight() (the pre-check) → .launch_local_net(), or seed individual configs via zebrad_config() / zainod_config() / wallet_source().
• From a shell, rather than a special test runner, the pre-check CLI wraps the existing one:

zcash-local-net preflight --manifest ci-artifacts.toml \
  && ZCASH_LOCAL_NET_MANIFEST=ci-artifacts.toml cargo nextest run

preflight verifies, all at once and before any launch: runtime client present and daemon reachable, images present locally (pulled per their never/if-missing/always policy — also enforced at launch via run --pull=), and host binaries resolvable and executable.

Breaking changes (0.x, changelogged)

• ZebradConfig, ZainodConfig, ZcashDevtoolConfig gain a public source field (struct-literal constructors must add it; Default/builder users unaffected).
• Zebrad::stop now tolerates an already-reaped child (matching Zainod::stop).

Dependencies

One new dependency: toml 0.8 (manifest parsing/serialization). Its stack (toml_edit, toml_datetime, serde_spanned, winnow) is MIT/Apache within the cargo-deny allow list, and supply-chain/config.toml already carries cargo-vet exemptions for the toml 0.8 line.

Verification

• 35 new unit tests: manifest parse/validation (unknown keys, version, image+local conflict, pull/runtime parsing, pinning acceptance/rejection matrix, track rules incl. track-on-local and digest-pinned-track rejection), preflight failure modes (missing binary, non-executable, report rendering), the container-run CLI contract (flag shape/order, identical-path volume syntax, entrypoint default, --pull mapping) pinned exactly, and the update/bump semantics (tracked and tag-pinned artifacts bumped, first-pin minting, artifact filter, no-op re-run, digest-only untouched, resolver failure leaves the file untouched, post-bump validation) — the registry interaction sits behind an injectable resolver seam, so update logic is tested hermetically.
• Live validation in-session against a real docker daemon with a scratch image (registry pulls are blocked in this sandbox, so a FROM scratch static test daemon stood in): foreground log streaming, host-loopback reachability, mount write-through, stdin piping, rm --force semantics (including modern docker's exit-0 on missing container), digest reporting in the preflight output, unpinned-manifest rejection, update CLI error paths (unreachable registry leaves the manifest untouched; track-only manifests refuse to launch with the remediation message), TOML template/preflight round-trip, and an end-to-end Zainod::launch in container mode — config consumed inside the container, readiness scanned from container logs, front proxy relaying into it, clean teardown with no leftover containers. What was not exercised live: real zebrad/zainod images (none pullable here); tests/containers.rs ships an #[ignore]d manifest-driven end-to-end test (preflight → LocalNet → mine → convergence) for environments that provide them.
• cargo check --workspace --all-targets (also with --features legacy-stack), cargo clippy clean on all new code under -D warnings (3 pre-existing integration.rs lints on the dev baseline are untouched), cargo fmt, cargo doc clean, doc tests pass. No external types added to the public API.

Notes for reviewers

• The Backend trait's design constraint ("a container backend could implement this without changing one line of front") is honored in spirit by a stronger move: containers reuse the same backends, so front.rs and friends are untouched.
• update rewrites the manifest in a canonical formatting (the serde schema also serializes, in declaration order) rather than preserving arbitrary user formatting — deny_unknown_fields means there is nothing but formatting (and comments, which a rewrite drops) to lose.
• Direct ContainerImage construction (programmatic, non-manifest) bypasses the pinning check by design; its field docs state that such callers carry the reproducibility responsibility themselves.
• Pre-existing quirk noticed while wiring mounts (not fixed here, caller-side expectations unclear): Zebrad::launch_once ignores load_chain's return value, so a non-regtest chain_cache is arguably mis-plumbed on dev already. Happy to open a tracking issue.

🤖 Generated with Claude Code

https://claude.ai/code/session_015dRfXaAyhrUC5LwgEGHaNi