Skip to content

feat(torture): I/O fault-injection runner for wsc-verify-core#131

Open
avrabe wants to merge 1 commit into
mainfrom
feat/torture-runner-poc
Open

feat(torture): I/O fault-injection runner for wsc-verify-core#131
avrabe wants to merge 1 commit into
mainfrom
feat/torture-runner-poc

Conversation

@avrabe
Copy link
Copy Markdown
Contributor

@avrabe avrabe commented May 24, 2026

Summary

Adds `verification/torture-runner/`, a workspace-excluded crate that torture-tests `wsc-verify-core`'s parsers against I/O fault injection — the Rust-faithful analog of curl's torture model.

This is the third leg of the curl-comparison rigor triangle: sigil already has mutation testing (`cargo-mutants`) and MC/DC (`witness`, Phase-1 PoC). Fault injection is the path-coverage complement that makes the other two load-bearing — per the curl writeup, branch coverage of an error path is meaningless if no test reaches it.

Why I/O, not allocation

The classic curl model overrides `malloc` to fail on the Nth call. That model does not transfer to Rust. Safe-Rust collection APIs (`Vec::with_capacity`, `Box::new`, …) abort the process via `handle_alloc_error` on null-return from `GlobalAlloc` — they do not return `Result<_, AllocError>`. The "fail Nth malloc" approach in Rust produces aborts, not the `Result` error-path exercise we want.

The Rust analog is I/O fault injection: every `?` on `std::io::Read` and every error variant in `CoreError` is a reachable error path by design. Drive a `Read` impl that returns `Err` at a chosen byte offset → exercise every byte-position where an OS could produce a real I/O error.

How it works

For target `f(reader) -> Result<_, _>` and a valid byte input of length `N`:

  1. Run `f` once per fault point `0..=N`.
  2. Each run feeds `f` a `FaultyReader` that returns the first `k` bytes cleanly then errors.
  3. Assert every run returns cleanly — `Ok()` if `f` consumed less than `k` bytes, `Err()` otherwise. A panic from any run is a torture failure.

The injected error uses `ErrorKind::Other` (NOT `Interrupted`) because stdlib wrappers silently auto-retry `Interrupted` and would loop forever — caught + fixed during PoC development.

Demo output

```
[Module::init_from_reader + iterate] input 15 bytes — 16 fault points exercised: ok=1 err=15 panic=0
✔ Module::init_from_reader survived I/O torture at every byte offset.

[SignatureForHashes::deserialize] input 7 bytes — 8 fault points exercised: ok=1 err=7 panic=0
✔ SignatureForHashes::deserialize survived I/O torture at every byte offset.
```

Both targets are clean today; the runner is now a regression gate any future verify-core change will run against.

Follow-ups (out of scope)

  • More targets — `PublicKey::verify_multi` with signed-module fixtures, larger / multi-section module inputs, every `*_from_reader` entry.
  • Filesystem torture for `secure_file::read_secure` (a `FaultyOpenOptions`).
  • Network torture for `wsc`'s keyless Fulcio/Rekor calls (a fault-injecting `ureq::Transport` — lives in `wsc`, not the verify-core).
  • CI gate (a workflow that runs the torture binaries on every PR).

Test plan

  • `cargo run --bin torture_module_parser` clean
  • `cargo run --bin torture_sig_parser` clean
  • CI green

🤖 Generated with Claude Code

Adds verification/torture-runner/, a workspace-excluded crate that
torture-tests wsc-verify-core's parsers against I/O fault injection —
the Rust-faithful analog of curl's allocation-failure torture model
(see README for why allocation torture does not transfer to Rust:
safe-Rust collection APIs abort on null-return from GlobalAlloc rather
than returning Err, so the "fail Nth malloc" model produces aborts not
Result error-path exercise).

The runner feeds the target a `FaultyReader` that returns the first
k bytes cleanly then errors, for every k in 0..=input.len(). Asserts
zero panics across all fault points. The injected error uses
`ErrorKind::Other` because stdlib wrappers (BufReader, read_to_end)
silently auto-retry `Interrupted` and would loop forever — caught and
fixed during PoC development.

Two demo targets ship:
- `torture_module_parser`: Module::init_from_reader + iterate on a
  15-byte valid WASM module. 16 fault points, all clean.
- `torture_sig_parser`: SignatureForHashes::deserialize on a 7-byte
  valid signature payload. 8 fault points, all clean.

Both targets survive torture with zero panics on the current
verify-core — the parsers already handle every byte-position read
error gracefully. This is now a regression gate available for any
future verify-core change.

This closes the third leg of the curl-comparison rigor triangle —
sigil already had mutation testing (cargo-mutants) and MC/DC
(witness, PoC); fault injection is the path-coverage complement that
makes the other two load-bearing (per the curl writeup, branch
coverage of an error path is meaningless if no test reaches it).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@codecov
Copy link
Copy Markdown

codecov Bot commented May 24, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant