feat(torture): I/O fault-injection runner for wsc-verify-core#131
Open
avrabe wants to merge 1 commit into
Open
feat(torture): I/O fault-injection runner for wsc-verify-core#131avrabe wants to merge 1 commit into
avrabe wants to merge 1 commit into
Conversation
Adds verification/torture-runner/, a workspace-excluded crate that torture-tests wsc-verify-core's parsers against I/O fault injection — the Rust-faithful analog of curl's allocation-failure torture model (see README for why allocation torture does not transfer to Rust: safe-Rust collection APIs abort on null-return from GlobalAlloc rather than returning Err, so the "fail Nth malloc" model produces aborts not Result error-path exercise). The runner feeds the target a `FaultyReader` that returns the first k bytes cleanly then errors, for every k in 0..=input.len(). Asserts zero panics across all fault points. The injected error uses `ErrorKind::Other` because stdlib wrappers (BufReader, read_to_end) silently auto-retry `Interrupted` and would loop forever — caught and fixed during PoC development. Two demo targets ship: - `torture_module_parser`: Module::init_from_reader + iterate on a 15-byte valid WASM module. 16 fault points, all clean. - `torture_sig_parser`: SignatureForHashes::deserialize on a 7-byte valid signature payload. 8 fault points, all clean. Both targets survive torture with zero panics on the current verify-core — the parsers already handle every byte-position read error gracefully. This is now a regression gate available for any future verify-core change. This closes the third leg of the curl-comparison rigor triangle — sigil already had mutation testing (cargo-mutants) and MC/DC (witness, PoC); fault injection is the path-coverage complement that makes the other two load-bearing (per the curl writeup, branch coverage of an error path is meaningless if no test reaches it). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
3 tasks
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds `verification/torture-runner/`, a workspace-excluded crate that torture-tests `wsc-verify-core`'s parsers against I/O fault injection — the Rust-faithful analog of curl's torture model.
This is the third leg of the curl-comparison rigor triangle: sigil already has mutation testing (`cargo-mutants`) and MC/DC (`witness`, Phase-1 PoC). Fault injection is the path-coverage complement that makes the other two load-bearing — per the curl writeup, branch coverage of an error path is meaningless if no test reaches it.
Why I/O, not allocation
The classic curl model overrides `malloc` to fail on the Nth call. That model does not transfer to Rust. Safe-Rust collection APIs (`Vec::with_capacity`, `Box::new`, …) abort the process via `handle_alloc_error` on null-return from `GlobalAlloc` — they do not return `Result<_, AllocError>`. The "fail Nth malloc" approach in Rust produces aborts, not the `Result` error-path exercise we want.
The Rust analog is I/O fault injection: every `?` on `std::io::Read` and every error variant in `CoreError` is a reachable error path by design. Drive a `Read` impl that returns `Err` at a chosen byte offset → exercise every byte-position where an OS could produce a real I/O error.
How it works
For target `f(reader) -> Result<_, _>` and a valid byte input of length `N`:
The injected error uses `ErrorKind::Other` (NOT `Interrupted`) because stdlib wrappers silently auto-retry `Interrupted` and would loop forever — caught + fixed during PoC development.
Demo output
```
[Module::init_from_reader + iterate] input 15 bytes — 16 fault points exercised: ok=1 err=15 panic=0
✔ Module::init_from_reader survived I/O torture at every byte offset.
[SignatureForHashes::deserialize] input 7 bytes — 8 fault points exercised: ok=1 err=7 panic=0
✔ SignatureForHashes::deserialize survived I/O torture at every byte offset.
```
Both targets are clean today; the runner is now a regression gate any future verify-core change will run against.
Follow-ups (out of scope)
Test plan
🤖 Generated with Claude Code