feat(bench): expand diff-core benchmark with realistic shapes (#29)#48
Merged
Conversation
SBOM diffNo dependency changes. |
Coverage reportLine coverage: 84.1% (9378 / 11150 lines) Full lcov report available as workflow artifact coverage-lcov: download from this run. v0.9.8 introduces this report; |
Closes #29. The previous benches/diff.rs covered the axios fixture pair and a 200-component synthetic group with a single workload shape. That's fine as a smoke test but not enough to catch perf regressions in the diff hot path — especially the kind that wouldn't show up until a monorepo-scale SBOM lands in production. This expands the bench to three sizes (500 / 5_000 / opt-in 20_000) crossed with three workload shapes per size: - end_to_end: realistic mix (80% unchanged, 10% version_changed, 5% license_changed, 5% added/removed). Production hot path. - self_diff: identical inputs. Isolates group_by_key BTreeMap construction + per-key traversal with no change-pair work. - all_license_changed: every intersecting pair has a different license set. Isolates the license-comparison branch in diff_one_key. Throughput is reported per group so reviewers can read a regression as 'X% per component' across sizes vs an algorithmic shape change. The 20_000-component group is gated behind a new bench-stress cargo feature so the default 'cargo bench --bench diff' stays under the 30s budget. On my laptop (--quick mode) the full set runs in ~1.4s; the default measurement profile fits comfortably under 30s. docs/src/architecture.md gains a 'Perf reference: diff core benchmark' section linking benches/diff.rs and stating the >5% regression threshold for the production end_to_end / 5000 workload, mirroring the established 'Binary size budget' style. Verified: - cargo bench --bench diff --no-run: builds clean. - cargo bench --bench diff -- --quick: all 7 benches produce stable medians (axios + 3 workloads x 2 sizes). - cargo clippy --bench diff --all-features --release -- -D warnings: clean.
Metbcy
force-pushed
the
feat/29-bench-diff-core
branch
from
1dd18f4 to
e67554a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #29.
What
Rewrites
benches/diff.rsto cover realistic diff shapes against the existing axios fixture plus synthetic workloads at 3 sizes × 3 patterns. A 20_000-entry stress run is gated behind abench-stresscargo feature so the defaultcargo benchstays under ~30s.Bench groups
axios_fixture_pair— real-world before/aftersynth/end_to_end/{500,5000}— mixed adds/removes/changessynth/self_diff/{500,5000}— proxy forgroup_by_key+ iteration cost without exposing internalssynth/all_license_changed/{500,5000}— worst-case change-heavy pathSample medians (--quick, laptop)
Files
benches/diff.rs— rewritten (7 bench groups)Cargo.toml—[features] bench-stress = []docs/src/architecture.md— "Perf reference: diff core benchmark" sectionVerification
cargo bench --bench diff --no-run✓cargo bench --bench diff -- --quick✓ (1.4s, stable medians)cargo clippy --bench diff --all-features --release -- -D warnings✓Notes
self_diffis used as a proxy to exercisegroup_by_key+ iteration cost without widening thediffmodule surface.