feat: test failure scores 0 + set vercel pkg to 1.7.15#67
Merged
Conversation
Contributor
Storage Benchmark Results10MB Files
View full run · SVGs available as build artifacts |
Contributor
Sandbox Benchmark ResultsSequential
Staggered
Burst
View full run · SVGs available as build artifacts |
There was a problem hiding this comment.
Pull request overview
Updates benchmark result handling so provider runs with 0 successful iterations are treated as real failures (score = 0) rather than being marked as “skipped”, and pins the Vercel provider dependency to a specific version.
Changes:
- Remove “all iterations failed ⇒ skipped” behavior across sandbox/storage/browser benchmarks and result renderers.
- Ensure stats/time-to-first-ready calculations don’t error on empty successful sets (default to zeros).
- Pin
@computesdk/vercelto1.7.15(and update lockfile accordingly).
Reviewed changes
Copilot reviewed 9 out of 10 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/storage/benchmark.ts | Stops marking all-failure storage runs as skipped; computes stats from successful-only set. |
| src/browser/benchmark.ts | Stops marking all-failure browser runs as skipped; stats already handle empty successful set. |
| src/sandbox/benchmark.ts | Removes skip-on-all-failures and defaults stats to zeros when no successes. |
| src/sandbox/concurrent.ts | Removes skip-on-all-failures; guards min/stats when no successes. |
| src/sandbox/staggered.ts | Removes skip-on-all-failures; guards min/stats when no successes. |
| src/sandbox/table.ts | Removes special FAILED row; always reports success ratio in “Status”. |
| src/run.ts | Adds (ok/total OK) to printed scores for storage and browser modes. |
| src/merge-results.ts | Removes special FAILED rows in merged storage/browser tables; always reports success ratio. |
| package.json | Pins @computesdk/vercel to 1.7.15. |
| package-lock.json | Updates lockfile to match the pinned @computesdk/vercel version. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.