Noah/browser benchmark

[kisernl]

Summary

• Adds a new src/browser/ benchmark module that tests the full browser session lifecycle: create → CDP connect → navigate → release
• Integrates browser mode into the main runner (--mode browser) with provider filtering support
• Adds browserbase as the first browser provider
• Adds a CI workflow that runs 10-iteration benchmarks on PR and posts results as a comment (no result publishing on merge)

New files

• src/browser/types.ts — types for browser benchmark configs, timings, and results
• src/browser/providers.ts — browser provider configurations (starting with Browserbase)
• src/browser/benchmark.ts — sequential benchmark runner with per-phase timing
• src/browser/scoring.ts — composite scoring weighted by total/create latency and success rate
• .github/workflows/browser-benchmarks.yml — CI workflow for PR checks

Usage

npm run bench:browser
npm run bench:browser:browserbase
npx tsx src/run.ts --mode browser --iterations 10

[github-actions]

Storage Benchmark Results

10MB Files

#	Provider	Score	Download	Throughput	Upload	Status
1	AWS S3	93.4	0.12s	688.8 Mbps	0.47s	10/10
2	Tigris	92.1	0.13s	627.2 Mbps	0.79s	10/10
3	Cloudflare R2	77.2	0.50s	167.8 Mbps	0.87s	10/10

---

View full run · SVGs available as build artifacts

[github-actions]

Sandbox Benchmark Results

Sequential

#	Provider	Score	Median TTI	P95	P99	Status
1	daytona	97.8	0.16s	0.31s	0.31s	10/10
2	blaxel	95.3	0.44s	0.52s	0.52s	10/10
3	e2b	94.3	0.45s	0.74s	0.74s	10/10
4	hopx	87.3	1.15s	1.44s	1.44s	10/10
5	runloop	78.8	1.95s	2.37s	2.37s	10/10
6	cloudflare	77.9	1.82s	2.80s	2.80s	10/10
7	modal	75.2	2.02s	3.16s	3.16s	10/10
8	namespace	69.5	2.06s	4.54s	4.54s	10/10
9	codesandbox	0.0	23.46s	25.48s	25.48s	10/10

Staggered

#	Provider	Score	Median TTI	P95	P99	Status
1	daytona	98.2	0.11s	0.29s	0.29s	10/10
2	blaxel	95.3	0.46s	0.48s	0.48s	10/10
3	e2b	92.3	0.43s	1.28s	1.28s	10/10
4	hopx	88.7	1.01s	1.30s	1.30s	10/10
5	cloudflare	79.6	1.60s	2.69s	2.69s	10/10
6	namespace	74.5	2.33s	2.89s	2.89s	10/10
7	runloop	73.1	2.56s	2.89s	2.89s	10/10
8	modal	46.0	2.34s	18.50s	18.50s	10/10
9	codesandbox	0.0	12.63s	28.75s	28.75s	10/10

Burst

#	Provider	Score	Median TTI	P95	P99	Status
1	daytona	97.6	0.17s	0.34s	0.34s	10/10
2	blaxel	93.5	0.60s	0.72s	0.72s	10/10
3	e2b	91.1	0.50s	1.47s	1.47s	10/10
4	hopx	83.8	1.60s	1.64s	1.64s	10/10
5	cloudflare	78.9	1.96s	2.33s	2.33s	10/10
6	runloop	77.0	1.96s	2.81s	2.81s	10/10
7	namespace	73.4	2.43s	3.02s	3.02s	10/10
8	modal	48.6	1.90s	26.90s	26.90s	10/10
9	codesandbox	0.0	23.98s	24.79s	24.79s	10/10

---

View full run · SVGs available as build artifacts

[github-actions]

Browser Benchmark Results

No browser benchmark results were generated.

---

View full run

[Copilot]

Pull request overview

This PR introduces a new browser benchmark mode that measures a full remote browser session lifecycle (create → CDP connect → navigate → release), integrates it into the main benchmark runner (--mode browser), and adds CI automation to run and comment benchmark results on PRs.

Changes:

• Added a new src/browser/* benchmark module (types, runner, scoring, providers).
• Integrated browser mode into src/run.ts and src/merge-results.ts (including score computation + merged result output).
• Added a PR workflow to run browser benchmarks and post a results table comment.

Reviewed changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
src/storage/providers.ts	Minor comment tweak in provider list.
src/run.ts	Adds --mode browser execution path, provider filtering, and JSON output to results/browser/.
src/merge-results.ts	Adds browser-mode merge + table output + writes merged results/browser/latest.json.
src/browser/types.ts	Defines browser benchmark config/result/timing/stats types.
src/browser/scoring.ts	Adds composite scoring and sorting for browser benchmark results.
src/browser/providers.ts	Adds browser provider registry (currently empty).
src/browser/benchmark.ts	Implements sequential browser benchmark loop, timing, and JSON writer.
package.json	Adds browser bench scripts + dependencies (@computesdk/browserbase, playwright-core), changes @computesdk/blaxel version range.
package-lock.json	Locks added/updated dependencies for browser benchmarking.
.github/workflows/browser-benchmarks.yml	New workflow to run browser benchmarks on PRs and comment results.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.