Skip to content

update simplew-tuned#893

Open
stratdev3 wants to merge 1 commit into
MDA2AV:mainfrom
stratdev3:update-simplew-tuned
Open

update simplew-tuned#893
stratdev3 wants to merge 1 commit into
MDA2AV:mainfrom
stratdev3:update-simplew-tuned

Conversation

@stratdev3

@stratdev3 stratdev3 commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Description

PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run all benchmark tests
/benchmark -f <framework> -t <test> Run a specific test
/benchmark -f <framework> --save Run and save results (updates leaderboard on merge)

Always specify -f <framework>. Results are automatically compared against the current leaderboard.

Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

@stratdev3

stratdev3 commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark -f simplew-tuned

@github-actions

github-actions Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions

github-actions Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: simplew-tuned | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 2,024,690 5488.4% 177MiB +0.4% +2.3%
baseline 4096 1,999,532 5487.5% 424MiB +2.2% -9.0%
pipelined 512 3,560,906 6088.7% 221MiB +2.3% +8.9%
pipelined 4096 3,432,622 5880.0% 263MiB +0.2% +28.9%
limited-conn 512 900,841 4128.0% 116MiB +4.6% +9.4%
limited-conn 4096 896,193 3964.3% 242MiB +0.6% ~0%
json 4096 857,916 5774.7% 268MiB +1.8% +10.7%
json-comp 512 480,865 5797.4% 181MiB +1.6% -3.7%
json-comp 4096 496,334 5739.8% 284MiB +2.1% -23.2%
json-comp 16384 493,129 5659.8% 382MiB +1.6% -3.3%
json-tls 4096 680,763 6044.3% 932MiB ~0% +0.2%
upload 32 1,822 2820.8% 2.3GiB +0.8% +4.5%
upload 256 1,974 6336.1% 6.7GiB +0.3% +1.5%
api-4 256 42,118 389.6% 132MiB -25.2% -5.7%
api-16 1024 143,435 1538.5% 228MiB -1.8% +3.2%
static 1024 302,607 5223.5% 361MiB -15.6% -13.4%
static 4096 225,518 5558.6% 952MiB -9.3% -4.4%
static 6800 242,613 5597.0% 2.2GiB -5.9% -18.5%
async-db 1024 174,908 3208.6% 326MiB +2.7% +0.6%
crud 4096 442,831 4006.0% 954MiB +6.1% -0.6%
echo-ws 512 2,002,602 5700.0% 201MiB +1.3% +2.0%
echo-ws 4096 1,870,970 5485.8% 502MiB -1.7% +15.9%
echo-ws 16384 1,623,143 5531.0% 1.2GiB +0.1% -7.7%
echo-ws-pipeline 512 3,122,380 6165.4% 170MiB +1.5% -20.6%
echo-ws-pipeline 4096 2,974,222 6200.8% 253MiB ~0% -4.9%
echo-ws-pipeline 16384 2,766,035 5730.3% 626MiB +0.6% +45.9%
Full log 
  Throughput: 2.82M req/s
  Bandwidth:  18.84MB/s
  WS upgrades: 508
  WS frames:   14109979
  Latency samples: 14105568 / 14109979 responses (100.0%)
[info] CPU 5810.3% | Mem 244MiB

=== Best: 3122380 req/s (CPU: 6165.4%, Mem: 170MiB) ===
[info] saved results/echo-ws-pipeline/512/simplew-tuned.json
httparena-bench-simplew-tuned
httparena-bench-simplew-tuned

==============================================
=== simplew-tuned / echo-ws-pipeline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    608us    351us    508us   2.08ms   8.51ms

  12462447 frames sent in 5.00s, 12472228 frames received
  Throughput: 2.49M req/s
  Bandwidth:  16.64MB/s
  WS upgrades: 668
  WS frames:   12472228
  Latency samples: 12454069 / 12472228 responses (99.9%)
[info] CPU 5907.3% | Mem 126MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    463us    317us    368us    892us   5.30ms

  14880942 frames sent in 5.00s, 14871113 frames received
  Throughput: 2.97M req/s
  Bandwidth:  19.87MB/s
  WS upgrades: 1088
  WS frames:   14871113
  Latency samples: 14870125 / 14871113 responses (100.0%)
  Latency overflow (>5s): 25
[info] CPU 6200.8% | Mem 253MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    881us    322us    372us   3.29ms   10.90ms

  14708798 frames sent in 5.00s, 14684319 frames received
  Throughput: 2.94M req/s
  Bandwidth:  19.65MB/s
  WS upgrades: 2036
  WS frames:   14684319
  Latency samples: 14684276 / 14684319 responses (100.0%)
  Latency overflow (>5s): 11
[info] CPU 6028.2% | Mem 394MiB

=== Best: 2974222 req/s (CPU: 6200.8%, Mem: 253MiB) ===
[info] saved results/echo-ws-pipeline/4096/simplew-tuned.json
httparena-bench-simplew-tuned
httparena-bench-simplew-tuned

==============================================
=== simplew-tuned / echo-ws-pipeline / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.36ms    363us    546us   3.18ms   152.00ms

  11592452 frames sent in 5.00s, 11566807 frames received
  Throughput: 2.31M req/s
  Bandwidth:  15.49MB/s
  WS upgrades: 1603
  WS frames:   11566807
  Latency samples: 11566807 / 11566807 responses (100.0%)
[info] CPU 5623.7% | Mem 202MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   2.69ms    338us    405us   3.93ms    1.01s

  13510248 frames sent in 5.00s, 13457472 frames received
  Throughput: 2.69M req/s
  Bandwidth:  18.07MB/s
  WS upgrades: 3299
  WS frames:   13457472
  Latency samples: 13457472 / 13457472 responses (100.0%)
[info] CPU 6131.5% | Mem 438MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   2.72ms    333us    392us   3.38ms   999.30ms

  13866217 frames sent in 5.00s, 13830176 frames received
  Throughput: 2.76M req/s
  Bandwidth:  18.53MB/s
  WS upgrades: 2509
  WS frames:   13830176
  Latency samples: 13830176 / 13830176 responses (100.0%)
[info] CPU 5730.3% | Mem 626MiB

=== Best: 2766035 req/s (CPU: 5730.3%, Mem: 626MiB) ===
[info] saved results/echo-ws-pipeline/16384/simplew-tuned.json
httparena-bench-simplew-tuned
httparena-bench-simplew-tuned
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/crud-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MDA2AV MDA2AV Awaiting requested review from MDA2AV MDA2AV is a code owner

@Kaliumhexacyanoferrat Kaliumhexacyanoferrat Awaiting requested review from Kaliumhexacyanoferrat Kaliumhexacyanoferrat is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@stratdev3