Skip to content

update simplew#892

Open
stratdev3 wants to merge 1 commit into
MDA2AV:mainfrom
stratdev3:update-simplew
Open

update simplew#892
stratdev3 wants to merge 1 commit into
MDA2AV:mainfrom
stratdev3:update-simplew

Conversation

@stratdev3

@stratdev3 stratdev3 commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Description

PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run all benchmark tests
/benchmark -f <framework> -t <test> Run a specific test
/benchmark -f <framework> --save Run and save results (updates leaderboard on merge)

Always specify -f <framework>. Results are automatically compared against the current leaderboard.

Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

@stratdev3

stratdev3 commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark -f simplew

@github-actions

github-actions Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions

github-actions Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: simplew | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 2,100,097 5626.1% 172MiB +5.4% -13.6%
baseline 4096 2,032,351 5653.9% 446MiB +4.1% -37.4%
pipelined 512 6,056,436 5699.6% 277MiB +1.7% -6.4%
pipelined 4096 6,093,677 5772.7% 356MiB +3.5% -19.1%
limited-conn 512 894,747 4052.2% 110MiB +2.3% -30.4%
limited-conn 4096 877,430 3653.4% 249MiB -3.3% -6.4%
json 4096 866,248 5805.3% 345MiB +2.8% -2.0%
json-comp 512 477,476 5660.3% 181MiB -0.9% -13.4%
json-comp 4096 489,255 5843.2% 226MiB -0.5% -27.1%
json-comp 16384 495,312 6016.7% 279MiB +2.0% -32.4%
json-tls 4096 685,508 6127.4% 1.0GiB +110.9% -9.1%
upload 32 1,820 2928.2% 2.2GiB +1.2% +4.8%
upload 256 1,958 5985.9% 6.4GiB +1.5% -3.0%
api-4 256 41,803 389.3% 134MiB -23.6% -14.1%
api-16 1024 142,503 1560.9% 242MiB -2.6% -10.0%
static 1024 71,313 4655.9% 918MiB -2.9% -3.0%
static 4096 73,122 4971.8% 1.3GiB -2.8% -13.3%
static 6800 72,158 4885.8% 1.7GiB -5.3% -10.5%
async-db 1024 175,115 3217.0% 340MiB -21.8% -9.1%
crud 4096 438,366 3993.1% 868MiB +1.9% -29.4%
echo-ws 512 2,087,329 5962.9% 143MiB +5.8% -0.7%
echo-ws 4096 1,891,710 5987.5% 453MiB -0.5% -10.8%
echo-ws 16384 1,677,331 5529.3% 1.4GiB +3.5% -22.2%
echo-ws-pipeline 512 4,503,177 5797.9% 250MiB +1.4% -31.5%
echo-ws-pipeline 4096 3,949,996 5765.7% 591MiB -8.9% +47.0%
echo-ws-pipeline 16384 3,621,294 5499.6% 627MiB -4.2% -6.1%
Full log 
  22521137 frames sent in 5.00s, 22515889 frames received
  Throughput: 4.50M req/s
  Bandwidth:  30.06MB/s
  WS upgrades: 512
  WS frames:   22515889
  Latency samples: 22515815 / 22515889 responses (100.0%)
[info] CPU 5797.9% | Mem 250MiB

=== Best: 4503177 req/s (CPU: 5797.9%, Mem: 250MiB) ===
[info] saved results/echo-ws-pipeline/512/simplew.json
httparena-bench-simplew
httparena-bench-simplew

==============================================
=== simplew / echo-ws-pipeline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    554us    229us    385us   2.31ms   9.02ms

  16829151 frames sent in 5.00s, 16806127 frames received
  Throughput: 3.36M req/s
  Bandwidth:  22.48MB/s
  WS upgrades: 1610
  WS frames:   16806127
  Latency samples: 16806030 / 16806127 responses (100.0%)
[info] CPU 5495.2% | Mem 202MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.20ms    209us    325us   2.65ms   12.60ms

  19710062 frames sent in 5.00s, 19652782 frames received
  Throughput: 3.93M req/s
  Bandwidth:  26.34MB/s
  WS upgrades: 4095
  WS frames:   19652782
  Latency samples: 19652674 / 19652782 responses (100.0%)
  Latency overflow (>5s): 2
[info] CPU 5872.7% | Mem 433MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    881us    211us    324us   2.54ms   11.60ms

  19806191 frames sent in 5.00s, 19749983 frames received
  Throughput: 3.95M req/s
  Bandwidth:  26.47MB/s
  WS upgrades: 3729
  WS frames:   19749983
  Latency samples: 19749983 / 19749983 responses (100.0%)
[info] CPU 5765.7% | Mem 591MiB

=== Best: 3949996 req/s (CPU: 5765.7%, Mem: 591MiB) ===
[info] saved results/echo-ws-pipeline/4096/simplew.json
httparena-bench-simplew
httparena-bench-simplew

==============================================
=== simplew / echo-ws-pipeline / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.65ms    252us    477us   3.75ms   430.10ms

  14946057 frames sent in 5.00s, 14894681 frames received
  Throughput: 2.98M req/s
  Bandwidth:  19.98MB/s
  WS upgrades: 3211
  WS frames:   14894681
  Latency samples: 14894681 / 14894681 responses (100.0%)
[info] CPU 5266.3% | Mem 334MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.66ms    218us    352us   3.51ms   254.30ms

  18201400 frames sent in 5.00s, 18106472 frames received
  Throughput: 3.62M req/s
  Bandwidth:  24.35MB/s
  WS upgrades: 5957
  WS frames:   18106472
  Latency samples: 18106472 / 18106472 responses (100.0%)
[info] CPU 5499.6% | Mem 627MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.85ms    214us    345us   3.25ms   310.00ms

  18112089 frames sent in 5.00s, 18023353 frames received
  Throughput: 3.60M req/s
  Bandwidth:  24.23MB/s
  WS upgrades: 5546
  WS frames:   18023353
  Latency samples: 18023353 / 18023353 responses (100.0%)
[info] CPU 5169.5% | Mem 847MiB

=== Best: 3621294 req/s (CPU: 5499.6%, Mem: 627MiB) ===
[info] saved results/echo-ws-pipeline/16384/simplew.json
httparena-bench-simplew
httparena-bench-simplew
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/crud-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MDA2AV MDA2AV Awaiting requested review from MDA2AV MDA2AV is a code owner

@Kaliumhexacyanoferrat Kaliumhexacyanoferrat Awaiting requested review from Kaliumhexacyanoferrat Kaliumhexacyanoferrat is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@stratdev3