Skip to content

[C#] GenHTTP: Test effect of backlog#890

Closed
Kaliumhexacyanoferrat wants to merge 2 commits into
MDA2AV:mainfrom
Kaliumhexacyanoferrat:backlog-tests
Closed

[C#] GenHTTP: Test effect of backlog#890
Kaliumhexacyanoferrat wants to merge 2 commits into
MDA2AV:mainfrom
Kaliumhexacyanoferrat:backlog-tests

Conversation

@Kaliumhexacyanoferrat

Copy link
Copy Markdown
Collaborator

Test PR to compare the effect of different backlog settings. Currently defaults to 1024.

Do not merge.

@Kaliumhexacyanoferrat Kaliumhexacyanoferrat self-assigned this Jun 19, 2026
@Kaliumhexacyanoferrat

Copy link
Copy Markdown
Collaborator Author

/benchmark -f genhttp-11

@github-actions

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: genhttp-11 | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 2,092,965 5798.1% 272MiB +0.8% +5.4%
baseline 4096 2,026,585 5841.0% 536MiB +5.1% +2.7%
pipelined 512 17,782,758 6444.5% 154MiB -0.6% +4.8%
pipelined 4096 16,874,355 6401.8% 262MiB +0.5% +0.8%
limited-conn 512 1,148,231 5618.9% 251MiB +5.0% -1.2%
limited-conn 4096 1,179,076 5545.6% 382MiB +8.1% +4.7%
json 4096 850,787 6037.8% 605MiB +1.9% +4.0%
json-comp 512 466,565 5720.9% 302MiB +2.4% +2.0%
json-comp 4096 467,158 6079.5% 624MiB +1.0% -1.4%
json-comp 16384 459,567 5985.3% 1.2GiB -0.2% -7.7%
json-tls 4096 698,191 6206.4% 1.0GiB +2.3% +1.2%
upload 32 2,596 2505.4% 188MiB +0.1% -2.1%
upload 256 2,910 3843.5% 217MiB -0.8% -6.5%
api-4 256 40,667 394.7% 224MiB -23.8% -2.6%
api-16 1024 150,350 1571.1% 358MiB +0.4% -2.5%
static 1024 901,836 5788.2% 267MiB -1.4% -15.5%
static 4096 843,843 6064.2% 549MiB +10.1% +18.1%
static 6800 804,583 6141.0% 775MiB +0.9% +18.5%
async-db 1024 197,492 3248.1% 419MiB +0.5% ~0%
crud 4096 530,277 4184.7% 760MiB +2.3% ~0%
echo-ws 512 2,391,191 5913.7% 267MiB +3.4% -7.3%
echo-ws 4096 2,204,010 5453.3% 778MiB -0.6% +2.4%
echo-ws 16384 2,112,944 5648.8% 1.7GiB +3.8% +6.2%
echo-ws-pipeline 512 28,395,686 5554.3% 301MiB +1.7% +4.2%
echo-ws-pipeline 4096 29,516,067 5863.7% 1.2GiB +3.1% +30.0%
echo-ws-pipeline 16384 27,219,734 5774.0% 1.9GiB -1.7% +18.7%
Full log
  Throughput: 28.39M req/s
  Bandwidth:  189.51MB/s
  WS upgrades: 512
  WS frames:   141978432
  Latency samples: 141977776 / 141978431 responses (100.0%)

  WARNING: 18446744073709551615/141978431 responses (12992638349207.8%) had unexpected status (expected 2xx)
[info] CPU 5554.3% | Mem 301MiB

=== Best: 28395686 req/s (CPU: 5554.3%, Mem: 301MiB) ===
[info] saved results/echo-ws-pipeline/512/genhttp-11.json
httparena-bench-genhttp-11
httparena-bench-genhttp-11

==============================================
=== genhttp-11 / echo-ws-pipeline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   2.73ms   2.04ms   5.02ms   11.50ms   19.90ms

  99567856 frames sent in 5.00s, 99507664 frames received
  Throughput: 19.90M req/s
  Bandwidth:  132.96MB/s
  WS upgrades: 4096
  WS frames:   99507664
  Latency samples: 99507616 / 99507664 responses (100.0%)
[info] CPU 5489.3% | Mem 643MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.99ms   1.76ms   3.74ms   7.28ms   15.50ms

  144578592 frames sent in 5.00s, 144536288 frames received
  Throughput: 28.90M req/s
  Bandwidth:  193.02MB/s
  WS upgrades: 4088
  WS frames:   144536288
  Latency samples: 144535664 / 144536288 responses (100.0%)
[info] CPU 5953.3% | Mem 1.1GiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.99ms   1.79ms   3.54ms   7.01ms   11.70ms

  147619200 frames sent in 5.00s, 147580336 frames received
  Throughput: 29.51M req/s
  Bandwidth:  197.08MB/s
  WS upgrades: 4095
  WS frames:   147580336
  Latency samples: 147580016 / 147580336 responses (100.0%)
[info] CPU 5863.7% | Mem 1.2GiB

=== Best: 29516067 req/s (CPU: 5863.7%, Mem: 1.2GiB) ===
[info] saved results/echo-ws-pipeline/4096/genhttp-11.json
httparena-bench-genhttp-11
httparena-bench-genhttp-11

==============================================
=== genhttp-11 / echo-ws-pipeline / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   5.42ms   4.43ms   9.61ms   20.40ms   31.60ms

  84439568 frames sent in 5.00s, 84288208 frames received
  Throughput: 16.85M req/s
  Bandwidth:  112.86MB/s
  WS upgrades: 9460
  WS frames:   84288208
  Latency samples: 84288208 / 84288208 responses (100.0%)
[info] CPU 5360.3% | Mem 1.1GiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   5.78ms   5.64ms   9.59ms   16.60ms   41.90ms

  134541520 frames sent in 5.00s, 134328288 frames received
  Throughput: 26.86M req/s
  Bandwidth:  179.77MB/s
  WS upgrades: 13327
  WS frames:   134328288
  Latency samples: 134328288 / 134328288 responses (100.0%)
[info] CPU 6016.7% | Mem 1.5GiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   5.40ms   5.17ms   9.31ms   15.70ms   27.10ms

  136301216 frames sent in 5.00s, 136098672 frames received
  Throughput: 27.21M req/s
  Bandwidth:  182.13MB/s
  WS upgrades: 12659
  WS frames:   136098672
  Latency samples: 136098672 / 136098672 responses (100.0%)
[info] CPU 5774.0% | Mem 1.9GiB

=== Best: 27219734 req/s (CPU: 5774.0%, Mem: 1.9GiB) ===
[info] saved results/echo-ws-pipeline/16384/genhttp-11.json
httparena-bench-genhttp-11
httparena-bench-genhttp-11
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/crud-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536

@Kaliumhexacyanoferrat

Copy link
Copy Markdown
Collaborator Author

/benchmark -f genhttp-11

@github-actions

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: genhttp-11 | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 1,939,875 5625.8% 254MiB -6.5% -1.6%
baseline 4096 2,011,986 5987.2% 503MiB +4.4% -3.6%
pipelined 512 17,868,249 6591.8% 154MiB ~0% +4.8%
pipelined 4096 16,805,417 6418.5% 260MiB +0.1% ~0%
limited-conn 512 1,120,421 5630.4% 230MiB +2.4% -9.4%
limited-conn 4096 1,120,293 5594.5% 471MiB +2.7% +29.0%
json 4096 835,941 6057.5% 699MiB +0.2% +20.1%
json-comp 512 459,930 5740.4% 317MiB +1.0% +7.1%
json-comp 4096 470,860 6051.6% 697MiB +1.8% +10.1%
json-comp 16384 443,249 5948.7% 1.3GiB -3.7% ~0%
json-tls 4096 674,931 6085.5% 997MiB -1.1% -1.5%
upload 32 2,596 2487.9% 187MiB +0.1% -2.6%
upload 256 2,967 3919.4% 222MiB +1.2% -4.3%
api-4 256 41,048 393.7% 225MiB -23.1% -2.2%
api-16 1024 148,689 1567.9% 361MiB -0.7% -1.6%
static 1024 889,978 5648.3% 228MiB -2.7% -27.8%
static 4096 759,086 5445.3% 508MiB -1.0% +9.2%
static 6800 739,539 6045.8% 711MiB -7.2% +8.7%
async-db 1024 194,922 3265.5% 423MiB -0.9% +1.0%
crud 4096 522,501 4137.5% 794MiB +0.8% +4.5%
echo-ws 512 2,370,606 5616.1% 371MiB +2.5% +28.8%
echo-ws 4096 2,215,561 5746.4% 919MiB -0.1% +20.9%
echo-ws 16384 2,027,105 5902.1% 2.1GiB -0.4% +31.2%
echo-ws-pipeline 512 28,543,929 5550.8% 304MiB +2.3% +5.2%
echo-ws-pipeline 4096 27,718,583 5755.0% 842MiB -3.2% -10.9%
echo-ws-pipeline 16384 26,534,918 6002.6% 2.6GiB -4.2% +62.5%
Full log
  WS upgrades: 512
  WS frames:   141529423
  Latency samples: 141529024 / 141529423 responses (100.0%)
[info] CPU 5565.6% | Mem 308MiB

=== Best: 28543929 req/s (CPU: 5550.8%, Mem: 304MiB) ===
[info] saved results/echo-ws-pipeline/512/genhttp-11.json
httparena-bench-genhttp-11
httparena-bench-genhttp-11

==============================================
=== genhttp-11 / echo-ws-pipeline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   2.97ms   2.10ms   5.84ms   14.40ms   24.30ms

  97551184 frames sent in 5.00s, 97486896 frames received
  Throughput: 19.49M req/s
  Bandwidth:  130.27MB/s
  WS upgrades: 4096
  WS frames:   97486896
  Latency samples: 97486880 / 97486896 responses (100.0%)
[info] CPU 5352.8% | Mem 584MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   2.35ms   1.88ms   4.57ms   8.81ms   15.90ms

  135142977 frames sent in 5.00s, 135083319 frames received
  Throughput: 27.01M req/s
  Bandwidth:  180.44MB/s
  WS upgrades: 4096
  WS frames:   135083314
  Latency samples: 135082865 / 135083319 responses (100.0%)

  WARNING: 5/135083319 responses (0.0%) had unexpected status (expected 2xx)
[info] CPU 5892.6% | Mem 803MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   2.34ms   1.99ms   4.16ms   8.68ms   17.00ms

  138646016 frames sent in 5.00s, 138592916 frames received
  Throughput: 27.71M req/s
  Bandwidth:  185.11MB/s
  WS upgrades: 4095
  WS frames:   138603616
  Latency samples: 138605184 / 138592916 responses (100.0%)

  WARNING: 18446744073709540916/138592916 responses (13310019448403.5%) had unexpected status (expected 2xx)
[info] CPU 5755.0% | Mem 842MiB

=== Best: 27718583 req/s (CPU: 5755.0%, Mem: 842MiB) ===
[info] saved results/echo-ws-pipeline/4096/genhttp-11.json
httparena-bench-genhttp-11
httparena-bench-genhttp-11

==============================================
=== genhttp-11 / echo-ws-pipeline / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   11.74ms   9.28ms   21.40ms   43.50ms   71.10ms

  80568608 frames sent in 5.00s, 80306464 frames received
  Throughput: 16.06M req/s
  Bandwidth:  107.79MB/s
  WS upgrades: 16384
  WS frames:   80306464
  Latency samples: 80306464 / 80306464 responses (100.0%)
[info] CPU 5242.9% | Mem 1.6GiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   8.25ms   8.31ms   13.30ms   20.90ms   40.40ms

  132849232 frames sent in 5.00s, 132587088 frames received
  Throughput: 26.51M req/s
  Bandwidth:  177.58MB/s
  WS upgrades: 16384
  WS frames:   132587088
  Latency samples: 132587088 / 132587088 responses (100.0%)
[info] CPU 6230.0% | Mem 2.3GiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   8.31ms   8.40ms   13.00ms   20.70ms   33.20ms

  132936736 frames sent in 5.00s, 132674592 frames received
  Throughput: 26.52M req/s
  Bandwidth:  177.67MB/s
  WS upgrades: 16384
  WS frames:   132674592
  Latency samples: 132674592 / 132674592 responses (100.0%)
[info] CPU 6002.6% | Mem 2.6GiB

=== Best: 26534918 req/s (CPU: 6002.6%, Mem: 2.6GiB) ===
[info] saved results/echo-ws-pipeline/16384/genhttp-11.json
httparena-bench-genhttp-11
httparena-bench-genhttp-11
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/crud-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536

@Kaliumhexacyanoferrat

Copy link
Copy Markdown
Collaborator Author

So basically no effect at all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant