Commit abf8cda
committed
feat(bench): implement iteration-level timing and add statistical formulas documentation
Changed from per-path timing to iteration-level timing to avoid mixing
path complexity variations in latency statistics. Each sample now represents
the average time per path for one complete iteration, providing more meaningful
performance variance analysis.
Key changes:
- Added sample_count field to LatencyStatistics to track iteration count
- Refactored timing to collect iteration_total_times and iteration_avg_times
- Each iteration times all paths together, then calculates per-path average
- For 100 paths × 10 iterations: now 10 samples (not 1000)
- Added comprehensive statistical formulas documentation printed to users
- Documents percentile calculation, consistency, variance, and stddev formulas
This prevents path-length variance from polluting execution variance statistics,
providing clearer insights into performance consistency.1 parent 1400822 commit abf8cda
1 file changed
+29
-7
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
| 54 | + | |
54 | 55 | | |
55 | 56 | | |
56 | 57 | | |
| |||
79 | 80 | | |
80 | 81 | | |
81 | 82 | | |
| 83 | + | |
| 84 | + | |
82 | 85 | | |
83 | 86 | | |
84 | 87 | | |
| |||
87 | 90 | | |
88 | 91 | | |
89 | 92 | | |
| 93 | + | |
90 | 94 | | |
91 | 95 | | |
92 | 96 | | |
| |||
125 | 129 | | |
126 | 130 | | |
127 | 131 | | |
| 132 | + | |
128 | 133 | | |
129 | 134 | | |
130 | 135 | | |
| |||
331 | 336 | | |
332 | 337 | | |
333 | 338 | | |
334 | | - | |
335 | | - | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
336 | 342 | | |
337 | 343 | | |
| 344 | + | |
338 | 345 | | |
339 | | - | |
340 | 346 | | |
341 | | - | |
342 | 347 | | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
343 | 354 | | |
344 | 355 | | |
345 | | - | |
346 | | - | |
| 356 | + | |
| 357 | + | |
347 | 358 | | |
348 | 359 | | |
349 | | - | |
| 360 | + | |
350 | 361 | | |
351 | 362 | | |
352 | 363 | | |
| |||
625 | 636 | | |
626 | 637 | | |
627 | 638 | | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
628 | 650 | | |
629 | 651 | | |
630 | 652 | | |
| |||
0 commit comments