feat(playback): burst-aware budgets — fix 8-cell cache-starvation freezes - #49
Merged
Conversation
…ezes The first 8-cell production run buffered ~9 min/2 hr (78 paused-for-cache freezes, 543 freeze-seconds), spread across ~15 files — systemic delivery starvation, not one bad asset and not decode (PERF stalls stayed <200ms). Root cause: 96.7% of plays go DIRECT (242 vs 8 transcode) since v10.3.0 dropped the resolution gate, so 8 cells pull raw high-bitrate files off greg at once, each with a 60s/1GB readahead. The fill-bursts starve each other's steady reads — 80% of freezes began within 8s of a stream-open. Two cell-count-aware budgets, both no-ops at <=4 cells (measured clean): - scale_readahead_s: demuxer_readahead 60s->30s at 8 cells (+cache_secs follows) — shrinks the per-open fill-burst. - scale_bitrate_budget_mbps: direct-play cap 60->~33 Mbps at 8 cells, so the rare 40-60+ Mbps outliers transcode (Emby throttles to ~realtime = smooth) while the library bulk stays direct. Explicit HYPERWALL_MAX_DIRECT_BITRATE_MBPS still wins verbatim. wall.py wires the resolved per-cell budget into the transcode heuristic and logs readahead + bitrate cap alongside the demuxer budget. Tests: 5 new (scale_readahead, graduated bitrate, env-override precedence, apply_cache_budget readahead scaling). Full suite green (v10.13.0). Note: scale_bitrate_budget_mbps assumes link_mbps=800 and caps per-file, not aggregate (8x33 ~= 264 Mbps sustained) — validate against greg's real disk/link ceiling before treating this as the complete fix. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jul 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The first 8-cell production run buffered heavily: 78
paused-for-cachefreezes / 543 freeze-seconds (~9 min) over a ~1h50m session, spread across ~15 different files. That signature is systemic delivery starvation — not one bad asset, and not decode (main-thread PERF stalls stayed <200ms all session; every prior 4-cell bench measured 0 drops / 0 stalls).Root cause: since v10.3.0 dropped the resolution gate, 96.7% of plays go DIRECT (242 direct vs 8 transcode). Eight cells therefore pull raw high-bitrate files straight off greg simultaneously, each with a 60s / 1 GB readahead. The per-open fill-bursts starve the other cells' steady reads — 80% of freezes began within 8s of a stream-open. This is the "8-stream concurrency ceiling" flagged as the tranche-2 target in SOAK #3.
Fix
Two cell-count-aware budgets, both no-ops at ≤4 cells (the configuration measured clean):
scale_readahead_s— demuxer readahead 60s → 30s at 8 cells (andcache_secsfollows), shrinking the per-open fill-burst so cells stop stepping on each other.scale_bitrate_budget_mbps— direct-play bitrate cap 60 → ~33 Mbps at 8 cells, so the rare 40–60+ Mbps outliers transcode server-side (Emby throttles a transcode to ~realtime = smooth by construction) while the library bulk stays direct. An explicitHYPERWALL_MAX_DIRECT_BITRATE_MBPSstill wins verbatim.wall.pyresolves the per-cell budget at grid setup, wires it into the transcode heuristic (urls.needs_transcode), and logs readahead + bitrate cap alongside the demuxer budget.Tests
5 new tests (readahead scaling, graduated bitrate, env-override precedence,
apply_cache_budgetreadahead scaling). Full suite green at v10.13.0 (version literal bumped in__init__.py+run_repo_guards.py::test_02).Caveats before treating this as the complete fix
scale_bitrate_budget_mbpsassumeslink_mbps=800— an assumption about the greg↔client path, not a measurement.link_mbpsaccordingly.Verify before trusting before/after numbers
Not yet built — the running artifact is still frozen v10.12.1. Rebuild (
build.ps1, pwsh 7) and confirm the log'sRuntime:banner reads v10.13.0 before A/B'ing; the stale-exe trap has caused false bug hunts here three times.🤖 Generated with Claude Code