Minimal set of changes for allowing faster than realtime recording (non live)#1192
Merged
Conversation
…mixing, different feeding tempo
…add a bool method, linter fixes
biglittlebigben
approved these changes
Apr 17, 2026
| // the source is constructed externally (e.g. TestSource for non-live pipeline | ||
| // testing, or ReplaySource for offline export). The source must have already | ||
| // populated the PipelineConfig with track information before calling this. | ||
| func NewWithSource(ctx context.Context, conf *config.PipelineConfig, src source.Source) (*Controller, error) { |
Contributor
There was a problem hiding this comment.
Can New be implemented, using NewWithSource? There is quite a bit of code duplication currently.
Contributor
Author
There was a problem hiding this comment.
yes - the only reason they got duplicated is because source creation needed controller callbacks for New - but if we apply inversion control that's no longer an issue and these could be compacted. Updated.
frostbyte73
approved these changes
Apr 20, 2026
Merged
milos-lk
added a commit
that referenced
this pull request
May 28, 2026
## Changelog ### Added - V2 egress API — unified source and output config (StartEgressRequest, MediaSource, Output, StorageConfig), per-participant audio channel routing (AudioRoute), and request-level storage overrides (#1155) - Support for egress auto retry (#1138) - Faster than realtime recording (non-live) (#1192) - Cgroup-aware memory monitoring for admission control and OOM kill (#1118) - Support for mulaw and alaw input codecs (#1105) - Add handler outcome Prometheus metric for SLO-based kill rate alerting (#1230) - Add livekit_load_ratio metric for composite load-based autoscaling (#1234) - Better multi-publisher support (#1214) - Read K8s CPU requests to automatically set GOMAXPROCS for cgroup-aware scheduling (#1204) - Backup storage observability — including storage event IPC for tracking uploads across primary/backup stores (#1120, #1184) - Enable one-shot sender report sync mode for room composite behind config (#1158) - Export InitLogger with configurable service name (#1218) - Allow specifying affinity timeout (#1104) - Instrument all leaky queues (#1116) - Instrument data loss on video leaky queues (#1109) - Add a 1G memory buffer for accepting requests (#1088) - Log rss periodically (#1151) - Log pipeline time to playing (#1103) - Log file upload stats (size, duration) (#1096) - Chrome 146 (#1173) ### Fixed - Fix s3-compat multi-part uploads (#1228) - Fix deadlock when AbortProcess/KillProcess call kill() under pm.mu lock (#1226) - Use statistical cadence check to tolerate WebRTC NetEQ time-stretching (#1223) - Fix MP3 duration metadata and CBR encoding (#1187) - Fix room composite duration to include post-participant silence tail (#1169) - Fix awaitMediaTracks race (#1165) - Fix for x264 encoder errors causing egress failures at the end of the recording (#1095) - Fix for data race inside monitor (#1091) - Fixing one of the causes of pipeline frozen errors (#1129) - Potential fix for rare issue causing tracks not to be recorded (#1130) - Handle EOS when removing source bin (#1093) - Drain appwriter (#1090) - Drain audio tracks before removing their appsource (#1085) - Heal track which enters into continuous flow flushing loop on pushing packets (#1142) - One pacer per audio track (#1235) - Set time provider only after pipeline reaches playing state (#1212) - AbortProcess is not safe to be executed by multiple goroutines (#1208) - Fixing unprotected state write from timer goroutine (#1206) - Disable PTS adjustments on sender reports for track egresses (#1134) - Disconnect from room on failing to await for some track (#1132) - Retry chrome egress navigation on chrome cert verifier change (#1194) - Return a 500 if handler fails to start (#1137) - Make sure all data is read from LK server (#1111) - Always check video dimensions (#1119) - Fail web egress on HTTP 4xx/5xx page load errors (#1106) - Avoid panic if we fail to parse a gst pipeline error (#1086) - Address unsafe int casting (#1126) - Get original room name from Start request (#1189) - Suppress noisy colorimetry warnings (#1163) - Don't log error on manifest upload failure if backup storage wasn't used (#1152) - Don't count room composite SDK source against Pulse limits (#1114) - Use a 10 min deadline for the RPC watchdog (#1207) - Remove enable_room_composite_sdk_source and always enable sdk source when conditions allow it (#1122) - Gstreamer logs based on configured level (#1161) - Reintroduce sdk logs filtering (#1213) - Using variant of OnDisconnected callback which passes a reason (#1221) - AV sync content verification in integration tests (#1215) - Fixing arm64 chrome installer (#1175) - Use separate token with delete permission (#1172) - Update module go.opentelemetry.io/otel to v1.41.0 [SECURITY] (#1201) - Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.97.3 [SECURITY] (#1178) - Update module github.com/go-jose/go-jose/v4 to v4.1.4 [SECURITY] (#1170) - Update module google.golang.org/grpc to v1.79.3 [SECURITY] (#1153) - Rename IOClient to SessionReporter (#1097) - Move ProcessManager to interface and create a fake implementation (#1147) - Enable staticcheck (#1094)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This branch generalizes the gstreamer pipeline to support non-live sources (replay/export) in addition to the existing live capture flow. Historically the pipeline assumed every source produced data in real time, which baked several implicit behaviors into the build: leaky queues to shed overruns,
is-live=trueappsrc properties, an asynchronous file sink, and a singleNew()constructor that owned source creation. Those assumptions break down the moment a source pushes buffers faster than wall-clock, so the changes here rework the pipeline to be driven by an explicit Live flag onPipelineConfiginstead.The centerpiece is the new
Livebool field, set to true by default in both pipeline config constructors. The pre-existingIsReplayboolean is replaced with an IsReplay() method derived from!Live, which keeps replay-specific integration points (IPC calls into the service, storage access) working without conflating them with generic pipeline concerns like queue leakiness or backpressure.Downstream of that flag, audio and video bins now build blocking queues when Live is false so backpressure actually throttles the source, the appsrc elements toggle
is-liveto match the pipeline mode and addblock=truein non-live mode, and the file sink setsasync=falseto disable preroll for non-live runs — there's no real-time clock to sync against, so making the sink wait for PAUSED→PLAYING preroll-complete serves no purpose and just stalls the state transition . The previously namedbuildLeakyVideoQueueis renamed tobuildVideoQueueto reflect that leakiness is now a property of the mode, not the call site.On the output side, updateOutputs rejects stream outputs (RTMP, WebSocket) for non-live pipelines upfront, since those destinations cannot ingest faster than 1x playback speed and would otherwise fail deep inside the pipeline.
The controller is refactored to decouple source construction from pipeline construction. The existing
New()keeps its behavior, but the shared setup is extracted into newController() and a newNewWithSource()constructor accepts a pre-built source which is an interface, implementors might choose to build for different kind of sources.Original PR draft with test sources feeding data as fast as possible: #1182