Dc/exp/lkcapture by chenosaurus · Pull Request #1210 · livekit/rust-sdks

chenosaurus · 2026-07-01T15:42:43Z

Before you submit your PR

Make sure the following is true before submitting your PR:

I have read the contributing guidelines and validated that this PR will be accepted.
I have read and followed the principles regarding breaking changes, testing, and code quality.

PR description

Describe the changes in this PR. Explain what the PR is meant to solve and how to reproduce the issue in the first place.

Breaking changes

If this PR introduces breaking changes, list them here and document the rationale for introducing such a change.

MSRV

If the PR modifies the crate's MSRV (Minimum Supported Rust Version), document it here.

Testing

Ideally, unit test the code you add, but ensure you're not repeating existing test cases. Use as many already written scaffolding, utilities as possible; write your own, when needed. If external services, APIs, tokens are required (e.g., running an LK server instance), provide the necessary information. Make sure your tests perform useful, context-aware assertions and do not simply emulate "happy paths".

Async

We want the project to be runtime-agnostic, so please reuse what's already in livekit-runtime and feel free to add anything missing. It's ok to use Tokio directly, when writing unit tests, if necessary. When testing, do not use artificial delays for the state to "catch up"; instead, respect the event flow and subscribe properly using channels or other mechanisms.

…ture # Conflicts: # Cargo.lock # Cargo.toml # examples/local_video/Cargo.toml

…urn-timestamp

ladvoc

Left some initial comments on the capture components, off to a great start!

Maybe we should break out the changes to libwebrtc and webrtc-sys into their own PRs?

ladvoc · 2026-07-02T17:10:17Z

+fn main() {
+    println!("cargo:rustc-check-cfg=cfg(livekit_capture_argus)");
+    println!("cargo:rerun-if-env-changed=JETSON_MULTIMEDIA_API_DIR");
+


suggestion: This might need to explicitly link Objective-C for Apple platforms when this is compiled outside of the workspace:

Suggested change

match std::env::var("CARGO_CFG_TARGET_OS").as_deref() {

Ok("macos") | Ok("ios") => println!("cargo:rustc-link-arg=-ObjC"),

_ => {}

}

ladvoc · 2026-07-02T17:12:48Z

+// limitations under the License.
+
+//! Capture helpers for publishing decoded, DMA-BUF, and encoded video with LiveKit.
+


suggestion: Add contents of README as module level doc comment so it appears in generated docs:

Suggested change

#![doc = include_str!("../README.md")]

ladvoc · 2026-07-02T17:16:33Z

+pub(crate) mod time;
+pub mod track;
+
+pub use device::{


suggestion: Instead of exporting everything at the top-level, consider exporting a prelude module that includes the types most consumers will be interested in that they can import as a wildcard. This pattern is used in the livekit crate currently.

ladvoc · 2026-07-02T17:19:50Z

+
+## Pre-encoded test sources
+
+The example ships GStreamer fixture scripts that exercise the H.264, H.265,


suggestion: This should link to the example in GitHub since it wouldn't be clear to someone discovering this crate through crates.io which example it is referring to.

ladvoc · 2026-07-02T17:26:58Z

    }
 }

+fn stats_poll_interval() -> Duration {


nitpick: Define this as a const.

ladvoc · 2026-07-02T18:30:42Z

+
+impl CaptureFormat {
+    /// Creates a raw-frame capture format.
+    pub const fn new(


suggestion: This should be removed since this struct has public fields and nothing is computed here.

ladvoc · 2026-07-02T18:36:22Z

+
+/// H.264 packetization mode for passthrough metadata.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum H264PacketizationMode {


suggestion: It looks like it might be necessary to add an interleaved case in the future? If so, this should be marked #[non_exhaustive].

ladvoc · 2026-07-02T18:45:38Z

+        let mut scratch = Vec::new();
+        let payload: &[u8] = match &access_unit.payload {
+            EncodedPayload::Contiguous(bytes) => bytes,
+            EncodedPayload::Owned(bytes) => bytes,
+            EncodedPayload::Fragments(_) => {
+                scratch = access_unit.payload.to_vec();
+                &scratch
+            }
+        };
+        let frame = EncodedVideoFrame {
+            codec: access_unit.codec.into(),
+            payload,
+            timestamp_us: access_unit.timestamp_us,
+            frame_type: access_unit.frame_type.into(),
+            width: access_unit.width,
+            height: access_unit.height,
+            frame_metadata: None,
+        };


suggestion: Consider rewriting using Cow:

Suggested change

let mut scratch = Vec::new();

let payload: &[u8] = match &access_unit.payload {

EncodedPayload::Contiguous(bytes) => bytes,

EncodedPayload::Owned(bytes) => bytes,

EncodedPayload::Fragments(_) => {

scratch = access_unit.payload.to_vec();

&scratch

}

};

let frame = EncodedVideoFrame {

codec: access_unit.codec.into(),

payload,

timestamp_us: access_unit.timestamp_us,

frame_type: access_unit.frame_type.into(),

width: access_unit.width,

height: access_unit.height,

frame_metadata: None,

};

let payload: Cow<'_, [u8]> = match &access_unit.payload {

EncodedPayload::Contiguous(bytes) => Cow::Borrowed(bytes),

EncodedPayload::Owned(bytes) => Cow::Borrowed(bytes),

EncodedPayload::Fragments(_) => Cow::Owned(access_unit.payload.to_vec()),

};

let frame = EncodedVideoFrame {

codec: access_unit.codec.into(),

payload: &payload,

timestamp_us: access_unit.timestamp_us,

frame_type: access_unit.frame_type.into(),

width: access_unit.width,

height: access_unit.height,

frame_metadata: None,

};

ladvoc · 2026-07-02T18:50:36Z

+    let shutdown_stream = stream.try_clone().context("failed to clone TCP stream")?;
+    let source = TcpEncodedSource::from_tcp_stream(stream, config)?;
+
+    publish_encoded_source(


suggestion: This is hard to read at the call site; consider breaking out args into a separate struct (i.e., PublishEncodedSourceOptions).

ladvoc · 2026-07-02T19:05:59Z

+    );
+
+    log::info!("Connecting to TCP {wire_format:?} encoded stream at {host}");
+    let stream = TcpStream::connect(&host)


issue: This blocks the async runtime; use the async variant of connect provided by tokio and allow the reset of the connection to be sync:

let stream = tokio::time::timeout(TCP_CONNECT_TIMEOUT, tokio::net::TcpStream::connect(&host)) .await??; .with_context(|| format!("timed out connecting to TCP source at {host}"))? .with_context(|| format!("failed to connect to TCP source at {host}"))?; let stream = stream.into_std().context("failed to convert TCP stream to blocking")?; stream.set_nonblocking(false).context("failed to set TCP stream to blocking mode")?;

chenosaurus added 25 commits June 22, 2026 21:45

adding initial livekit-capture crate

4113717

refactor livekit-capture

fda0671

add avfoundation & pre-encoded support

1c2fb9d

adding v4l support

c32c7cb

avfoundation implementation

a4c78d4

fix avfoundation

6e9973f

rename for clarity

a6cca8a

implement rtsp & tcp ingest path

26bb266

dont touch frame metadata from capture crate

22d567e

Merge branch 'main' of github.com:livekit/rust-sdks into dc/exp/lkcap…

856604a

…ture # Conflicts: # Cargo.lock # Cargo.toml # examples/local_video/Cargo.toml

reduce getStat calls

e064490

add animated test pattern

86e407c

implement zero copy encode path for avfoundation

fde64ee

remove nokhwa dep

80fae4a

add support for GREY cameras

d628101

add gstreamer app sink

4dc8f54

libargus capture

3e04e3b

support h264 avc

6f278bc

fix: add numFilled to argus session

9d6dcab

prefer zero copy on platforms that support it, warn when using w/ --b…

6e792d7

…urn-timestamp

add vp8 vp9 av1 passthru mode

e451820

add scripts to run gstreamer pipelines for testing

97afedf

update readme

ba28354

cleanup

566b9cc

add missing files

7708627

ladvoc reviewed Jul 2, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Dc/exp/lkcapture#1210

Dc/exp/lkcapture#1210
chenosaurus wants to merge 25 commits into
mainfrom
dc/exp/lkcapture

chenosaurus commented Jul 1, 2026

Uh oh!

ladvoc left a comment

Uh oh!

ladvoc Jul 2, 2026

Uh oh!

ladvoc Jul 2, 2026

Uh oh!

ladvoc Jul 2, 2026

Uh oh!

ladvoc Jul 2, 2026

Uh oh!

ladvoc Jul 2, 2026

Uh oh!

ladvoc Jul 2, 2026

Uh oh!

ladvoc Jul 2, 2026

Uh oh!

ladvoc Jul 2, 2026

Uh oh!

ladvoc Jul 2, 2026

Uh oh!

ladvoc Jul 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

+match std::env::var("CARGO_CFG_TARGET_OS").as_deref() {
+    Ok("macos") | Ok("ios") => println!("cargo:rustc-link-arg=-ObjC"),
+    _ => {}
+}

		// limitations under the License.

		//! Capture helpers for publishing decoded, DMA-BUF, and encoded video with LiveKit.


		## Pre-encoded test sources

		The example ships GStreamer fixture scripts that exercise the H.264, H.265,

Uh oh!

Conversation

chenosaurus commented Jul 1, 2026

Before you submit your PR

PR description

Breaking changes

MSRV

Testing

Async

Uh oh!

ladvoc left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants