rmp3: add a streaming API - #19317
Merged
Merged
Conversation
rmp3_init_memory takes a borrowed pointer to the whole file and seeks by re-scanning it from the front, so a caller must hold the entire MP3 to decode any of it. A caller reading from storage has no such buffer, and building one costs the file in memory to play a few kilobytes at a time. MPEG audio needs far less. A frame is self-contained apart from the bit reservoir, which the decoder already carries between frames, so what a decode needs resident is one frame - a couple of kilobytes at the extremes of the format. What was missing was a way to hand frames over as they arrive rather than pointing at all of them at once. Nothing here decodes. rmp3dec_decode_frame already took a buffer and a length and told the caller what it consumed; this wraps it with the window bookkeeping that turns that into a stream. Same shape as rvorbis and rflac: set_in, set_out, process. Input is consumed rather than borrowed, so a window may slide freely, and a frame split across two windows is reassembled internally rather than asking the caller for an alignment it has no way to guarantee. Three things the format forces. Handed fewer bytes than a frame occupies, the frame finder reports what it scanned as bytes to skip: it cannot tell the head of a frame that has not all arrived from junk, and acting on that discards the frame. So a partly-filled hold is never decoded - which leaves the tail of a stream waiting for a window that will never fill, and rmp3_stream_set_eof is what says the short tail is all there is. A stream states no length. Short of a Xing header the only way to know is to walk the frames, so setting no output does exactly that: frames are located and counted, nothing is decoded. In that mode process() returns per frame, so rmp3_stream_frame_offset names the frame just counted and a caller can build a seek index as it walks. And not every frame decodes to samples. Layer III puts part of a frame's data in the frames before it, so a decoder that joined the stream partway meets frames whose data begins before where it started, and those produce nothing. How many is a property of how the encoder spent its bits - one frame in some places, three in others - so a caller resuming at a known frame cannot assume its position advances one frame per emission. rmp3_stream_frames_in counts frames consumed whether they emitted or not, which is what states where the stream actually stands. A located frame fills in the header fields whether or not it went on to produce samples, and junk merely scanned past leaves them clear; that is what separates the two. Verified against the resident decoder as oracle on 19 real CD audio tracks: bit-exact at input windows from 512 bytes up, and at output chunk sizes from 4 bytes to 256KB. Stream length agrees with an independent frame walk of the same files. ASan, UBSan and LeakSan clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
rmp3_init_memory takes a borrowed pointer to the whole file and seeks by re-scanning it from the front, so a caller must hold the entire MP3 to decode any of it. A caller reading from storage has no such buffer, and building one costs the file in memory to play a few kilobytes at a time.
MPEG audio needs far less. A frame is self-contained apart from the bit reservoir, which the decoder already carries between frames, so what a decode needs resident is one frame - a couple of kilobytes at the extremes of the format. What was missing was a way to hand frames over as they arrive rather than pointing at all of them at once.
Nothing here decodes. rmp3dec_decode_frame already took a buffer and a length and told the caller what it consumed; this wraps it with the window bookkeeping that turns that into a stream. Same shape as rvorbis and rflac: set_in, set_out, process. Input is consumed rather than borrowed, so a window may slide freely, and a frame split across two windows is reassembled internally rather than asking the caller for an alignment it has no way to guarantee.
Three things the format forces.
Handed fewer bytes than a frame occupies, the frame finder reports what it scanned as bytes to skip: it cannot tell the head of a frame that has not all arrived from junk, and acting on that discards the frame. So a partly-filled hold is never decoded - which leaves the tail of a stream waiting for a window that will never fill, and rmp3_stream_set_eof is what says the short tail is all there is.
A stream states no length. Short of a Xing header the only way to know is to walk the frames, so setting no output does exactly that: frames are located and counted, nothing is decoded. In that mode process() returns per frame, so rmp3_stream_frame_offset names the frame just counted and a caller can build a seek index as it walks.
And not every frame decodes to samples. Layer III puts part of a frame's data in the frames before it, so a decoder that joined the stream partway meets frames whose data begins before where it started, and those produce nothing. How many is a property of how the encoder spent its bits - one frame in some places, three in others - so a caller resuming at a known frame cannot assume its position advances one frame per emission. rmp3_stream_frames_in counts frames consumed whether they emitted or not, which is what states where the stream actually stands. A located frame fills in the header fields whether or not it went on to produce samples, and junk merely scanned past leaves them clear; that is what separates the two.
Verified against the resident decoder as oracle on 19 real CD audio tracks: bit-exact at input windows from 512 bytes up, and at output chunk sizes from 4 bytes to 256KB. Stream length agrees with an independent frame walk of the same files. ASan, UBSan and LeakSan clean.
Guidelines
C89_BUILD=1Description
[Description of the pull request, detail any issues you are fixing or any features you are implementing]
Related Issues
[Any issues this pull request may be addressing]
Related Pull Requests
[Any other PRs from related repositories that might be needed for this pull request to work]
Reviewers
[If possible @mention all the people that should review your pull request]