Add libplacebo GPU module with render and shader filters by ddennedy · Pull Request #1236 · mltframework/mlt

ddennedy · 2026-05-06T05:30:07Z

supersedes #1201

New module 'placebo' providing GPU-accelerated video processing via libplacebo. Includes two filters: - placebo.render: GPU scaling, debanding, dithering, and tonemapping with quality presets (fast/default/high_quality) - placebo.shader: Custom mpv-compatible .hook shader support Backend priority: D3D11 (Windows) -> Vulkan -> OpenGL. Vulkan loader is dynamically loaded on Windows when libplacebo is built without vk-proc-addr support. Features: - Singleton GPU context with thread-safe access - Shader cache persistence - Multiple scaling algorithms (ewa_lanczos, lanczos, mitchell, etc.) - Tone mapping (auto, clip, mobius, reinhard, hable, bt.2390, spline) - Graceful fallback to passthrough when no GPU is available The module is enabled by default but skipped automatically when libplacebo is not installed.

Use PRIu64/PRId64 from <inttypes.h> instead of %zu/%ld for size logging in the placebo module. Add libplacebo-dev packages to Ubuntu, Debian, and Fedora 42 CI workflows, and mingw-w64-x86_64-libplacebo to the MSYS2 MinGW64 workflow.

Break long mlt_log_info() call into multi-line format to match the project's clang-format rules (same style as load_cache above).

When multiple placebo filters are stacked on one clip, each filter previously did a full RAM→GPU upload and GPU→RAM download. The intermediate uploads are redundant because the next placebo filter would re-upload the same pixels immediately. Each filter now attaches its output texture to the mlt_frame via placebo_frame_put_tex(). The next placebo filter calls placebo_frame_take_tex() to grab it directly as source, skipping the upload. The download to RAM still happens every time (MLT expects the image buffer to be current for non-GPU filters). Staleness detection: put_tex records the RAM buffer pointer, take_tex compares it against the current pointer. If a CPU filter ran in between and requested a writable buffer (triggering a copy and new allocation), the pointers differ and take_tex returns NULL, falling back to a fresh upload. Also cleans up internal ticket-style comments (C1/W2/etc.) with descriptions of actual logic and pitfalls.

Add apply_shader_params() to override pl_hook DYNAMIC parameters from MLT animated properties (shader_param.* prefix) on every frame. Uses mlt_properties_anim_get_double/int to correctly resolve keyframe strings ("0=200;50=100") at the current frame position. Add base64 decoding for shader_text values prefixed with "base64:" to support inline shaders with characters that are problematic in MLT property strings.

Run clang-format-14 (matching CI) on filter_placebo_shader.c and gpu_context.c to fix designated initializer spacing, ternary line breaks, and long argument lists.

…into libplacebo

- Introduced new `mlt_image_private` format for module internal use. - Improved placebo texture reuse across filters.

ddennedy · 2026-05-06T20:42:35Z

@copilot resume review

- Remove incorrect pl_tex_destroy(src_tex) in error paths of the render and shader filters: src_tex is owned by the frame and freed automatically when placebo_frame_set_tex() replaces the "image" property, so manual destruction caused a double-free on frame close. - Add *image = NULL after failed placebo_frame_set_tex() in render and shader filters (consistent with filter_placebo_convert.c). - Guard filter_imageconvert's conversion_matrix access: mlt_image_private (and mlt_image_invalid) are out of range for the matrix indexed by [format - 1]; early-return 1 for any format outside [mlt_image_rgb, mlt_image_rgba64] to avoid an out-of-bounds read. Agent-Logs-Url: https://github.com/mltframework/mlt/sessions/f71fd929-5b62-4e71-864a-13291a3ec28e Co-authored-by: ddennedy <1146683+ddennedy@users.noreply.github.com>

Copilot · 2026-05-06T20:53:40Z

@copilot resume review

Resumed review and found three issues in the placebo module and related code, fixed in commit 739224f:

Double-free of src_tex (filter_placebo_render.c and filter_placebo_shader.c): when pl_tex_create failed for the destination texture, the code called pl_tex_destroy(gpu, &src_tex) — but src_tex is owned by the frame (registered via placebo_frame_set_tex upstream) and freed automatically when placebo_frame_set_tex replaces the "image" property. Manual destruction caused a double-free when the frame closed.
Dangling *image pointer (same files): when placebo_frame_set_tex failed after *image = dst_tex was set and dst_tex was manually freed, *image was left pointing to freed memory. Added *image = NULL (consistent with filter_placebo_convert.c).
Out-of-bounds array access (filter_imageconvert.c): conversion_matrix[*format - 1] is sized for formats mlt_image_rgb…mlt_image_rgba64 (indices 0–9). Now that mlt_image_private (12) can flow through the converter chain if the placebo converter fails, accessing index 11 was an out-of-bounds read. Added a bounds guard that returns 1 (not handled) for any format outside (mlt_image_none, mlt_image_invalid).

Copilot

Pull request overview

Copilot reviewed 32 out of 32 changed files in this pull request and generated 10 comments.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

bmatherly

Thanks for this great PR. Here are some comments for your consideration. I did not run the code, so I apologize in advance if my comments are just lacking understanding.

bmatherly · 2026-05-07T02:35:16Z


 # image filters
-image_convert=movit.convert,avcolor_space,imageconvert
+image_convert=placebo.convert,movit.convert,avcolor_space,imageconvert


Using this pattern, will we be able to mix placebo, movit, avfilter, and other modules that use the new private type? What if we change the placebo shader filter_process() to call push_convert()? If we do that, frames with placebo images will have the converter they need without the need for a normalizer. I wonder if there is any need for a placebo normalizer at all (or a normalizer for other future modules that use private type).

yes, because the private format is tagged in a property. I used the wrong key. See below

What if we change the placebo shader filter_process() to call push_convert()?

Push appends, and the functions handling private should come earlier because they often do not handle all the image formats. For example, when placebo converts away from GPU, it can only do rgba, but the request might be different.

I added a mlt_frame_prepend_convert_image(), called it in placebo.shader's process callback, and removed it from loader.ini, and that is working. I will clean that up and push it.

bmatherly · 2026-05-07T02:51:12Z

+
+int placebo_frame_is_tex(mlt_frame frame, mlt_image_format format)
+{
+    const char *image_format = mlt_properties_get(MLT_FRAME_PROPERTIES(frame), "mlt_image_format");


I would suggest a different key for this property because "placebo" is, by definition, NOT a valid mlt_image_format. How about "_priv_format_id"? This could be a standard convention that we use for other modules that use the private type.

That was supposed to be "mlt_image_private" as described in #1235 (comment)
It is not really necessary to preface frame methods with underscore to hide them from serialization because they don't serialize, and here the value is possibly meaningful across modules.

changed to "mlt_image_private"

bmatherly · 2026-05-07T02:58:52Z

+void placebo_frame_set_requested_tex(mlt_frame frame, int requested)
+{
+    mlt_properties props = MLT_FRAME_PROPERTIES(frame);
+    int count = mlt_properties_get_int(props, "_placebo_requested");


Maybe we don't want to support or worry about this hypothetical future idea...

But, in a future world where avfilter also uses mlt_image_private, would this still work if we had a filter stack: placebo->avfilter->placebo?

I think the second placebo might try to interpret the private avframe type as a placebo. Maybe we could use a stack to push/pop private type requests?

It is not a hypothetical. It was needed now to make placebo->brightness(rgba)->placebo, placebo->sepia(yuv422)->placebo, and placebo->movit->placebo work, which is all I have tested so far (in addition to placebo alone and placebo->placebo). If avfilter follows this pattern, it can intermingle.

I think the second placebo might try to interpret the private avframe type as a placebo.

No, that is why it tags the requested mlt_format_private with property "_placebo_requested" and placebo.convert requires for that (placebo_frame_wants_tex()). It cannot trust mlt_image_format==mlt_image_private alone per my comment in the other PR linked above. The placebo render and shader filters also check property "mlt_image_private"=="placebo" after requesting it through mlt_frame_get_image() to be sure it can continue.

and mlt_properties_clear()

D-Ogi and others added 10 commits February 1, 2026 22:48

Fix clang-format violation in save_cache log line

394d73c

Break long mlt_log_info() call into multi-line format to match the project's clang-format rules (same style as load_cache above).

Fix clang-format-14 violations in placebo module

cc02961

Run clang-format-14 (matching CI) on filter_placebo_shader.c and gpu_context.c to fix designated initializer spacing, ternary line breaks, and long argument lists.

change a log to debug level

d26b3f3

Merge branch 'feature/placebo-module' of https://github.com/D-Ogi/mlt …

11bf94d

…into libplacebo

add MLT_PLACEBO_CACHE_PATH and replace atext()

3b112bb

Add placebo.convert filter

ce99364

- Introduced new `mlt_image_private` format for module internal use. - Improved placebo texture reuse across filters.

ddennedy changed the title ~~Libplacebo~~ Add libplacebo GPU module with render and shader filters May 6, 2026

ddennedy added this to the v7.40.0 milestone May 6, 2026

ddennedy added 2 commits May 6, 2026 09:54

fix cppcheck and redundant workflow runs

1def44b

fix ctest on msvc

e80a020

ddennedy marked this pull request as ready for review May 6, 2026 17:38

ddennedy requested a review from Copilot May 6, 2026 17:38

Copilot started reviewing on behalf of ddennedy May 6, 2026 17:39 View session

ddennedy added 2 commits May 6, 2026 11:03

docs for placebo private image handling

626cf8a

Enhance image conversion logic to handle private format cases

9f97a83

Copilot started work on behalf of ddennedy May 6, 2026 20:42 View session

ddennedy review requested due to automatic review settings May 6, 2026 20:53

ddennedy requested a review from Copilot May 6, 2026 20:53

Copilot started reviewing on behalf of ddennedy May 6, 2026 20:54 View session

Copilot finished work on behalf of ddennedy May 6, 2026 20:54

Copilot AI reviewed May 6, 2026

View reviewed changes

ddennedy and others added 2 commits May 6, 2026 14:16

Apply suggestions from code review

86df189

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

fix lazy allocation not thread-safe

7cd956a

ddennedy added 3 commits May 6, 2026 14:39

fix races in placebo.shader

9a6e134

Add permanent failure flag for GPU initialization and fix leak

f2a9e83

Add permanent failure and shutdown flags for GPU management

8d7260a

ddennedy mentioned this pull request May 6, 2026

Add libplacebo GPU module with render and shader filters #1201

Closed

bmatherly reviewed May 7, 2026

View reviewed changes

use "mlt_image_private"

34b2717

and mlt_properties_clear()

Conversation

ddennedy commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ddennedy commented May 6, 2026

Uh oh!

Copilot AI commented May 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bmatherly left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ddennedy commented May 6, 2026 •

edited

Loading