Skip to content

Refactor image conversion#1235

Draft
ddennedy wants to merge 3 commits intomasterfrom
refactor-image-convert
Draft

Refactor image conversion#1235
ddennedy wants to merge 3 commits intomasterfrom
refactor-image-convert

Conversation

@ddennedy
Copy link
Copy Markdown
Member

@ddennedy ddennedy commented May 5, 2026

  • Refactored image conversion handling across various modules to use new framework functions for better clarity and maintainability.
  • Now, there can be multiple possible image converters per frame, stopping after the first success. But also, one converter can do part of the work (e.g. download from GPU) while a later one converts it to the actual requested format.
  • Replaced direct calls to convert_image with mlt_frame_convert_image and added checks for the existence of conversion functions.
  • Improved documentation in YAML files to clarify the purpose of image conversion callbacks.
  • Added tests to verify the correct propagation of image conversion functions through frames and producers.

- Refactored image conversion handling across various modules to use new framework functions for better clarity and maintainability.
- Now, there can be multiple possible image converters per frame stopping after the first success. But also, one converter can part of the work (e.g. dowload from GPU) while a later one converts it to the actual requested format.
- Replaced direct calls to `convert_image` with `mlt_frame_convert_image` and added checks for the existence of conversion functions.
- Improved documentation in YAML files to clarify the purpose of image conversion callbacks.
- Added tests to verify the correct propagation of image conversion functions through frames and producers.
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors frame image conversion in MLT from a single frame->convert_image callback to a list-based dispatcher, then updates loaders, filters, and tests to use the new API. It mainly affects frame conversion plumbing in the framework plus the producer/filter paths that register or consume image converters.

Changes:

  • Added new mlt_frame_*convert_image* APIs and updated frame/tractor/timeremap propagation to use callback lists.
  • Switched producers/filters from direct frame->convert_image assignment/calls to mlt_frame_push_convert_image(), mlt_frame_has_convert_image(), and mlt_frame_convert_image().
  • Added loader/config/test updates for ordered image converter registration and propagation.

Reviewed changes

Copilot reviewed 32 out of 32 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/tests/test_tractor/test_tractor.cpp Adds a tractor test for convert-image callback propagation.
src/tests/test_mod_avformat/test_mod_avformat.cpp Adds loader/loader-nogl tests for attached image converters.
src/tests/test_frame/test_frame.cpp Adds unit tests for the new frame conversion dispatcher APIs.
src/tests/CMakeLists.txt Links Qt tests against mlt and fixes parsed link-arg usage.
src/modules/xine/link_deinterlace.c Replaces direct image-convert callback use with frame API.
src/modules/xine/filter_deinterlace.c Replaces direct image-convert callback use with frame API.
src/modules/qt/transition_qtblend.cpp Uses new convert-image helpers in Qt blend transition.
src/modules/openfx/filter_openfx.c Switches OFX filter back-conversion to frame API.
src/modules/movit/filter_movit_crop.cpp Uses new convert-image helpers before Movit cropping.
src/modules/movit/filter_movit_convert.yml Updates notes to describe callback registration instead of pointer assignment.
src/modules/movit/filter_movit_convert.cpp Refactors Movit conversion to participate in ordered callback dispatch.
src/modules/gdk/producer_pixbuf.c Uses new convert-image helpers for pixbuf output conversion.
src/modules/gdk/producer_pango.c Uses new convert-image helpers for cached pango frames.
src/modules/frei0r/transition_frei0r.c Uses new convert-image helpers for frei0r transition inputs.
src/modules/frei0r/filter_frei0r.c Uses new convert-image helpers for rgba64→rgba fallback.
src/modules/core/transition_luma.c Replaces direct callback checks/calls with frame helpers.
src/modules/core/producer_loader.c Loads ordered image converters from loader.ini.
src/modules/core/loader.ini Adds data-driven image_convert converter list.
src/modules/core/link_timeremap.c Copies convert-image callback lists between frames.
src/modules/core/filter_mask_start.c Removes now-redundant direct convert-image field copy.
src/modules/core/filter_imageconvert.yml Updates notes to describe callback registration.
src/modules/core/filter_imageconvert.c Registers image conversion via mlt_frame_push_convert_image().
src/modules/core/filter_fieldorder.c Uses new convert-image helpers for YUV conversion.
src/modules/core/filter_crop.c Uses new convert-image helpers for requested-format conversion.
src/modules/core/consumer_multi.c Refactors multi-consumer normalizer attachment for image converters.
src/modules/avformat/filter_avcolour_space.yml Updates notes to describe callback registration.
src/modules/avformat/filter_avcolour_space.c Registers avcolor conversion via mlt_frame_push_convert_image().
src/framework/mlt.vers Exports the new public frame conversion APIs.
src/framework/mlt_tractor.c Propagates convert-image callback lists through tractor frames.
src/framework/mlt_frame.h Declares new convert-image callback type and public APIs.
src/framework/mlt_frame.c Implements list-based image conversion dispatch and propagation helpers.
NEWS Documents the new image conversion API and behavior changes.

Comment thread src/framework/mlt_tractor.c Outdated
Comment thread src/modules/core/consumer_multi.c
Comment thread src/modules/movit/filter_movit_convert.cpp
Comment thread src/framework/mlt.vers Outdated
ddennedy and others added 2 commits May 5, 2026 17:28
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@bmatherly
Copy link
Copy Markdown
Member

Will modules specific image formats still need to be added to the mlt_image_format enum?

I aspire to pass AVFrame objects through the image stack so that we do not need to convert from AVFrame to MLT Image only to convert back to AVFrame when the next service is an FFMpeg service. To do that, should I add a new image format: mlt_image_format_avframe?

I was toying with an idea where we would make a new opaque image type called "mlt_image_format_private". If a service sets the type to private, then it must provide an appropriate conversion function that can convert the private type to an MLT image format.

I wonder what you would think about that idea.

@ddennedy
Copy link
Copy Markdown
Member Author

ddennedy commented May 6, 2026

We can add a mlt_image_private if the modules set a frame property such as "mlt_image_private" to a unique tag such as "placebo" and check that to determine compatibility. I can do that in my libplacebo branch if you like.

@bmatherly
Copy link
Copy Markdown
Member

That would be great. It would make a good example to follow.

Would mlt_image_format_movit also be considered a private format? Maybe we want to deprecate the movit format and change it to identify as a private format. I am not sure.

@ddennedy
Copy link
Copy Markdown
Member Author

ddennedy commented May 6, 2026

Re: mlt_image_private, a thing that makes the property for the private tag a little messy is: the difference between request vs. output paths especially when the image goes out of private and back in between GPU-CPU-GPU. This is explained more in the placebo branch's gpu_context.c helper functions around this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants