Clean Aperture semantics should be clarified

[cconcolato]

The current semantics indicate:

The pixel aspect ratio and clean aperture of the video may be specified using the PixelAspectRatioBox and CleanApertureBox sample entry boxes, respectively. These are both optional; if present, they over-ride the declarations (if any) in structures specific to the video codec, which structures should be examined if these boxes are absent.

I find the sentence "overriding structures specific to the video codec" unclear, especially for Clean Aperture.
I see 2 use cases related to cropping:

• cropping because the codec can only code X pixels (e.g. multiple of 16 pixels) but the actual source is Y pixels. In this use case, there are 2 sub-cases:
  a) The codec has built-in cropping (e.g. HEVC). Cropping is applied by the decoder according to the "conformance cropping window" without the container level/application knowing about it. The extra pixels are not seen in the sample entry width and height. In that case, the clap cannot override that behavior.
  b) The codec does not have that cropping feature. In that case, the clean aperture box can be used to signal the cropping, and obviously it overrides the codec output (i.e. the sample entry still documents the additional pixels).
• cropping because of the source content contains undesired pixels. I "think" this is the original intent of the QuickTime clean aperture. This can also be used to crop to the active picture.

I think the ISOBMFF spec should be update to say that:

• in-stream cropping features that are mandated by the codec apply
• in-stream cropping features that are optional, e.g. as metadata or sei, should be reflected at the container level (i.e. values should match) and if they don't the container should take precedence.

[podborski]

in-stream cropping features that are optional, e.g. as metadata or sei, should be reflected at the container level (i.e. values should match) and if they don't the container should take precedence.

The container shall always take precedence, no?

[cconcolato]

The group discussed and wonders how real-world implementations would behave if the clap would ask to crop a stream (512 px) with dimensions larger (e.g. 508px) than the cropped dimensions indicated by the in-stream structures (e.g. 500px).
We also note that ISOBMFF should not be dictating player behavior, but rather document the stream, as integration of decoder and cropping tool may be different from implementation to implementation.
We welcome further inputs on how to improve this text.
We may want to create conformance files to test various player behaviors.

[bradh]

Was there an output from this activity? Or was it "left as is"?

[cconcolato]

It was left as is. The last comment still holds: we welcome text and/or conformance files.