Improve ICC profile manipulation

[Socolin]

Hello,

Note

It's not a current need that I have, more like an idea, that I have in my head and I'm curious to know if this make sense or not and if it could be doable in the futur.

I've been looking into the ICC Profiles and color conversion and I'm wondering, if something like that make sense and would be possible someday.

var img1 = Image.Load(new DecoderOptions {ColorProfileHandling = ColorProfileHandling.Preserve, "image-using-rgb-profile");
var img2 = Image.Load(new DecoderOptions {ColorProfileHandling = ColorProfileHandling.Preserve, "image-using-some-cmyk-profile");

// So here the idea is to create an image, working with CIELab or CIEXYZ with like `Vector4` for each pixel 
using var canvas = Image.CreateCieLabImage(100, 100);

// When writting the images, it convert it, but only stop to the the CIELab / CIEXYZ color space
// This aim to have no loss in scenario where all the colors of the image are contained in the gamut 
// of the source and destination profile, but not in sRGB
canvas.Mutate(x => x.DrawImage(img1))
canvas.Mutate(x => x.DrawImage(img2))

// Then we save it to the file with the destination profile (for example CMYK)
var destinationProfile = new IccProfile(File.ReadAllBytes("destination.icc"))
canvas.SaveAndApplyProfile(destinationProfile, "some-file.png")

So at the end the color from each image would have not been converted to the sRGB colors space.
Instead of
Image Color Space => PCS => sRGB then sRGB => PCS => Destination Color Space
it would only do
Image Color Space => PCS => Destination Color Space

Maybe this make no sense and I misunderstood something, let me know if this is the case.

[JimBobSquarePants]

I'm afraid that will never be possible.

TPixel formats require direct conversion to/from an RGBA sequenced Vector4 instance. These formats and rules are based on established formats from DirectX etc.

[Socolin]

Ok, so follow up questions so I'm sure I understand the limitation (I feel I miss-understood the scope of IPixel)

So TPixel in ImageSharp is always representing RGBA like color, and IPixel cannot be extended to represent CMYK / HSL etc.. and this is by design. So It's not possible to create a canvas where the pixel are not RGBA ?

So we should be able to load a CMYK image file, but it should be converted to sRGB (or other rgb color space) before it can be used in ImageSharp ?

[JimBobSquarePants]

In ImageSharp, Image<TPixel> is the in-memory surface type. TPixel is any unmanaged struct that implements IPixel<TPixel>, which in turn extends the non generic IPixel. The key part of that base interface is the conversion contract:

• Every TPixel must be able to:
  • Convert to and from Rgba32 (ToRgba32, FromRgba32)
  • Convert to and from a Vector4 view (ToScaledVector4, FromScaledVector4, ToVector4, FromVector4)
    See IPixel and IPixel<TSelf>.

ToScaledVector4 and FromScaledVector4 use normalized 0-1 values in least to most significant channel order (R, G, B, A for Rgba32). This is documented on IPixel<TSelf>.

All processors operate on that RGBA Vector4 view. Blend modes, filters, drawing, quantizers and so on all assume “up to four channels interpreted as RGB plus alpha”. The concrete pixel structs (Rgba32, Bgra32, L8, RgbaVector, etc) are just different memory layouts that can always be expanded to that RGBA representation and round-tripped.

TPixel formats require direct conversion to and from an RGBA sequenced Vector4 instance.

Practical consequences

• In practice, TPixel always represents something that can be interpreted as RGBA when expanded. Even grayscale formats define a mapping to RGBA or Vector4. For example, see L8.
• You could implement a custom Cmyk32 : IPixel<Cmyk32>, but the interface forces you to define its RGBA projection, and the entire processing pipeline operates on that RGBA projection, not on native CMYK or Lab math.
• CMYK, YCCK, Lab, YCbCr and similar file encodings are converted through the ICC and color space system into an RGB working space before being written into Image<TPixel>. The in-memory surface you mutate is always RGB like, not CMYK.

So you can load CMYK images, but you cannot keep them as “raw CMYK pixels” all the way through the pipeline under the current IPixel contract. Everything inside Image<TPixel> is designed around something that can be viewed as RGBA.

Pixel format naming and lineage

ImageSharp’s pixel formats and naming follow the same lineage as XNA, MonoGame and DirectX.

• Rgba32 is a 32 bit packed pixel with 8 bits per component, laid out as R, G, B, A from least to most significant byte. See Rgba32.
• This mirrors the classic XNA and MonoGame Color struct, which is also a packed RGBA value. See Microsoft.Xna.Framework.Color.
• XNA’s SurfaceFormat enum is a thin abstraction over Direct3D and later DXGI texture formats, and MonoGame reuses those semantics. For example, see MonoGame Color.
• DirectX defines the canonical packed GPU formats such as DXGI_FORMAT_R8G8B8A8_UNORM, DXGI_FORMAT_B8G8R8A8_UNORM, DXGI_FORMAT_B5G6R5_UNORM, DXGI_FORMAT_B5G5R5A1_UNORM, and others. See DXGI_FORMAT.

ImageSharp’s built in pixel formats are deliberately aligned with that ecosystem:

• Names such as Rgba32, Bgra32, Bgr565, Bgra5551, Rgba1010102, HalfVector4 map directly to the same mental model as MonoGame and XNA surface formats and DXGI formats.
• Bit layouts (channel order, byte order, bit depth) follow the same conventions to simplify interop with GPU APIs and game frameworks.

That is why TPixel must always map cleanly to an RGBA Vector4: it is designed as a CPU side analogue of those GPU surface element formats, not as a free form N channel color space container.

[Socolin]

Thanks a lot for the explanations. It's now clear why it's not possible, this make sense.