Tracking issue for improving encoding performance

[197g]

According to #562 (comment) the encoder lacks behind (Chromium's patch of) libpng in various aspects, that is both file size and runtime performance are behind in many percentiles of real-world use. This issue should track PRs and ideas for closing this gap.

Known issues

• fdeflate is very fast, but can blow up the filesize significantly especially also for small images. Small often coincides on the web with a use as previous or icon, which are unfortunately also highly sensitive to bandwidth requirements, i.e. must be optimized for size.
• Produces invalid PNG for indexed with too large palette #584
• Indexed png + alpha #583

Ideas

• Parallel encoding for wall clock time? Parallel PNG encoding #530

[fintelia]

I've been doing some experimentation with alternative compression strategies in a private branch of fdeflate. It is a continuation of image-rs/fdeflate#29 but greatly expanded. Hoping to cleanup the code, actually design the public API, and then put together a draft PR reasonably soon.

Results so far are quite promising for compression ratios in the range of zlib levels 1-3. I've been basing the implementation partially on lz4's compressor which gets a big speedup by identifying which parts of the file are/aren't compressing well and moves quickly over the incompressible parts.

I've also tried out slower compression levels and while I've been able to match the compression ratios from zlib / zlib-ng by adjusting parameters, performance has been mostly on-par (for levels 4-7ish range) and actually worse for the higher levels.

[fintelia]

Created a draft PR with the current state: image-rs/fdeflate#51

[Shnatsel]

Performance on par with zlib-ng and possibly trailing behind zlib-rs slightly is still valuable because fdeflate is implemented entirely in safe code, which is a key requirement for Chromium.

The numbers already compare favorably to miniz_oxide on the relevant metric for each of the high-level compression presets: fdeflate 1 has better speed and compression ratio than miniz_oxide level 1 and could be mapped to Compression::Fast, fdeflate level 5 similarly beats miniz_oxide on both metrics and would be a good candidate for Compression::Balanced, and level 9 has a higher compression ratio than any of the other libraries, which is the most important parameter for the maximum compression level.

So I think it would be a good idea to make fdeflate compression the default already, and relegate flate2 to an optional opt-in dependency.