Issues with taking the maximum

[ruuda]

Currently the score that Butteraugli outputs is the maximum over the entire image. If I understood correctly, the rationale is that errors (high scores) are noticeable, so even if there is a large error in even a small part of the image, this is unforgivable. However, this has a few implications:

• The maximum is sensitive to small changes. The pixel with the maximum error could be an outlier. If an image is altered only slightly, the worst error might be in a different part of the image, and the score will change by a lot. Something like the 90th or 98th percentile would still penalize large errors in small regions, while being more stable.
• The converse of penalizing large errors in small regions, is that improvements on large regions are not reflected in the score at all, if there is no improvement in the region where the score is the worst.

Of course, capturing both in one single number is difficult. Outputting a distribution or even just a few percentiles would be more useful, but it would also make the output harder to interpret and compare.