Skip to content

BUG: Streaming test baselines stale for OpenSlide 3.4.1 (itkOpenSlideTestStreaming, itkOpenSlideTestApproximateStreaming) #40

Description

@hjmjohnson

Two streaming tests fail with enormous image-comparison errors when built against Ubuntu 22.04's OpenSlide 3.4.1; the stored baselines were produced by an older OpenSlide with a different JPEG decompression path. main reproduces (latest "Build, test, package" run on main is red).

Failing tests and magnitudes
Test ImageError
itkOpenSlideTestApproximateStreaming 21,622,187
itkOpenSlideTestStreaming 18,887,606

Millions of differing pixels, mean error > 100 — this is a decode-path difference, not numeric drift. 7 of 9 tests pass, so the module itself is healthy.

Environment and reproduction
  • Runner: ubuntu-22.04, system libopenslide-dev 3.4.1+dfsg-5build1.
  • main reproduces: latest "Build, test, package" run on main is a failure (2026-04-23); prior main runs in Nov 2025 and Mar 2025 fail the same way.
  • First surfaced by PR COMP: Modernize CMake and update CI to v5.4.6 with OpenSlide system packages #37 (COMP: Modernize CMake and update CI to v5.4.6 with OpenSlide system packages, merged 2026-04-03), which is what first let CI actually build and run the test suite against a system OpenSlide.
  • Stale baseline: test/Input/CMU-1-level-1-stream-200.mha (and its sibling streaming baseline).
Suggested fix
  1. Build locally against OpenSlide 3.4.1 and run only the two streaming tests.
  2. Capture the new outputs and confirm by eye that they are correct renderings (not a second wrong answer).
  3. Upload the regenerated .mha baselines to ExternalData and update the content links.
    Note: this repo migrated from .md5 to .cid ExternalData tags in PR ENH: Convert ExternalData .md5 tags to .cid (IPFS content IDs) #38 (merged 2026-04-22), and PR ENH: Convert ExternalData .md5 tags to .cid (IPFS content IDs) #39 continues that migration and is still open — regenerate after ENH: Convert ExternalData .md5 tags to .cid (IPFS content IDs) #39 lands, and write .cid tags, not .md5.
  4. Alternative if the decode path is expected to keep drifting across OpenSlide versions: give these two tests an explicit comparison tolerance / an allowed number of differing pixels rather than pinning a byte-exact baseline to one OpenSlide release.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions