calculateOverlap on aligned image fails to capture overlaps

[kunleng]

Hi there,

I really appreciate the awesome support that the Giotto team has been providing!

I have a Xenium dataset for which I want to extract overlaps with an aligned image. The aligned image is a mask created from segmentation of the H&E image obtained from the same Xenium sample. I derived the affine matrix from Xenium Explorer and used it to align both the H&E image and the mask image.

It seems like the alignment is working -- see attached output from spatInSituPlotPoints overlaying the cell polygons over the H&E image and the mask image.

However, when I run calculateOverlap(g2, name_overlap = 'protein', spatial_info = 'cell', image_names = 'mask'), no overlaps are identified. All entries of the overlap matrix are 0.

calculateOverlap does work when I run it using image_names = 'dapi' for example using the Xenium immunofluorescence images.

Could there be some issue with the affine transformation not being carried over when calculateOverlap is run on the aligned image?

I was thinking about loading the mask image as a giottoLargeImage but haven't found a way to subsequently apply the affine transformation.

Many thanks in advance.

Best,
Kun

[kunleng]

Another thing I noticed is that when I run calculateOverlap with image_names = 'dapi' for example vs image_names = 'mask', the number of raster cells is different:

For mask image:
Cannot preload entire working area of 57035750 cells with max_cells_in_memory = 3e+07. Raster values will be read for each feature individually.

For dapi image:
Cannot preload entire working area of 1262766725 cells with max_cells_in_memory = 3e+07. Raster values will be read for each feature individually.

What determines the number of raster cells? Could this be related to why calculateOverlap fails to find overlaps with the mask image?

[jiajic]

HI @kunleng,

Happy to help, and sorry about the confusion here.

Unfortunately, like you noticed, calculateOverlap() is currently not working for affine transformed images, but should be straightforward to finish implementing as soon as I get some time. I'll update on this thread when it's fixed.

More detail:
Affine transforms on images are a delayed operation due to how big images are. For visualization purposes, a reasonable ROI and amount of pixels are cropped and sampled from the image. This downsampled image is then transformed in a process that produces a .png output and 1. likely causes anti-aliasing-based numerical inaccuracies, and 2. messes with the range of values allowed per pixel. Fine for visualization purposes, but worrisome for analysis.

The part I still have to do is set up handling of polygon and image for calculateOverlap with giottoAffineImage in the background that performs the affine transform in reverse on the polygons and applies it to the untransformed image values.

For your second comment, the cells referred to there should be image cells/pixels. The message you see should not be related to this issue, and is more like diagnostic info about the {exactextractr} package is processing the data based on available computer resources.

Best,
George

[kunleng]

Hi George thank you so much for the fast response!! I'll wait with fingers crossed for your update, and in the meanwhile I'll work on other aspects of the analysis :)

[kunleng]

Hi @jiajic , hope your 2025 is off to a good start! I just wanted to follow up on whether the Giotto team has had any progress with implementing calculateOverlap() for affine transformed images. Thanks!