Skip to content

Fix indexing an empty DynamicTableRegion selection in memory - #1549

Open
h-mayorquin wants to merge 1 commit into
devfrom
fix_data_access_hdmf
Open

Fix indexing an empty DynamicTableRegion selection in memory#1549
h-mayorquin wants to merge 1 commit into
devfrom
fix_data_access_hdmf

Conversation

@h-mayorquin

Copy link
Copy Markdown
Contributor

Motivation

Selecting a zero-length region from a DynamicTableRegion raised IndexError: arrays used as indices must be of integer (or boolean) type when the target table's columns held their data as numpy arrays. This happens for an empty slice and, more commonly, for a ragged region row that references no target rows, which is the shape produced when rows are appended to a table whose DynamicTableRegion column has no targets for them. DynamicTableRegion.get takes np.unique of the selected row indices, and np.unique of an empty selection returns a float64 array, which numpy rejects as an index. A column whose data is a list takes a different branch in Data.get, indexing elementwise, and tolerates the float array, which is why the existing TestDynamicTableRegion suite, whose fixture is built from lists, stayed green.

The same selection works for files on disk, because the indices come back from HDF5 already integer-typed, so the container behaves differently in memory than it did after a round trip. I think it should work for both cases. I cast the unique row indices to an integer dtype rather than returning early on an empty selection, because region data are row indices: the cast is a no-op when the selection is non-empty, and it fixes the class of problem (a non-integer index array) rather than the one input that produces it.

I found this downstream in AllenNeuralDynamics/aind-ephys-pipeline#121, reported by @alejoe91 @CodyCBakerPhD, where units from a second probe are appended to an NWB units table that already has an electrodes region column. Those units are written with an empty region, the file writes and reads back correctly, and only inspecting the in-memory NWBFile before writing raised.

How to test the behavior?

import numpy as np
from hdmf.common import DynamicTable, VectorData

target = DynamicTable(
    name="target", description="t",
    columns=[VectorData(name="x", description="x", data=np.array([1, 2, 3]))],
)
table = DynamicTable(name="table", description="t")
table.add_column(name="region", description="r", index=True, table=target)
table.add_row(region=[0])
table.add_row(region=[])   # a row that references no target rows

table["region"][0]   # one-row DataFrame, before and after
table["region"][1]   # IndexError before, empty DataFrame after

Checklist

  • Did you update CHANGELOG.md with your changes?
  • Does the PR clearly describe the problem and the solution?
  • Have you reviewed our Contributing Guide?
  • Does the PR use "Fix #XXX" notation to tell GitHub to close the relevant issue numbered XXX when the PR is merged?

@codecov

codecov Bot commented Jul 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.22%. Comparing base (f1b2eb4) to head (8c658ed).

Additional details and impacted files
@@           Coverage Diff           @@
##              dev    #1549   +/-   ##
=======================================
  Coverage   93.22%   93.22%           
=======================================
  Files          41       41           
  Lines       10221    10221           
  Branches     2115     2115           
=======================================
  Hits         9529     9529           
  Misses        415      415           
  Partials      277      277           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant