Skip to content

fix, nested std::array reads fail against flat multi-dim H5T_ARRAY datasets #279

Description

@steven-varga

Symptom

Reading an HDF5 dataset whose on-disk datatype is a multi-dimensional H5T_ARRAY (e.g. H5T_ARRAY { [3][5] H5T_STD_I64LE }) into the canonical-mapped C++ type fails with HDF5 type-conversion errors:

H5T__path_find_real(): no appropriate function for conversion path
unable to convert between src and dest datatype

Reproducer

Against the HDFGroup canonical h5ex_t_array.h5 (publicly hosted at https://rhdf5-public.s3.eu-central-1.amazonaws.com/h5ex_t_array.h5):

auto ds = h5::open(fd, "/DS1");
using row_t = std::array<std::array<int64_t, 5>, 3>;
auto data = h5::read<std::vector<row_t>>(ds);  // throws

On-disk type per h5dump:

DATATYPE  H5T_ARRAY { [3][5] H5T_STD_I64LE }
DATASPACE  SIMPLE { ( 4 ) / ( 4 ) }

Workaround used in examples/s3/s3.cpp: drop to raw H5Dread with H5Dget_type(ds) and a flat std::vector<int64_t>.

Hypothesis

The Winston canonical mapping spec at h5cpp/H5Tmeta.hpp:371 produces H5T_ARRAY{[3] H5T_ARRAY{[5] i64}} (nested) for std::array<std::array<i64,5>,3>, but the file stores H5T_ARRAY{[3][5] i64} (flat multi-dim). These have identical memory layouts but HDF5's H5T_path_find treats them as distinct types and refuses conversion.

Suggested fix direction

Collapse contiguous nested std::array<std::array<T,M>,N> chains into a single multi-dim H5T_ARRAY call: H5Tarray_create2(dt_t<T>, 2, {N,M}) instead of nested 1-D H5Tarray_create2 calls. This matches what h5ex_t_array.c (HDFGroup canonical example) emits and what any third-party writer using the documented multi-dim form will produce.

Out of scope

  • ROS3-specific concerns — the bug reproduces on any local file with multi-dim H5T_ARRAY; S3 is just the convenient public reproducer.
  • Forward-compat writes (already correct? worth verifying separately).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions