Skip to content

Sentinel-3 SLSTR/OLCI Zarr stores inaccessible via HTTP for certain months #39

@emmanuelmathot

Description

@emmanuelmathot

Summary

Several monthly buckets of Sentinel-3 data (SLSTR LST, OLCI LFR) in the EOPF Zarr Sample Service are inaccessible via standard HTTP/Zarr clients (xarray.open_datatree, zarr.open_group, fsspec). Two distinct issues were identified:

Issue 1: Literal /../ in S3 object keys (Jun 2025)

The June 2025 buckets (202506-s03slslst, 202506-s03olcil2lfr) contain objects whose keys include a literal /../ path segment:

30/../products/cpm_v256/S3A_SL_2_LST____20250630T224247_....zarr/.zgroup

The STAC catalog returns these paths as-is in asset hrefs. HTTP clients (requests, fsspec, urllib3) resolve /../ during URL normalization, turning 30/../products/ into products/ — but no object exists at the resolved key. This makes all data in these buckets inaccessible via any standard HTTP-based Zarr client.

Affected buckets (confirmed):

  • 202506-s03slslst
  • 202506-s03olcil2lfr (sentinel-3-olci-l2-lfr)

Not affected (clean keys like 28/products/...):

  • 202504-s03slslst
  • 202505-s03slslst
  • 202510-s03slslst through 202603-s03slslst-eu

Issue 2: Missing Zarr metadata files (Sep 2025)

The September 2025 bucket (202509-s03slslst) has clean key paths (no /../) and chunk data files exist, but the root Zarr metadata files (.zgroup, .zattrs, .zmetadata) are missing. This means the Zarr store cannot be opened by any client.

# These all return 404:
21/products/cpm_v256/S3A_SL_2_LST____20250921T...zarr/.zgroup
21/products/cpm_v256/S3A_SL_2_LST____20250921T...zarr/.zattrs
21/products/cpm_v256/S3A_SL_2_LST____20250921T...zarr/.zmetadata
21/products/cpm_v256/S3A_SL_2_LST____20250921T...zarr/zarr.json

# But chunk data exists:
21/products/cpm_v256/S3A_SL_2_LST____20250921T...zarr/conditions/auxiliary/biome/0.0  ← exists

For comparison, working buckets (e.g., Apr 2025) have proper Zarr v2 metadata:

28/products/cpm_v256/S3B_SL_2_LST____20250428T...zarr/.zgroup  ← 200 OK, {"zarr_format": 2}

Reproducibility

import pystac_client, requests

catalog = pystac_client.Client.open("https://stac.core.eopf.eodc.eu")
search = catalog.search(
    collections=["sentinel-3-slstr-l2-lst"],
    bbox=[0, 40, 10, 50],
    datetime="2025-06-01/2025-06-30",
    max_items=1,
)
items = list(search.items())
href = items[0].assets["product"].href
print(href)  # Contains /../
print(requests.get(href + "/.zgroup").status_code)  # 404

Impact

This issue was discovered during the EOPF Zarr Community Notebook Competition judging. At least one submission (northflow_sentinel3_lst_climval) was unable to read Sentinel-3 SLSTR LST data and reported a GroupNotFoundError in their submission feedback.

Monthly status summary (sentinel-3-slstr-l2-lst)

Month Key format .zgroup Status
Apr 2025 28/products/... 200 ✅ OK
May 2025 clean 200 ✅ OK
Jun 2025 30/../products/... 404 ❌ Broken (literal /../)
Sep 2025 21/products/... 404 ❌ Broken (missing metadata)
Oct 2025 clean 200 ✅ OK
Nov 2025 clean 200 ✅ OK
Dec 2025 clean 200 ✅ OK
Jan 2026 clean 200 ✅ OK
Feb 2026 clean 200 ✅ OK
Mar 2026 clean 200 ✅ OK

Suggested fix

  1. Jun 2025: Re-ingest or rename the objects to remove the /../ from the key paths, and update the STAC catalog accordingly
  2. Sep 2025: Re-ingest the Zarr stores with proper root metadata files (.zgroup, .zattrs, .zmetadata)
  3. Consider adding a CI check to validate that newly ingested Zarr stores are accessible via xr.open_datatree(href, engine="zarr")

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions