Iceberg metadata can reference a parent-snapshot-id that is absent from the snapshots list

[sfc-gh-mslot]

Summary

After snapshot expiration, pg_lake writes table metadata whose current snapshot carries a parent-snapshot-id pointing to a snapshot that is no longer present in the snapshots array.

Example

A table where snapshot 2 is current and snapshot 1 has been expired:

"current-snapshot-id": 3073646702985352826,
"snapshots": [
  {
    "snapshot-id": 3073646702985352826,
    "parent-snapshot-id": 8284728199407151760,   // not in this list
    "sequence-number": 2,
    ...
  }
],
"snapshot-log": [
  { "timestamp-ms": ..., "snapshot-id": 3073646702985352826 }
]

Snapshot 8284728199407151760 is referenced as the parent but was dropped from both snapshots and snapshot-log when it was expired (this happened well within the default max_snapshot_age of 1800s in the observed case).

The mechanism: pg_lake_iceberg/src/iceberg/write_table_metadata.c:328 emits parent-snapshot-id whenever the retained snapshot's parent id is non-zero, without checking whether that parent is still retained.

Is this a spec violation?

Unclear. The spec only defines parent-snapshot-id as "the snapshot ID of the snapshot's parent," with no explicit requirement that the parent remain in snapshots, and Apache Iceberg's own expireSnapshots similarly leaves dangling parent references. So this may be compliant.

Why it still matters

Some Iceberg consumers that validate snapshot lineage assume a non-null parent-snapshot-id resolves to a snapshot in the metadata, and can fail (e.g. NPE / validation errors) when it does not. This makes pg_lake tables unreadable by such consumers after the first expiration.

Possible fixes

1. Omit parent-snapshot-id when the parent is no longer in snapshots.
2. Retain the parent snapshot (keep >= 2, or do not expire a snapshot still referenced as a parent).