Feature Request / Improvement
Today we only support full deletes (overwrite). Supporting partial deletes can be achieved at several places in Iceberg:
- Pure delete operations:
- Deleting a manifest from the manifest list
- Deleting a manifest entry from a manifest
- Rewrite operation:
- Rewriting existing Parquet files that contain rows that match the predicate
Pure metadata deletes can be achieved using #518 and #539. For the rewrite operations, we need an API to ensure we're pluggable for other query engines. This would entail a source path (including delete-on-read files), destination-path, and a residual filter that needs to be applied to the file.
Feature Request / Improvement
Today we only support full deletes (overwrite). Supporting partial deletes can be achieved at several places in Iceberg:
Pure metadata deletes can be achieved using #518 and #539. For the rewrite operations, we need an API to ensure we're pluggable for other query engines. This would entail a source path (including delete-on-read files), destination-path, and a residual filter that needs to be applied to the file.