feat(table): support delete file removal in overwrite commits#851
Merged
zeroshade merged 1 commit intoapache:mainfrom Apr 7, 2026
Merged
feat(table): support delete file removal in overwrite commits#851zeroshade merged 1 commit intoapache:mainfrom
zeroshade merged 1 commit intoapache:mainfrom
Conversation
laskoviymishka
force-pushed
the
feat/delete-file-removal
branch
from
efd83f1 to
a790ad5
Compare
zeroshade
requested changes
Apr 7, 2026
Member
zeroshade
left a comment
zeroshade
left a comment
There was a problem hiding this comment.
Generally looks good to me, but needs tests
laskoviymishka
force-pushed
the
feat/delete-file-removal
branch
from
a790ad5 to
8af15ae
Compare
Extend the overwrite snapshot producer to atomically remove delete files (position + equality) alongside data file replacement. This is the commit primitive needed for compaction (apache#832) to clean up fully-applied delete files. - Add deletedDeleteFiles tracking to snapshotProducer with removeDeleteFile() - existingManifests() cross-checks content type when filtering entries - deletedEntries() splits by content type into separate data/delete manifests with correct ManifestContent in both Avro metadata and manifest list - newManifestWriter accepts ManifestWriterOption for content type propagation - New Transaction.ReplaceFiles() validates both data and delete files exist in the table with content-type gating before commit
laskoviymishka
force-pushed
the
feat/delete-file-removal
branch
from
8af15ae to
72f1881
Compare
zeroshade
approved these changes
Apr 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extend the overwrite snapshot producer to atomically remove delete files (position + equality) alongside data file replacement. This is the commit primitive needed for compaction (#832) to clean up fully-applied delete files.