Skip to content

Commit 695c971

Browse files
committed
Match Java's key bound when serializing deletion vector bitmaps
Java's RoaringPositionBitmap.readKey rejects keys above Integer.MAX_VALUE - 1, while the serializer allowed one more key, so PyIceberg could write a deletion vector Java cannot read.
1 parent 5fdd51f commit 695c971

1 file changed

Lines changed: 5 additions & 2 deletions

File tree

pyiceberg/table/deletion_vector.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -102,8 +102,11 @@ def _serialize_bitmap(bitmaps: list[BitMap]) -> bytes:
102102
with io.BytesIO() as out:
103103
out.write(len(non_empty).to_bytes(8, "little"))
104104
for key, bitmap in non_empty:
105-
if key > MAX_JAVA_SIGNED:
106-
raise ValueError(f"Key {key} is too large, max {MAX_JAVA_SIGNED} to maintain compatibility with Java impl")
105+
# Java's RoaringPositionBitmap.readKey rejects keys above Integer.MAX_VALUE - 1
106+
if key > MAX_JAVA_SIGNED - 1:
107+
raise ValueError(
108+
f"Key {key} is too large, max {MAX_JAVA_SIGNED - 1} to maintain compatibility with Java impl"
109+
)
107110
out.write(key.to_bytes(4, "little"))
108111
# Run-length encode before serializing so run-heavy vectors (e.g. contiguous
109112
# deletes) stay compact, matching Java's BitmapPositionDeleteIndex.

0 commit comments

Comments
 (0)