fix(upsert): bound file match filter to avoid segfault

[abnobdoss]

Rationale for this change

Upserts with large composite (multi-column) keys segfault during scan planning. The initial scan builds one And(EqualTo, EqualTo) per unique source key and Ors them into a deep tree, which PyArrow's C++ engine recurses over in GetFragments → Canonicalize, overflowing the C stack (SIGSEGV) at tens of thousands of keys.

This PR uses a bounded create_file_match_filter for the initial file-pruning scan: per-column min/max bounds (plus IsNull where needed), so predicate size scales with the number of join columns, not source rows. The exact create_match_filter is still used downstream for overwrite and insert filtering, so row-level correctness is unchanged.

The bounds are loose by design. With composite keys they can pull in false positives, but exact matching drops those before anything is written, so the only cost is scanning a few extra files in exchange for upserts that no longer crash. A follow-up could tighten this, for example per-prefix bounds or a grouped In, as long as the predicate size stays bounded.

Benchmarks

Timings cover expression construction, schema binding, PyArrow conversion, and applying the resulting Arrow filter to an in-memory table. They do not measure end-to-end upsert time, which also depends on table layout, file pruning, IO, and downstream exact matching. Benchmark script.

Before, exact match-filter path:

Unique source keys	1 join column	2 join columns	3 join columns
100	1.5 ms	7.3 ms	11 ms
1k	4.7 ms	87 ms	147 ms
10k	33 ms	SIGSEGV	SIGSEGV
100k	650 ms	SIGSEGV	SIGSEGV
1m	6.59 s	>30 s	>30 s

After, bounded file-match filter path:

Unique source keys	1 join column	2 join columns	3 join columns
100	1.0 ms	1.1 ms	1.2 ms
1k	1.0 ms	1.1 ms	1.2 ms
10k	1.1 ms	1.1 ms	1.3 ms
100k	1.3 ms	1.6 ms	1.9 ms
1m	3.7 ms	6.8 ms	9.0 ms

Related

• Upserting large table extremely slow #2159: upsert slow from building a massive BooleanExpression in create_match_filter before any IO. This PR replaces that pre-IO full-source predicate with the bounded filter. The overwrite and insert predicates are still exact and unchanged.
• Upsert with 1M rows extremely slow due to create_match_filter and txn.delete() performance #3129: large-upsert performance. Removes the full-source create_match_filter call from upsert(). Does not change the txn.delete() path or the direct create_match_filter use in that report's flow.
• Segfault on large multi-column Iceberg upserts #3508: segfault on large multi-column upserts. This PR fixes the initial scan path. The overwrite and insert paths still hit the same recursion and are left to follow-up.

Are these changes tested?

Yes. Added coverage for bounded-filter construction, the false-positive culling path, and a large composite-key regression test. The regression test runs the upsert in a subprocess, so a runtime that would segfault reports a pytest failure instead of killing the worker: the base branch exits 139 under Python 3.10, and this branch exits 0.

Ran:

• uv run python -m pytest tests/table/test_upsert.py -k "large_composite_key_initial_scan or file_match_filter or create_match_filter"
• uv run python -m pytest tests/table/test_upsert.py
• commit hooks

Are there any user-facing changes?

No API changes. Large composite-key upserts no longer crash during initial scan planning, and may scan a conservative superset of candidate files before exact row matching.

[github-actions]

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that's incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

[abnobdoss]

Still active - happy to address any feedback.

[ericyuanhui]

Are there any updates on this? Has the issue been resolved? I noticed there are two PRs, but neither has been merged yet. Could someone provide some clarification on the current status?