Skip to content

Binary search optimization for Parquet Variant's object field ID lookup - #23638

Open
abigalekim wants to merge 4 commits into
NVIDIA:mainfrom
abigalekim:ak/variant-binary-search-id
Open

Binary search optimization for Parquet Variant's object field ID lookup#23638
abigalekim wants to merge 4 commits into
NVIDIA:mainfrom
abigalekim:ak/variant-binary-search-id

Conversation

@abigalekim

@abigalekim abigalekim commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

locate_object_field maps an integer dictionary ID to the encoded bytes of a field value within a Parquet Variant object blob. This function previously did a linear scan over all field IDs to find the matching entry. This PR implements binary search for locate_object_field when the Variant metadata's sorted_strings bit is set, which guarantees that field IDs within an object are sorted.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@abigalekim
abigalekim requested a review from a team as a code owner August 12, 2026 19:29
@abigalekim
abigalekim requested review from vyasr and wence- August 12, 2026 19:29
@copy-pr-bot

copy-pr-bot Bot commented Aug 12, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@abigalekim
abigalekim marked this pull request as draft August 12, 2026 19:29
@github-actions github-actions Bot added the libcudf Affects libcudf (C++/CUDA) code. label Aug 12, 2026
@abigalekim abigalekim added 3 - Ready for Review Ready for review by team improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Aug 12, 2026
@abigalekim
abigalekim marked this pull request as ready for review August 12, 2026 20:21
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8f5f87cb-2b21-4310-8dee-c02b27363abd

📥 Commits

Reviewing files that changed from the base of the PR and between 481e42a and e0fe68d.

📒 Files selected for processing (1)
  • cpp/src/io/parquet/experimental/variant_extract.cu

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes
    • Improved field lookup when reading variant data with sorted object fields.
    • Preserved support for unsorted object fields while maintaining validation for bounds and read failures.
    • Enhanced path resolution to interpret object metadata correctly.

Walkthrough

The change adds sortedness-aware object field lookup for Parquet variant extraction. Sorted field IDs use binary search. Unsorted field IDs use linear scanning. Path resolution reads sortedness metadata and passes it to the lookup.

Changes

Variant object path resolution

Layer / File(s) Summary
Sorted-aware object field lookup and path wiring
cpp/src/io/parquet/experimental/variant_extract.cu
locate_object_field selects binary search or linear scanning from is_sorted. Both paths validate reads and offset bounds. resolve_path forwards the metadata sortedness flag.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested reviewers: vyasr, wence-, vuule

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the binary search optimization for Parquet Variant object field ID lookup.
Description check ✅ Passed The description explains the binary search change, its sortedness condition, and the completed testing and documentation checks.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@abigalekim abigalekim changed the title (draft) binary search variant extract Binary search optimization for Parquet Variant's object field ID lookup Aug 12, 2026
@abigalekim
abigalekim requested a review from vuule August 12, 2026 20:24
@abigalekim

Copy link
Copy Markdown
Contributor Author

/ok to test 3d840b7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3 - Ready for Review Ready for review by team improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant