Skip to content

Partitioning: Use pyiceberg_core transform for all pyarrow transforms #1738

@kevinjqliu

Description

@kevinjqliu

Feature Request / Improvement

We expose pyiceberg_core's transform functions (powered by iceberg-rust) for partition transform
#1074 and #1345 added this functionality for bucket and truncate transform.

Let's expand to that include all other transformation types.

The assumption is that pyiceberg_core will be faster, but we should benchmark it

cc @sungwy

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions