Skip to content

Conversation

@WTFox
Copy link

@WTFox WTFox commented Aug 6, 2025

Note: This PR should not be merged. These changes already exist at sqlalchemy-redshift/sqlalchemy-redshift. Once that PR is merged, this can be closed.


Description

This pull request introduces support for two new column types from the Iceberg table format: string and binary. These types are increasingly common in data warehousing scenarios, and their inclusion in sqlalchemy-redshift will enhance compatibility with Iceberg tables and improve the user experience for those working with Redshift and Iceberg.

This PR is specifically tailored to add/improve interactions with Spectrum tables in Iceberg format.

Changes

  • Added a new custom type ICEBERG_STRING that maps Iceberg's string type to Redshift/Postgre's VARCHAR type with a default max length.
  • Added a new custom type ICEBERG_BINARY that maps Iceberg's binary type to SQLAlchemy's LargeBinary type.
  • Included unit tests
  • Updated the documentation to reflect the new type support.

Motivation

As Iceberg tables become more prevalent, it is important for sqlalchemy-redshift to evolve and support the diverse data types used in these tables. The lack of native support for string and binary types can lead to challenges when querying Iceberg tables, and this PR aims to address those challenges.

Backward Compatibility

These changes are fully backward-compatible as they introduce new types without altering the existing functionality of the dialect.

Additional Notes

I look forward to feedback from the maintainers and community on this PR. If there are any additional changes or enhancements needed, I am open to discussion and collaboration.

image

Todos

  • MIT compatible
  • Tests
  • Documentation
  • Updated CHANGES.rst

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants