Fix nested namespace encoding in REST catalog for Iceberg #1268
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add encodeNamespaceForURI function to properly encode multi-level namespaces using the unit separator (%1F) as required by the Iceberg REST Catalog API specification. This fixes the issue where tables in nested namespaces (e.g., teslasource.kafkav6.source-data) were not discoverable because ClickHouse was using dot separators in the URL path instead of %1F. Fixes: Tables in hierarchical namespaces return 404 errors Reference: https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes an issue where tables in hierarchical/nested Iceberg namespaces are not discoverable when using the REST catalog (e.g., Snowflake Polaris).
Problem
When querying tables in nested namespaces like
teslasource.kafkav6.source-data, ClickHouse constructs the URL path incorrectly:According to the Iceberg REST Catalog OpenAPI spec, multi-level namespaces should be joined with the unit separator character (
0x1F) which is URL-encoded as%1F.The current implementation uses a dot (
.) separator, causing the Polaris API to return:{"error":{"message":"Namespace does not exist: teslasource.kafkav6","type":"NoSuchNamespaceException","code":404}}Changes
encodeNamespaceForURI()function that replaces.with%1Fin namespace pathsgetTables()when constructing the tables endpointgetTableMetadataImpl()when constructing the table metadata endpointTesting
Tested with Snowflake Polaris catalog containing 117 tables across 15 namespaces with nested sub-namespaces (e.g.,
azugasource.kafkav2,teslasource.kafkav6).Before fix:
SHOW TABLESreturns 0 tables (all 404 errors in logs)After fix: Tables in nested namespaces are discoverable
Related
src/Databases/DataLake/RestCatalog.cpp