WIP: Add support for all ClickHouse builtin functions, QUALIFY and WITH FILL FROM/TO/STEP#2357
Open
nastynaz wants to merge 15 commits intoquarylabs:mainfrom
Open
WIP: Add support for all ClickHouse builtin functions, QUALIFY and WITH FILL FROM/TO/STEP#2357nastynaz wants to merge 15 commits intoquarylabs:mainfrom
nastynaz wants to merge 15 commits intoquarylabs:mainfrom
Conversation
Extend ClickHouse WITH FILL grammar so STEP accepts a full expression rather than only numeric or INTERVAL literals. Add sqruff dialect fixtures for WITH FILL TO ... STEP ..., WITH FILL FROM ... STEP ..., and WITH FILL FROM ... TO ... STEP ... to prevent regressions. Regenerate the existing sqlfluff order_by_with_fill AST fixture to reflect STEP values now being parsed as expression nodes.
Add a ClickHouse-specific CP03 path that fixes mapped built-in functions to canonical case and avoids unsafe rewrites for unknown functions. Introduce a canonical function casing map (seeded with toInterval* functions) and add CP03 fixture coverage for default/lower/upper policies plus mixed known/unknown function calls.
Complete the first ClickHouse function batch (seq_id 1..50) with parser and CP03 test coverage. - add batch parser fixture covering every function row in batch-0001 - extend ClickHouse CP03 canonical function casing map and fixture cases - fix ClickHouse tuple-of-interval function argument parsing in dialect grammar - refresh impacted ClickHouse dialect snapshots Validation: - cargo test --package sqruff-lib --test rules -- CP03.yml - cargo test --package sqruff-lib-dialects --test dialects -- clickhouse - clickhouse-client EXPLAIN SYNTAX on tuple interval examples
Merging this PR will not alter performance
Comparing 
Footnotes
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I'm a big user of sqlfluff and got tired with the lack of ClickHouse support so I decided to implement the missing parts that are most valuable to me.
This PR introduces:
Support for CilckHouse QUALIFY
Filters window queries and runs after
HAVINGSupport for ClickHouse WITH FILL FROM / WITH FILL TO / STEP
Used for filling in timeseries data.
Support for all builtin ClickHouse functions (WIP. Progress: 500/1618 added)
These are all the functions from the system.functions table along with their aliases and variants.
Some of these are case sensitive whilst others are case in-sensitive. For the case sensitive variants I've made sure that capitalisation policies won't automatically 'fix' them to incorrect values. Case insensitive functions still adhere to the capitalisation policy.
Summary
I'm making this a WIP PR whilst I finish the rest of the functions so that I can get feedback and make sure everything adheres to the coding standards here. Once this PR is merged
sqruffwill support much more of the ClickHouse dialect thatsqlfluff.It takes me a long time to check all of the aliases and variants to make sure I haven't missed any, so this might take a couple of weeks until it's done. Any feedback during this time would be much appreciated.
I also have helper Python scripts that can generate support for functions that are added in the future. I've left them out of this PR but if you think it's helpful I can add them.