-
Notifications
You must be signed in to change notification settings - Fork 14.9k
KAFKA-19782: improve Authorizer by using (Patricia) trie #20911
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: trunk
Are you sure you want to change the base?
Conversation
|
Hi seems we make parallel changes :) on the same day Maybe we should keep just one branch, and I'll merge the benchmarks into it and fix the bug with them? |
|
Also take a look that this PR connected with big problem with benchmarks
I am stuck at this more than a month |
Small world :) Sure, happy to collaborate. I'll take a look at your benchmark changes. Once ready, are you able to push those benchmark changes to this branch? Otherwise I am also happy to merge your benchmark changes. Let me know! |
|
@lucasbru We accidentally created two PRs for the same task, and we’ll actually be working together on it going forward. Could you coordinate us a bit and take a look at both PRs? One PR introduces a PatriciaTrie-based Set on top of PatriciaTrie, - #20911 (this pr) In your opinion, which option is better to use? Both approaches work. By the way, I’ve already written to a colleague from the library Apache common collections and suggested adding a PatriciaTrie Set, but he’s very busy and hasn’t responded yet - apache/commons-collections#589 |
|
A label of 'needs-attention' was automatically added to this PR in order to raise the |
|
A label of 'needs-attention' was automatically added to this PR in order to raise the |
|
A label of 'needs-attention' was automatically added to this PR in order to raise the |
This PR address KAFKA-19782 by replacing hashset with trie for prefix search.
Previously the Authorizer uses character-by-character comparisons to search for prefix patterns to deny within a set of allowed literals. This prefix search is suboptimal when the set contains many strings that share the same prefix -- the complexity is linear in number of strings sharing a prefix. By constrast, a trie will reduce the complexity to constant.
In particular, this PR uses Apache Commons Collection's PATRICIA Trie, which is an efficient algorithm that offers worst-case
O(k)-time operations, wherekis the largest key bit-length.The resulting code appears also clearer in logic.