experiment(tag_filterlist): try trie-based tag membership set approach#1304
experiment(tag_filterlist): try trie-based tag membership set approach#1304tobz wants to merge 10 commits intotobz/rayz/tagfilter-implementation-tagset-mut-viewfrom
Conversation
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
There was a problem hiding this comment.
Pull request overview
This PR introduces a trie-based approach for tag membership checking in the metric tag filterlist component. Instead of using HashSet-based lookups, it uses the trie-hard library to build a trie data structure for O(log n) prefix-based lookups. The change includes refactoring the core filtering logic, adding comprehensive benchmarks, and introducing a new TagNameSet struct that wraps the trie with unsafe code to maintain static lifetimes.
Changes:
- Introduces
TagNameSetstruct usingtrie-hardfor tag name membership checking with artificial'staticlifetime management - Converts
compile_filtersto use a two-pass approach: first merge with HashSets, then convert to tries - Adds comprehensive benchmarks comparing three approaches (std HashSet, foldhash HashSet, and trie) across different tag set sizes and lookup patterns
- Updates dependencies to include
trie-hard 0.2andcriterionfor benchmarking
Reviewed changes
Copilot reviewed 2 out of 6 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| LICENSE-3rdparty.csv | Adds Apache-2.0 licensed trie-hard library from Cloudflare |
| Cargo.toml | Adds trie-hard 0.2 dependency to workspace |
| Cargo.lock | Adds trie-hard 0.2.0 and related phf dependencies |
| bin/agent-data-plane/Cargo.toml | Adds trie-hard and criterion dependencies to agent-data-plane |
| bin/agent-data-plane/src/components/tag_filterlist/mod.rs | Implements TagNameSet with trie-based lookups, refactors compile_filters, and updates filter_metric_tags function |
| bin/agent-data-plane/benches/tag_filterlist.rs | New benchmark file comparing performance of different tag lookup implementations |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
4e4254f to
11e8387
Compare
1bfb364 to
b8b7477
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 7 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| chumsky = { version = "0.12", default-features = false } | ||
| logos = { version = "0.16", default-features = false } | ||
| lru-slab = { version = "0.1.2", default-features = false } | ||
| trie-hard = { git = "https://github.com/tobz/trie-hard.git", branch = "tobz/optimize", default-features = false } |
There was a problem hiding this comment.
This PR adds a dependency on trie-hard from a custom git branch (tobz/optimize). Using a git branch dependency instead of a published release version can create reproducibility and maintenance issues. Consider either publishing this as a proper release, using a specific commit hash, or waiting for the changes to be merged upstream and released officially.
| trie-hard = { git = "https://github.com/tobz/trie-hard.git", branch = "tobz/optimize", default-features = false } | |
| trie-hard = { git = "https://github.com/tobz/trie-hard.git", rev = "<current-commit-sha>", default-features = false } |
| ] } | ||
| snafu = { version = "0.9", default-features = false, features = ["std"] } | ||
| tokio = { version = "1.50", default-features = false } | ||
| tokio = { version = "1.49", default-features = false } |
There was a problem hiding this comment.
The tokio version has been downgraded from 1.50 to 1.49. This appears to be intentional, but the PR description is empty and doesn't explain why. Verify that this downgrade is intentional and doesn't cause any compatibility issues with other dependencies.
| ndarray-stats = { version = "0.6", default-features = false } | ||
| noisy_float = { version = "0.2", default-features = false } | ||
| libc = { version = "0.2.183", default-features = false } | ||
| libc = { version = "0.2.169", default-features = false } |
There was a problem hiding this comment.
The libc version has been downgraded from 0.2.183 to 0.2.169. The PR description is empty and doesn't explain why this downgrade was made. Verify that this is intentional and understand the reason for this version change.
| [build] | ||
| rustflags = ["--cfg", "tokio_unstable"] | ||
| rustflags = "--cfg tokio_unstable" | ||
|
|
||
| [target.x86_64-unknown-linux-gnu] | ||
| rustflags = "--cfg tokio_unstable -C target-feature=+sse,+sse2,+sse3,+sse4.1,+sse4.2,+popcnt" | ||
|
|
||
| [target.x86_64-unknown-linux-musl] | ||
| rustflags = "--cfg tokio_unstable -C target-feature=+sse,+sse2,+sse3,+sse4.1,+sse4.2,+popcnt" |
There was a problem hiding this comment.
The .cargo/config.toml has been significantly modified: the rustflags format changed from an array to a string, and target-specific rustflags with SSE features were added. The PR description is empty and doesn't explain these infrastructure changes. Verify that these changes are intentional and compatible with the build system.
11e8387 to
be3ee86
Compare
b8b7477 to
3bd63f3
Compare
3bd63f3 to
7a3fd5e
Compare

Summary
Change Type
How did you test this PR?
References