Skip to content

experiment(tag_filterlist): try trie-based tag membership set approach#1304

Draft
tobz wants to merge 10 commits intotobz/rayz/tagfilter-implementation-tagset-mut-viewfrom
tobz/tag-filterlist-trie-tagnameset
Draft

experiment(tag_filterlist): try trie-based tag membership set approach#1304
tobz wants to merge 10 commits intotobz/rayz/tagfilter-implementation-tagset-mut-viewfrom
tobz/tag-filterlist-trie-tagnameset

Conversation

@tobz
Copy link
Copy Markdown
Member

@tobz tobz commented Apr 2, 2026

Summary

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

How did you test this PR?

References

Copy link
Copy Markdown
Member Author

tobz commented Apr 2, 2026

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a trie-based approach for tag membership checking in the metric tag filterlist component. Instead of using HashSet-based lookups, it uses the trie-hard library to build a trie data structure for O(log n) prefix-based lookups. The change includes refactoring the core filtering logic, adding comprehensive benchmarks, and introducing a new TagNameSet struct that wraps the trie with unsafe code to maintain static lifetimes.

Changes:

  • Introduces TagNameSet struct using trie-hard for tag name membership checking with artificial 'static lifetime management
  • Converts compile_filters to use a two-pass approach: first merge with HashSets, then convert to tries
  • Adds comprehensive benchmarks comparing three approaches (std HashSet, foldhash HashSet, and trie) across different tag set sizes and lookup patterns
  • Updates dependencies to include trie-hard 0.2 and criterion for benchmarking

Reviewed changes

Copilot reviewed 2 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
LICENSE-3rdparty.csv Adds Apache-2.0 licensed trie-hard library from Cloudflare
Cargo.toml Adds trie-hard 0.2 dependency to workspace
Cargo.lock Adds trie-hard 0.2.0 and related phf dependencies
bin/agent-data-plane/Cargo.toml Adds trie-hard and criterion dependencies to agent-data-plane
bin/agent-data-plane/src/components/tag_filterlist/mod.rs Implements TagNameSet with trie-based lookups, refactors compile_filters, and updates filter_metric_tags function
bin/agent-data-plane/benches/tag_filterlist.rs New benchmark file comparing performance of different tag lookup implementations

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings April 2, 2026 19:34
@tobz tobz force-pushed the tobz/tag-filterlist-trie-tagnameset branch from 4e4254f to 11e8387 Compare April 2, 2026 19:34
@tobz tobz force-pushed the tobz/rayz/tagfilter-implementation-tagset-mut-view branch from 1bfb364 to b8b7477 Compare April 2, 2026 19:34
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 7 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

chumsky = { version = "0.12", default-features = false }
logos = { version = "0.16", default-features = false }
lru-slab = { version = "0.1.2", default-features = false }
trie-hard = { git = "https://github.com/tobz/trie-hard.git", branch = "tobz/optimize", default-features = false }
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR adds a dependency on trie-hard from a custom git branch (tobz/optimize). Using a git branch dependency instead of a published release version can create reproducibility and maintenance issues. Consider either publishing this as a proper release, using a specific commit hash, or waiting for the changes to be merged upstream and released officially.

Suggested change
trie-hard = { git = "https://github.com/tobz/trie-hard.git", branch = "tobz/optimize", default-features = false }
trie-hard = { git = "https://github.com/tobz/trie-hard.git", rev = "<current-commit-sha>", default-features = false }

Copilot uses AI. Check for mistakes.
] }
snafu = { version = "0.9", default-features = false, features = ["std"] }
tokio = { version = "1.50", default-features = false }
tokio = { version = "1.49", default-features = false }
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tokio version has been downgraded from 1.50 to 1.49. This appears to be intentional, but the PR description is empty and doesn't explain why. Verify that this downgrade is intentional and doesn't cause any compatibility issues with other dependencies.

Copilot uses AI. Check for mistakes.
ndarray-stats = { version = "0.6", default-features = false }
noisy_float = { version = "0.2", default-features = false }
libc = { version = "0.2.183", default-features = false }
libc = { version = "0.2.169", default-features = false }
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The libc version has been downgraded from 0.2.183 to 0.2.169. The PR description is empty and doesn't explain why this downgrade was made. Verify that this is intentional and understand the reason for this version change.

Copilot uses AI. Check for mistakes.
Comment on lines 1 to +8
[build]
rustflags = ["--cfg", "tokio_unstable"]
rustflags = "--cfg tokio_unstable"

[target.x86_64-unknown-linux-gnu]
rustflags = "--cfg tokio_unstable -C target-feature=+sse,+sse2,+sse3,+sse4.1,+sse4.2,+popcnt"

[target.x86_64-unknown-linux-musl]
rustflags = "--cfg tokio_unstable -C target-feature=+sse,+sse2,+sse3,+sse4.1,+sse4.2,+popcnt"
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The .cargo/config.toml has been significantly modified: the rustflags format changed from an array to a string, and target-specific rustflags with SSE features were added. The PR description is empty and doesn't explain these infrastructure changes. Verify that these changes are intentional and compatible with the build system.

Copilot uses AI. Check for mistakes.
@tobz tobz force-pushed the tobz/tag-filterlist-trie-tagnameset branch from 11e8387 to be3ee86 Compare April 3, 2026 14:44
@tobz tobz force-pushed the tobz/rayz/tagfilter-implementation-tagset-mut-view branch from b8b7477 to 3bd63f3 Compare April 3, 2026 14:44
@rayz rayz force-pushed the tobz/rayz/tagfilter-implementation-tagset-mut-view branch from 3bd63f3 to 7a3fd5e Compare April 7, 2026 19:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants