Skip to content

feat(dursto): Add structs for lazy-loading of Merkle Layer#868

Merged
NSant215 merged 2 commits intomainfrom
santnr/lazy-loading
Mar 11, 2026
Merged

feat(dursto): Add structs for lazy-loading of Merkle Layer#868
NSant215 merged 2 commits intomainfrom
santnr/lazy-loading

Conversation

@NSant215
Copy link
Copy Markdown
Contributor

@NSant215 NSant215 commented Mar 4, 2026

Part of RV-892.

What

  • enables AVL nodes to be loaded on-demand from a persistence-layer.
  • Added LazyNodeId and LazyResolver types to support lazy loading and caching of AVL nodes from a key-value store, including implementation of Resolver for LazyNodeId and methods for resolving nodes and hashes on demand.
  • Implemented serialization and deserialization for AVL tree root representations using NodeHashRepresentation, including methods for serializing the root and constructing a tree from serialized data.
  • Added Decode and BorrowDecode implementations for Bytes<Normal> to support owned and borrowed deserialization, and derived Decode for BytesTemplate.

Why

  • necessary steps for adding checkout capability for durable storage.

Manually Testing

make all

Tasks for the Author

  • Link all Linear issues related to this MR using magic words (e.g. part of, relates to, closes).
  • Eliminate dead code and other spurious artefacts introduced in your changes.
  • Document new public functions, methods and types.
  • Make sure the documentation for updated functions, methods, and types is correct.
  • Add tests for bugs that have been fixed.
  • Explain changes to regression test captures when applicable.
  • Write commit messages in agreement with our guidelines.
  • Self-review your changes to ensure they are high-quality.
  • Complete all of the above before assigning this MR to reviewers.

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 4, 2026

Codecov Report

❌ Patch coverage is 0% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.89%. Comparing base (2269509) to head (641f777).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
data/src/components/bytes.rs 0.00% 10 Missing ⚠️

❌ Your patch status has failed because the patch coverage (0.00%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #868      +/-   ##
==========================================
- Coverage   90.96%   90.89%   -0.08%     
==========================================
  Files         112      112              
  Lines       21493    21503      +10     
  Branches    21493    21503      +10     
==========================================
- Hits        19552    19545       -7     
- Misses       1561     1578      +17     
  Partials      380      380              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@NSant215 NSant215 force-pushed the santnr/lazy-loading branch 10 times, most recently from 82bc7c2 to 79784a4 Compare March 4, 2026 17:32
@NSant215 NSant215 marked this pull request as ready for review March 4, 2026 17:37
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 4, 2026

Benchmark results for revision ed4d828:

Metric Duration TPS
Mean 1.530089979s 26.143
Worst 1.538756439s 25.995
Best 1.516288565s 26.380
Standard Deviation ±5.942137ms ±0.102
Full results
Run Transfers Duration TPS
1 40 1.537627345s 26.014
2 40 1.530647873s 26.133
3 40 1.538756439s 25.995
4 40 1.529561545s 26.151
5 40 1.532902881s 26.094
6 40 1.533415139s 26.086
7 40 1.525628159s 26.219
8 40 1.535356531s 26.053
9 40 1.521583161s 26.288
10 40 1.529014584s 26.161
11 40 1.538088698s 26.006
12 40 1.530801283s 26.130
13 40 1.518475714s 26.342
14 40 1.535172151s 26.056
15 40 1.528757161s 26.165
16 40 1.529167619s 26.158
17 40 1.52778618s 26.182
18 40 1.533196023s 26.089
19 40 1.529572528s 26.151
20 40 1.516288565s 26.380

Compare the results above with those for the default branch.

Comment thread data/src/components/bytes.rs Outdated
Comment thread durable-storage/benches/avl_tree.rs Outdated
Comment thread data/src/components/bytes.rs Outdated
Comment thread durable-storage/src/avl/node.rs Outdated
Comment thread durable-storage/src/avl/node.rs Outdated
Comment thread durable-storage/src/avl/resolver.rs Outdated
Comment thread durable-storage/src/avl/resolver.rs Outdated
Comment thread durable-storage/src/avl/tree.rs Outdated
Comment thread durable-storage/src/merkle_worker.rs Outdated
Comment thread durable-storage/src/merkle_worker.rs Outdated
@NSant215 NSant215 force-pushed the santnr/lazy-loading branch 8 times, most recently from 5f247df to b6417fa Compare March 5, 2026 10:09
@NSant215 NSant215 requested a review from vapourismo March 5, 2026 10:56
@NSant215 NSant215 requested a review from vapourismo March 10, 2026 10:05
@NSant215 NSant215 force-pushed the santnr/lazy-loading branch from 80e8ff9 to d187feb Compare March 10, 2026 10:48
@NSant215 NSant215 enabled auto-merge March 10, 2026 11:06
Comment thread durable-storage/src/avl/resolver.rs
Comment thread durable-storage/src/avl/resolver.rs Outdated
@NSant215 NSant215 force-pushed the santnr/lazy-loading branch 4 times, most recently from 6e435d7 to 2e3647b Compare March 10, 2026 17:40
Comment thread durable-storage/src/avl/resolver.rs Outdated
Comment thread durable-storage/src/avl/node.rs Outdated
Comment thread durable-storage/src/avl/resolver.rs Outdated
Comment thread durable-storage/src/avl/resolver.rs Outdated
Comment thread durable-storage/src/avl/resolver.rs Outdated
Comment thread durable-storage/src/avl/resolver.rs Outdated
Comment thread durable-storage/src/avl/resolver.rs Outdated
Comment thread durable-storage/src/avl/tree.rs Outdated
@NSant215 NSant215 force-pushed the santnr/lazy-loading branch 8 times, most recently from 2996563 to efd9f09 Compare March 11, 2026 09:17
@NSant215 NSant215 requested a review from vapourismo March 11, 2026 09:18
@NSant215 NSant215 force-pushed the santnr/lazy-loading branch from efd9f09 to 4afb690 Compare March 11, 2026 09:23
Comment thread durable-storage/src/avl/resolver.rs Outdated
@NSant215 NSant215 force-pushed the santnr/lazy-loading branch from 4afb690 to 641f777 Compare March 11, 2026 11:58
@NSant215 NSant215 requested a review from vapourismo March 11, 2026 12:02
@NSant215 NSant215 added this pull request to the merge queue Mar 11, 2026
Merged via the queue into main with commit 6216c2c Mar 11, 2026
7 of 8 checks passed
@NSant215 NSant215 deleted the santnr/lazy-loading branch March 11, 2026 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants