feat: allow for accountsdb checksum computation

[bmuddha]

Summary

Adds method to compute the checksum of current state of accountsdb.
The checksum computation is performed after acquisition of global lock.

Compatibility

• No breaking changes

Testing

• 2 new tests are added

Checklist

• closes feat: add checksum computation to AccountsDb snapshots #957

Summary by CodeRabbit

• New Features

  • Added a checksum to verify account database integrity and consistency.

• Improvements

  • Updated dependencies to newer revisions and added a hashing dependency to support deterministic checksums.
  • Refined database size reporting to reflect actual used data instead of allocated capacity.

• Tests

  • Added tests ensuring checksum determinism and detecting state changes.

[bmuddha]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat: forward original bincoded transaction #991
• feat: allow for accountsdb checksum computation #990 👈 (View in Graphite)
• feat: modify account cloning to use transactions #974
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds AccountsDb::checksum() returning a deterministic u64 by acquiring the write lock, iterating all accounts in key-sorted order, and hashing pubkey bytes plus serialized account data using xxhash3. Adds twox-hash dependency and unit tests for determinism and state-change detection. Also changes storage.size_bytes() to report active data segment length instead of allocated capacity.

Assessment against linked issues

Objective	Addressed	Explanation
New method on AccountsDb to compute state checksum [#957]	✅
Scans all active accounts and hashes them cumulatively [#957]	✅
Returns single hash value (u64) [#957]	✅
Unit tests for correctness and determinism [#957]	✅
Integrates with existing snapshot flow [#957]	❌	Implementation does not integrate checksum into snapshot flow; no snapshot hooks or calls added.

Out-of-scope changes

Code Change	Explanation
Changed storage.size_bytes() to return active_segment().len() (magicblock-accounts-db/src/storage.rs)	This alters existing method semantics from reporting allocated capacity to reporting used payload size; unrelated to checksum objective in #957.

Suggested reviewers

• thlorenz
• GabrielePicco

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch bmuddha/accountsdb/checksum

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@magicblock-accounts-db/src/lib.rs`:
- Around line 386-397: Change checksum() to return AccountsDbResult<u64> (not
u64), propagate any LMDB/index errors instead of swallowing them, and surface
failures from iter_all() rather than treating an empty iterator as success;
specifically update checksum() to call the underlying index/get_all_accounts
path in a fallible way and return Err(...) on any index/LMDB error. While
iterating accounts inside checksum(), do not silently skip Owned accounts: when
acc.as_borrowed() is None, emit a tracing::error! (including the pubkey) and
return an Err variant indicating an unexpected Owned account (so callers see the
failure). Keep the existing hashing logic (xxhash3_64::Hasher) for Borrowed
accounts only, but ensure all error paths map to AccountsDbResult and are
returned to the caller.

In `@magicblock-accounts-db/src/storage.rs`:
- Around line 374-377: The current size_bytes() change breaks reload() because
it uses active_segment().len() (used bytes) to grow the snapshot file, shrinking
capacity after rollback; update reload() to ensure the file is grown to the full
mmap extent instead of calling size_bytes(): call ensure_file_size(...) with the
full configured size (compute from the configured capacity_blocks × block_size +
METADATA_STORAGE_SIZE or use the stored mmap extent/initial file size) so
map_file recomputes capacity_blocks from the original full allocation; keep
size_bytes() semantics if it must represent used bytes, but do not use it in
reload(); also adjust or add a test (e.g. test_db_size_after_rollback) to
exercise an actual rollback (target_slot < current slot) to verify capacity is
preserved.

In `@magicblock-accounts-db/src/tests.rs`:
- Around line 513-521: The second assertion is comparing env.checksum() to
original_checksum even though accounts[5] was already mutated, so save the
intermediate checksum immediately after inserting accounts[5] (e.g., let
intermediate_checksum = env.checksum()) and then modify
accounts[10].1.set_lamports(...) and env.insert_account(&accounts[10].0,
&accounts[10].1).unwrap(); finally assert_ne!(env.checksum(),
intermediate_checksum, "checksum must detect lamport change") to ensure the
lamport mutation on accounts[10] is the change being tested.

[coderabbitai]

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@magicblock-accounts-db/src/lib.rs`:
- Around line 386-397: Change checksum() to return AccountsDbResult<u64> and
propagate LMDB/iteration errors instead of silently returning an empty checksum:
call iter_all() and if it returns Err (originating from
self.index.get_all_accounts()) return that error wrapped with context; when
iterating, treat acc.as_borrowed() returning None as an error (do not continue
silently) and return a descriptive AccountsDbError indicating a failed account
borrow/unsupported Owned variant (include pubkey info). Keep using
xxhash3_64::Hasher but only finish and Ok(hasher.finish()) on success; update
callers to handle the AccountsDbResult<u64>.

In `@magicblock-accounts-db/src/storage.rs`:
- Around line 374-377: size_bytes() now returns only the active-used portion
(cursor*block_size + METADATA_STORAGE_SIZE) which causes reload() (which calls
ensure_file_size and then map_file that computes capacity_blocks = (file_size -
METADATA_STORAGE_SIZE) / block_size) to see a reduced file_size after rollbacks
and incorrectly shrink capacity; fix reload() so it uses the actual on-disk file
length (e.g., via file metadata or the raw segment file length) when calling
ensure_file_size/map_file instead of self.size_bytes(), leaving size_bytes()
semantics unchanged; update references in reload(), ensure_file_size/map_file
invocation, and any logic around capacity_blocks/METADATA_STORAGE_SIZE to read
the real file size rather than active_segment()/size_bytes().

In `@magicblock-accounts-db/src/tests.rs`:
- Around line 518-527: The test's second assertion doesn't isolate the lamport
change because env.checksum() already diverged after mutating and committing
accounts[5]; capture the checksum immediately after the accounts[5] commit
(e.g., let checksum_after_commit = env.checksum()) and then mutate accounts[10]
via accounts[10].1.set_lamports(...) and env.insert_account(...), then
assert_ne!(env.checksum(), checksum_after_commit) to ensure the assertion
specifically tests the lamports change; reference env.checksum,
accounts[10].1.set_lamports, env.insert_account and
original_checksum/accounts[5] to locate the relevant lines.