perf(txn): reduce per-commit allocations on the write path#2307
Open
shaunpatterson wants to merge 1 commit into
Open
perf(txn): reduce per-commit allocations on the write path#2307shaunpatterson wants to merge 1 commit into
shaunpatterson wants to merge 1 commit into
Conversation
Replace pendingWrites map[string]*Entry with a fingerprint-keyed store (pendingEntries: map[uint64][]*Entry with byte-compare collision buckets). This removes the per-write string(e.Key) allocation that Go performs on every map insert. Get/read-your-writes, last-write-wins, managed duplicate-version handling and iteration semantics are preserved. Carve all key+ts buffers at commit from a single per-commit arena (keyArena) instead of one y.KeyWithTs allocation per key. The arena copies user keys (never aliases them) and is kept alive by the Entry.Key slices handed to the async write channel. Add white-box and end-to-end regression tests and BenchmarkManagedCommit (-benchmem) demonstrating the allocs/op reduction. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NtGkC4K2J2XYwcAKwjhHbM
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Two allocations fire on every write/commit.
pendingWrites map[string]*Entryallocates a string per insert (m[string(key)] = eis not elided by the compiler, only the lookup conversion is), andcommitAndSendallocates a freshy.KeyWithTsbuffer per key. On a high-throughput managed write path this is steady GC pressure.What changed
pendingWritesbecomes a fingerprint-keyed store (map[uint64][]*Entrykeyed byz.MemHash, with byte-compare collision buckets) — no per-insert string allocation. Read-your-writes, last-write-wins for same(key, version), and the managed multi-version →duplicateWritesrule are preserved exactly.keyWithTs) instead of onemakeper key; output is byte-identical toy.KeyWithTs, cap-clamped so slices can't corrupt each other, and kept alive by theEntry.Keyreferences handed to the write channel.Compatibility & correctness
Semantics are unchanged for all update transactions (default and managed); conflict detection (
conflictKeys) is a separate, untouched map. The representation is fully encapsulated intxn.go(no external readers; iteration order was already unspecified).-benchmem: ~1 fewer alloc per write at realistic batch sizes (e.g. 1187→1061 allocs/op at 128 keys/commit).Testing
go build,go vet, fullgo test . -count=1green under-race. Tests cover collision buckets, same/different-version dedup, a range-vs-map oracle over thousands of ops, read-your-writes (incl. deletes), managed multi-version round-trips, conflict detection, the sorted pending-writes iterator, andkeyWithTsbyte-identity/non-aliasing.🤖 Generated with Claude Code
https://claude.ai/code/session_01NtGkC4K2J2XYwcAKwjhHbM