[MeshSync] Fix data race on informer stores map - #592
Conversation
startDiscovery replaces h.stores wholesale on the discovery goroutine while handleInformerStoreRequest ranges it on the request-listener goroutine, an unsynchronized read/write of the map field. Guard it with storesMu (RWMutex) and route both accesses through replaceStores and snapshotStores, mirroring the sessions-map fix. snapshotStores copies the store slice under the read lock and releases it before any store List(), so a store read never blocks the next discovery swap. Signed-off-by: Harsh Singh <hs1663531@gmail.com>
There was a problem hiding this comment.
Code Review
This pull request introduces thread-safe access to the 'stores' map in the 'Handler' struct by adding a 'sync.RWMutex' ('storesMu') and implementing 'replaceStores' and 'snapshotStores' helper methods. These changes prevent race conditions between the discovery goroutine, which updates the stores, and the request listener goroutine, which reads them. Additionally, a new test file 'meshsync/stores_test.go' has been added to verify concurrent access and correct snapshot behavior. I have no further feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
📝 WalkthroughWalkthroughThe handler now protects discovery store replacement with an RWMutex, snapshots stores before listing objects, and adds tests for concurrent access, complete snapshots, and nil store maps. ChangesStore concurrency
Estimated code review effort: 3 (Moderate) | ~20 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
Friendly ping on this — it's been open a couple of weeks and CI is green (DCO, CodeRabbit, triage all passing). It's a small change: guards the informer |
What's going on
Handler.stores— the per-GVR informer store set — gets replaced wholesale bystartDiscoveryon the discovery goroutine every time (re)discovery runs, and it's read on a different goroutine byhandleInformerStoreRequest→listStoreObjectswhenever Meshery Server asks for aninformer-store. Those two run at the same time with nothing synchronizing them, so the field is read and written with no happens-before between them — a data race, andgo test -raceflags it. It's the same shape as the bug already fixed for the exec/log-streamsessionsmap.What I changed
I put an
RWMutex(storesMu) around thestoresfield. The wholesale swap now goes throughreplaceStoresunder the write lock, and reads go throughsnapshotStores, which copies the current store values under the read lock and hands those back — so the lock is never held acrosscache.Store.List().listStoreObjectsiteratessnapshotStores()instead of the raw map.Nothing exported changes — no API, wire format, CRD, or config. Access is read-mostly, which is why
RWMutex, and the set of objects returned is exactly what it was before.Testing
Added
meshsync/stores_test.go.TestStoresConcurrentAccessruns 16 goroutines racing replace against snapshot; it passes under-race, and it reports the race if you remove the guard, so it actually catches the regression. There are also completeness and nil-map cases.go test --short ./... -raceis green across all packages, andgo vet/gofmtare clean.Why send it on its own
It's a prerequisite for the periodic-reconciliation work (
docs/design/fd5-periodic-reconciliation.md), which adds a timer-driven reconcile loop that would be the second concurrent reader ofh.storesalongside the informer-store path. I'm keeping it separate so that feature builds on a race-free store instead of bundling the fix into it.Summary by CodeRabbit
Bug Fixes
Tests