fix(whatsmeow): reuse shared sqlstore.Container instead of recreating it on every reconnect - #131
Open
wellpelomundo wants to merge 1 commit into
Conversation
… it on every reconnect StartClient() called sqlstore.New(...) fresh on every reconnection attempt (both the internal auto-reconnect "attempt 1/2, 2/2" retry and any external POST /instance/connect), opening a brand new Postgres connection pool each time via sql.Open. The previous container was never closed (no container.Close() call anywhere in the file), so every reconnect cycle left a whole pool (up to 25 connections, per the limits sqlstore.New already sets) orphaned against Postgres. Under frequent reconnection this exhausts max_connections even with a high limit configured, and restarting only the app (not Postgres) doesn't clear it since the leaked connections live on the database side. Cache a single shared *sqlstore.Container (same PostgresAuthDB/DSN for every instance, so one container per process is correct, not one per instance) in a new containerPointer map, created lazily on first use and reused by every subsequent StartClient() call. Uses the same map-as-reference-type pattern already established by clientPointer, so it works correctly even though StartClient (like most methods here) uses a value receiver. Guarded by a *sync.Mutex (pointer, not embedded value — go vet correctly flags an embedded sync.Mutex here as "passes lock by value" given the value-receiver pattern). Verified: `go build ./pkg/whatsmeow/...` and `go vet ./pkg/whatsmeow/...` both clean. Not able to run the full binary/integration build locally (unrelated CGO failure in github.com/chai2010/webp on this machine), so only the package containing the change was verified to compile — happy to run any additional checks the maintainers want. Observed in production (self-hosted, Postgres 16): this same `too many clients already` exhaustion pattern recurred 3 times over two weeks (max_connections raised 60->200 after the first occurrence, which only delayed recurrence). Support recommended idle_session_timeout + periodic restarts as an operational mitigation, which we've applied, but this fix addresses the actual leak at its source.
Reviewer's GuideReuses a single shared sqlstore.Container across all whatsmeow instances and reconnect attempts to prevent leaking database connection pools, by adding a lazily initialized, mutex-guarded container cache on the service struct and refactoring StartClient to obtain the container from this cache instead of creating a new one each time, plus a minor JSON payload formatting cleanup. Sequence diagram for StartClient container reuse and lazy initializationsequenceDiagram
participant whatsmeowService
participant containerPointer
participant sqlstore
participant Postgres
whatsmeowService->>containerPointer: read [sharedContainerKey]
alt container exists
containerPointer-->>whatsmeowService: *sqlstore.Container
else container missing
whatsmeowService->>whatsmeowService: containerInitMu.Lock()
whatsmeowService->>containerPointer: read [sharedContainerKey] (double check)
alt still missing
alt config.WaDebug != ""
whatsmeowService->>sqlstore: waLog.Stdout("Database", config.WaDebug, true)
end
alt config.PostgresAuthDB != ""
whatsmeowService->>sqlstore: sqlstore.New(context.Background(), "postgres", config.PostgresAuthDB, dbLog)
sqlstore->>Postgres: sql.Open("postgres", PostgresAuthDB)
else sqlite fallback
whatsmeowService->>sqlstore: sqlstore.New(context.Background(), "sqlite", dsn, dbLog)
end
whatsmeowService->>containerPointer: set [sharedContainerKey] = *sqlstore.Container
end
whatsmeowService->>whatsmeowService: containerInitMu.Unlock()
end
whatsmeowService-->>whatsmeowService: use shared *sqlstore.Container for client start
File-Level Changes
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- Access to containerPointer in StartClient is currently racy (read outside the mutex, write inside); consider wrapping both the initial read and subsequent write in the same lock or using sync.Once/atomic pointer to avoid data races under concurrent reconnects.
- Since you only ever use a single sharedContainerKey, a map may be unnecessary; replacing containerPointer with a single *sqlstore.Container field (initialized once with proper synchronization) would simplify the implementation and make the intent clearer.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Access to containerPointer in StartClient is currently racy (read outside the mutex, write inside); consider wrapping both the initial read and subsequent write in the same lock or using sync.Once/atomic pointer to avoid data races under concurrent reconnects.
- Since you only ever use a single sharedContainerKey, a map may be unnecessary; replacing containerPointer with a single *sqlstore.Container field (initialized once with proper synchronization) would simplify the implementation and make the intent clearer.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
StartClient()callssqlstore.New(...)fresh on every reconnectionattempt — both the internal auto-reconnect retry (
Client disconnected on attempt 1/2,2/2in the logs) and any externalPOST /instance/connect.Each call opens a brand new Postgres connection pool via
sql.Open. Theprevious container is never closed (no
container.Close()call anywherein the file), so every reconnect cycle leaves a whole pool (up to 25
connections, per the limits
sqlstore.Newalready sets internally)orphaned against Postgres.
Under frequent reconnection this exhausts
max_connectionseven with ahigh limit configured:
Restarting only the app container doesn't clear it, since the leaked
connections live on the database side — only restarting Postgres itself
recovers, until it happens again.
Observed in production
Self-hosted, Postgres 16. This exact exhaustion pattern recurred 3 times
over two weeks.
max_connectionswas raised 60→200 after the firstoccurrence, which only delayed the recurrence, not fixed it — consistent
with a genuine per-reconnect leak rather than a capacity issue. Support
recommended
idle_session_timeout+ periodic restarts as an operationalmitigation (applied, helps), but doesn't address the root cause.
Fix
Cache a single shared
*sqlstore.Containerin a newcontainerPointermap, created lazily on first use and reused by every subsequent
StartClient()call instead of recreating it. One container per processis correct here (not one per instance) since every instance shares the
same
PostgresAuthDBDSN.Uses the same map-as-reference-type pattern already established by
clientPointerin this file, so the cache works correctly even thoughStartClient(like most methods onwhatsmeowService) uses a valuereceiver — a plain (non-map) field would silently fail to persist across
calls. Guarded by a
*sync.Mutex(pointer, not an embedded value —go vetcorrectly flags an embeddedsync.Mutexhere as "passes lock byvalue" given the value-receiver pattern throughout this file).
Testing
go build ./pkg/whatsmeow/...— cleango vet ./pkg/whatsmeow/...— clean (no warnings, including the mutex-by-value check)unrelated CGO failure in
github.com/chai2010/webpon my machine, soonly the package containing the change was verified to compile. Happy
to run any additional checks the maintainers want, or adjust the
approach if there's a reason the container needs to be recreated per
attempt that isn't obvious from this file alone.
🤖 Generated with Claude Code
Summary by Sourcery
Reuse a single shared sqlstore.Container across all Whatsmeow instances and reconnect attempts to prevent database connection pool leaks on client restarts.
Enhancements: