fix(sql): close the opened client on New's post-open error paths (connection-handle leak) - #10
Merged
Conversation
The mysql, postgres and sqlite `New` constructors open a `*sqlx.DB` with `sqlx.Open` (which does not dial), then verify liveness with a retried `Ping` and run `validation.Validate`. On the ping-failure and validation-failure returns the opened handle was neither returned to the caller nor closed, leaking the underlying `database/sql` connection pool every time construction failed after Open (e.g. an unreachable DB). Guard the opened client with a single deferred close controlled by a `success` flag that only flips true immediately before the successful return. Every post-open error path (ping, validation, and any future one) now closes the client exactly once; the success path does not. Behavior is otherwise unchanged: signatures, the success path, and error values are identical. Add a per-package constructor test that drives the ping-failure path against an injected, in-process counting `database/sql` driver (Ping always fails; Open/Close increment atomic counters) and asserts the opened handle was closed (closes >= opens), proving the leak is gone. The test is fully offline (no real DB, no network), shrinks the ping backoff so it runs in ~1s, and was negative-control verified: with the guard neutralised it fails with "opened 1 connection(s) but closed only 0". An unexported `driverName` package var (defaulting to `Name`) is the test seam used to point `New` at the counting driver; production never changes it.
…cy-safe
storage.New (called by every SQL storage's New) registers 14 per-storage
*expvar.Int counters via metrics.NewInt. Go's expvar.NewInt -> expvar.Publish
log.Panics on a duplicate name ("Reuse of exported var name: ..."), so a second
storage.New for the same storage type in one process crashed. This broke the SQL
integration CI runs: the new offline leak test calls New (registering the
counters), then the existing E2E TestNew calls New again -> panic.
Make metrics.NewInt get-or-reuse under a package mutex:
- Drop the previous `name + "--" + RFC3339` timestamp suffix so the published
name is the deterministic caller-supplied name (optionally DAL_METRICS_PREFIX
prefixed). The timestamp was itself a crude, timing-dependent workaround that
still collided within a single second and polluted the metric namespace with a
fresh name every process/second.
- Before publishing, look the name up with expvar.Get; if it is already an
*expvar.Int, reuse that exact instance (no re-registration, no double-count,
value preserved); if it exists but is not an *expvar.Int, return a fresh
unpublished counter rather than panicking; only when absent do we
expvar.NewInt + Set(0).
- Hold a package-level sync.Mutex across the whole Get + NewInt so the
check-and-publish is atomic: concurrent first registrations of the same name
resolve to one published counter instead of the loser panicking in
expvar.Publish.
First-registration behaviour is otherwise unchanged (Set(0), same
storage.<type>.<metric>.counter shape, same signature). storage.New is now safe
to call >= 2x per process and from concurrent goroutines.
metrics.NewInt is imported only by storage/storage.go, and internal/metrics is an
internal package, so this is invisible to external consumers' own metric names
(e.g. proj-ringboost-vendor has its own internal/metrics); UVS creates one
storage per type and thus only hits the unchanged first-registration path.
Tests (offline, negative-control verified):
- internal/metrics: TestNewInt_Idempotent (2nd NewInt reuses, value preserved),
TestNewInt_Prefix (DAL_METRICS_PREFIX honoured + reused), TestNewInt_Concurrent
(64 goroutines register the same name at once -> one shared counter, -race).
Negative controls: dropping the reuse branch panics "Reuse of exported var
name: metrics_test.idempotent.counter"; removing the mutex panics
"...metrics_test.concurrent.counter" under -race.
- postgres/mysql: TestNew_CalledTwice_NoExpvarPanic drives the package New twice
in-process via the injected counting driver (ping always fails) and asserts no
panic. Negative control: without the guard the 2nd New panics "Reuse of
exported var name: storage.<type>.counted.counter" at storage.go:150.
thalesfsp
force-pushed
the
fix/mysql-new-leak
branch
from
July 2, 2026 05:54
7b2b76b to
0c4b977
Compare
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Class: correctness / resource-hygiene (not SecOps).
The
mysql,postgres, andsqliteNewconstructors open a*sqlx.DBwithsqlx.Open(which does not dial), then verify liveness with a retriedPingand runvalidation.Validate. On the ping-failure and validation-failure returns, the opened handle was neither returned to the caller nor closed — leaking the underlyingdatabase/sqlconnection pool every time construction failed afterOpen(e.g. an unreachable DB, a flapping endpoint, or startup before the DB is ready).*sqlx.DB(and its pool) on every failed-after-openNew.Identical bug in all three storages:
mysql/mysql.goNew— ping-failure return + validation-failure returnpostgres/postgres.goNew— samesqlite/sqlite.goNew— sameFix
Guard the opened client with a single deferred close controlled by a
successflag that only flips true immediately before the successful return. Every post-open error path (ping, validation, and any future one) now closes the client exactly once; the success path does not. Signatures, the success path, and error values are unchanged.Test (TDD + negative control, offline)
Each package gets
*_leak_test.gothat drives the ping-failure path against an injected, in-process countingdatabase/sqldriver (Ping always fails;Open/Closeincrement atomic counters,Closeguarded bysync.Once), then asserts the opened handle was closed (closes >= opens). It also assertsopens > 0so it can't pass vacuously.shared.TimeoutPingto 1ms (restored viat.Cleanup) so the retrier exhausts in ~1s instead of ~70s.opened 1 connection(s) but closed only 0; restored, it passes.An unexported
driverNamepackage var (defaulting toName) is the test seam used to pointNewat the counting driver; production never changes it.make testis green;gofmtandgo vetare clean.Validation
Independently reviewed with codex (REFUTE gate): confirmed all post-open error returns close the client, no double-close, no success-path regression, signatures unchanged, and the test is offline and negative-controlled.