Skip to content

fix(sql): close the opened client on New's post-open error paths (connection-handle leak) - #10

Merged
thalesfsp merged 3 commits into
mainfrom
fix/mysql-new-leak
Jul 2, 2026
Merged

fix(sql): close the opened client on New's post-open error paths (connection-handle leak)#10
thalesfsp merged 3 commits into
mainfrom
fix/mysql-new-leak

Conversation

@thalesfsp

Copy link
Copy Markdown
Owner

Problem

Class: correctness / resource-hygiene (not SecOps).

The mysql, postgres, and sqlite New constructors open a *sqlx.DB with sqlx.Open (which does not dial), then verify liveness with a retried Ping and run validation.Validate. On the ping-failure and validation-failure returns, the opened handle was neither returned to the caller nor closed — leaking the underlying database/sql connection pool every time construction failed after Open (e.g. an unreachable DB, a flapping endpoint, or startup before the DB is ready).

  • What this fixes: the leaked *sqlx.DB (and its pool) on every failed-after-open New.
  • Why it matters: repeated failed constructions (retry loops, health-check-then-reconnect, transient DB outages) accumulate orphaned pools/goroutines/FDs that are never reclaimed.
  • Blocking: nothing — standalone hygiene fix.

Identical bug in all three storages:

  • mysql/mysql.go New — ping-failure return + validation-failure return
  • postgres/postgres.go New — same
  • sqlite/sqlite.go New — same

Fix

Guard the opened client with a single deferred close controlled by a success flag that only flips true immediately before the successful return. Every post-open error path (ping, validation, and any future one) now closes the client exactly once; the success path does not. Signatures, the success path, and error values are unchanged.

client, err := sqlx.Open(cfg.DriverName, cfg.DataSourceName)
if err != nil { return nil, ... } // no client on this path

success := false
defer func() { if !success { _ = client.Close() } }()

// ... retried Ping (on failure: return nil, err) ...
// ... build storage; validation.Validate (on failure: return nil, err) ...
singleton = storage
success = true
return storage, nil

Test (TDD + negative control, offline)

Each package gets *_leak_test.go that drives the ping-failure path against an injected, in-process counting database/sql driver (Ping always fails; Open/Close increment atomic counters, Close guarded by sync.Once), then asserts the opened handle was closed (closes >= opens). It also asserts opens > 0 so it can't pass vacuously.

  • Fully offline: in-process driver, ignored fake DSN — no real DB, no network.
  • Fast: shrinks shared.TimeoutPing to 1ms (restored via t.Cleanup) so the retrier exhausts in ~1s instead of ~70s.
  • Negative-control verified: with the guard neutralised the test fails with opened 1 connection(s) but closed only 0; restored, it passes.

An unexported driverName package var (defaulting to Name) is the test seam used to point New at the counting driver; production never changes it.

make test is green; gofmt and go vet are clean.

Validation

Independently reviewed with codex (REFUTE gate): confirmed all post-open error returns close the client, no double-close, no success-path regression, signatures unchanged, and the test is offline and negative-controlled.

thalesfsp added 3 commits July 1, 2026 15:19
The mysql, postgres and sqlite `New` constructors open a `*sqlx.DB` with
`sqlx.Open` (which does not dial), then verify liveness with a retried
`Ping` and run `validation.Validate`. On the ping-failure and
validation-failure returns the opened handle was neither returned to the
caller nor closed, leaking the underlying `database/sql` connection pool
every time construction failed after Open (e.g. an unreachable DB).

Guard the opened client with a single deferred close controlled by a
`success` flag that only flips true immediately before the successful
return. Every post-open error path (ping, validation, and any future one)
now closes the client exactly once; the success path does not. Behavior is
otherwise unchanged: signatures, the success path, and error values are
identical.

Add a per-package constructor test that drives the ping-failure path
against an injected, in-process counting `database/sql` driver (Ping always
fails; Open/Close increment atomic counters) and asserts the opened handle
was closed (closes >= opens), proving the leak is gone. The test is fully
offline (no real DB, no network), shrinks the ping backoff so it runs in
~1s, and was negative-control verified: with the guard neutralised it fails
with "opened 1 connection(s) but closed only 0".

An unexported `driverName` package var (defaulting to `Name`) is the test
seam used to point `New` at the counting driver; production never changes
it.
…cy-safe

storage.New (called by every SQL storage's New) registers 14 per-storage
*expvar.Int counters via metrics.NewInt. Go's expvar.NewInt -> expvar.Publish
log.Panics on a duplicate name ("Reuse of exported var name: ..."), so a second
storage.New for the same storage type in one process crashed. This broke the SQL
integration CI runs: the new offline leak test calls New (registering the
counters), then the existing E2E TestNew calls New again -> panic.

Make metrics.NewInt get-or-reuse under a package mutex:
- Drop the previous `name + "--" + RFC3339` timestamp suffix so the published
  name is the deterministic caller-supplied name (optionally DAL_METRICS_PREFIX
  prefixed). The timestamp was itself a crude, timing-dependent workaround that
  still collided within a single second and polluted the metric namespace with a
  fresh name every process/second.
- Before publishing, look the name up with expvar.Get; if it is already an
  *expvar.Int, reuse that exact instance (no re-registration, no double-count,
  value preserved); if it exists but is not an *expvar.Int, return a fresh
  unpublished counter rather than panicking; only when absent do we
  expvar.NewInt + Set(0).
- Hold a package-level sync.Mutex across the whole Get + NewInt so the
  check-and-publish is atomic: concurrent first registrations of the same name
  resolve to one published counter instead of the loser panicking in
  expvar.Publish.

First-registration behaviour is otherwise unchanged (Set(0), same
storage.<type>.<metric>.counter shape, same signature). storage.New is now safe
to call >= 2x per process and from concurrent goroutines.

metrics.NewInt is imported only by storage/storage.go, and internal/metrics is an
internal package, so this is invisible to external consumers' own metric names
(e.g. proj-ringboost-vendor has its own internal/metrics); UVS creates one
storage per type and thus only hits the unchanged first-registration path.

Tests (offline, negative-control verified):
- internal/metrics: TestNewInt_Idempotent (2nd NewInt reuses, value preserved),
  TestNewInt_Prefix (DAL_METRICS_PREFIX honoured + reused), TestNewInt_Concurrent
  (64 goroutines register the same name at once -> one shared counter, -race).
  Negative controls: dropping the reuse branch panics "Reuse of exported var
  name: metrics_test.idempotent.counter"; removing the mutex panics
  "...metrics_test.concurrent.counter" under -race.
- postgres/mysql: TestNew_CalledTwice_NoExpvarPanic drives the package New twice
  in-process via the injected counting driver (ping always fails) and asserts no
  panic. Negative control: without the guard the 2nd New panics "Reuse of
  exported var name: storage.<type>.counted.counter" at storage.go:150.
@thalesfsp
thalesfsp force-pushed the fix/mysql-new-leak branch from 7b2b76b to 0c4b977 Compare July 2, 2026 05:54
@thalesfsp
thalesfsp merged commit 53c32ee into main Jul 2, 2026
1 check passed
@thalesfsp
thalesfsp deleted the fix/mysql-new-leak branch July 2, 2026 06:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant