Skip to content

fix(database): slow migrations block startup with no feedback #1372

Description

@wizzomafizzo

Summary

A slow database migration blocks service startup and reports nothing while it runs. On a MiSTer with a 229k-item library the service was unavailable for 2m14s with no output between "opening databases" and goose's completion line.

What the log showed

20:04:18  opening databases
20:04:18  effective database pragmas   db=user
20:04:18  goose: no migrations to run. current version: 20260818120000
20:04:19  created scheduled user database backup
20:04:19  effective database pragmas   db=media
          ... 2m13s of nothing ...
20:06:32  goose: successfully migrated database to version: 20260830140000

Why

Everything in pkg/database/migrations.go is logged at Debug: acquiring the mutex, setting the dialect, the filesystem, the logger. Debug logging is off by default, so a user sees none of it. The only Info line is goose's own, and it is emitted after the migration finishes — precisely when the feedback is no longer needed.

There is also no non-log channel at that point in startup: the API server is not listening yet, so no notification can be sent and nothing on a display can be updated.

Why it matters

Migrations are unbounded in cost and scale with library size, and MiSTer SD storage is roughly 680x slower than a desktop for index work — the same statement measured 198ms locally and 2m14s on the device. A migration that looks free in development can freeze startup for minutes in the field, and from the user's side it is indistinguishable from a hang.

The specific migration that caused this was fixed in #1371, but the gap is general: any future schema change on a large library hits it.

Worth considering

  • An Info line naming each migration before it is applied, so the log shows what is being waited on.
  • Some signal outside the log, since the API is not up yet — the platform display or the launcher are the only channels available that early.
  • Whether unbounded table work in a migration should be allowed at all, or pushed to the indexing path where the cost is expected and already reported.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions