Skip to content

Split pool-indexer bootstrap from serve via --bootstrap-only flag#4559

Open
AryanGodara wants to merge 1 commit into
aryan/pool-indexer-migration-isolationfrom
aryan/pool-indexer-split-bootstrap
Open

Split pool-indexer bootstrap from serve via --bootstrap-only flag#4559
AryanGodara wants to merge 1 commit into
aryan/pool-indexer-migration-isolationfrom
aryan/pool-indexer-split-bootstrap

Conversation

@AryanGodara

Copy link
Copy Markdown
Member

Description

The pool-indexer does bootstrap-then-serve in one process: pool-indexer --config <toml> runs the initial subgraph seed plus catch-up to the finalized head, then serves the HTTP API. On a large chain the bootstrap phase can take minutes, which forces the K8s pod's startupProbe to wait that long before it can declare failure, hiding genuine startup bugs in the serve path.

This adds a --bootstrap-only mode so the two phases can run in separate containers: an idempotent initContainer does the bootstrap, and the serve container then comes up with a tight startupProbe because the DB is already seeded.

Changes

  • Added a --bootstrap-only flag. With it, start runs the new bootstrap entrypoint, which connects the DB, validates the RPC chain_id, runs the existing per-factory bootstrap (seed + catch-up to the finalized head), and exits 0. It binds no HTTP ports. It is idempotent: a factory with an existing checkpoint is skipped, so re-running on an already-seeded DB is a fast no-op that never contacts the subgraph.
  • Without the flag, pool-indexer --config <toml> is unchanged: it bootstraps when the DB has no checkpoint, then serves. Existing single-container deploys keep working as-is.
  • Extracted a build_provider_checked helper (provider build plus chain_id assertion) shared by the serve and bootstrap paths, so the misconfigured-RPC guard runs in both.

How to test

Unit and e2e tests:

  • cargo nextest run -p pool-indexer — includes bootstrap_only_flag_parses.
  • cargo nextest run -p e2e local_node_pool_indexer --run-ignored ignored-only --test-threads 1 — includes local_node_pool_indexer_bootstrap_idempotent, which pre-seeds a checkpoint and asserts bootstrap returns immediately (no ports bound, subgraph untouched, checkpoint unchanged), alongside the existing serve-path tests.

Locally tested on ink subgraph:

Simulated the init-container flow (fresh ink_pool_indexer DB

  1. pool-indexer --bootstrap-only --config <ink.toml> seeded 1718 pools / 8266 ticks, caught up to the finalized head, and exited 0 in ~21s without binding any ports.
  2. Re-running --bootstrap-only exited 0 immediately, logging "existing checkpoint found, skipping bootstrap" and never contacting the subgraph.
  3. pool-indexer --config <ink.toml> then flipped /startup to 200 within about a second of binding (bootstrap already done), /health returned 200, and /pools served real Ink pool data.
  4. Restart recovery is RPC-only: with the subgraph URL deliberately unreachable, restarting the serve process against the seeded DB backfilled a 6217-block downtime gap purely from RPC (checkpoint 48790485 to 48796702) and kept serving, never contacting the subgraph. The subgraph is used only for the first cold seed (empty DB / no checkpoint); restarts, OOM kills, and rollout gaps all recover from RPC eth_getLogs.

@AryanGodara AryanGodara marked this pull request as ready for review June 24, 2026 11:05
@AryanGodara AryanGodara requested a review from a team as a code owner June 24, 2026 11:05
@claude

claude Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Claude finished @AryanGodara's task in 2m 24s —— View job


PR Review: Split pool-indexer bootstrap from serve

  • Read the PR diff and changed files
  • Analyze run.rs, arguments.rs, lib.rs changes
  • Review e2e test changes
  • Post inline findings
  • Final summary

This is a clean, well-scoped change. The two-phase split is sound and the idempotency claim holds up under inspection: catch_up writes the checkpoint transactionally right after seeding (uniswap_v3.rs:159-161), so even a crash mid-catch-up leaves a checkpoint — a re-run skips the subgraph, and the serve container's indexer.run fills the remaining gap from RPC. Good design. Extracting build_provider_checked so the chain_id guard runs on both paths is a nice touch, and the test coverage (flag-parse unit test + idempotent e2e) matches the new behavior.

One non-blocking finding (posted inline on run.rs):

  • Bootstrap is sequential, serve is concurrent. bootstrap loops over factories one at a time, while the serve path fans them into a JoinSet. On a multi-factory network the init-container takes ~N× longer than the old in-process bootstrap, which partly works against the goal of shrinking the slow phase. Suggest mirroring serve's concurrency (or a comment if sequential is intentional to cap RPC load). The per-factory indexer construction is also duplicated between the two paths — a small shared helper would prevent drift.

Nothing here blocks merge.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a "--bootstrap-only" CLI flag to the pool-indexer, splitting its startup into a separate bootstrap phase (initial seed and catch-up) and a serve phase. This allows running the bootstrap phase as an idempotent Kubernetes initContainer. The changes include adding the new CLI argument, implementing the "bootstrap" execution path, updating documentation, and adding integration and unit tests to verify the idempotent behavior. No critical issues were found, and there is no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread crates/pool-indexer/src/run.rs
@AryanGodara AryanGodara force-pushed the aryan/pool-indexer-split-bootstrap branch from c628569 to a1aa54b Compare June 24, 2026 22:49
@AryanGodara AryanGodara self-assigned this Jun 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant