Skip to content

evmigration(batch): classify vesting-locked targets as needs-funding#177

Open
mateeullahmalik wants to merge 3 commits into
masterfrom
matee/migrate-batch-vesting-spendable-classify
Open

evmigration(batch): classify vesting-locked targets as needs-funding#177
mateeullahmalik wants to merge 3 commits into
masterfrom
matee/migrate-batch-vesting-spendable-classify

Conversation

@mateeullahmalik

Copy link
Copy Markdown
Contributor

Summary

migrate-batch.sh classifier silently routed vesting-locked multisigs onto the no-funder code path, where they failed at broadcast.

On a fresh v1.20.0-rc5 local devnet booted with mainnet-shape genesis (testnet-2 bundle, voting=15m, expedited=5m, max-txs=10000), running migrate-batch.sh execute over the 31 foundation accounts produced:

Before After
✅ Migrated 12/31 31/31
❌ Failed 19/31 0/31

Every one of the 19 failures was a continuous-vesting multisig with balance > 0 but spendable = 0. The chain-side error was always:

spendable balance 0ulume is smaller than 5000ulume: insufficient funds

Root cause

_mb_classify_target only queried bank balances (TOTAL). For vesting accounts that's misleading — the ante handler charges fees against spendable, not total. The decision was:

if (( acct_known == 1 )) || [[ "$balance" != "0" ]]; then
  printf 'needs-pubkey\n'    # ← funder NEVER fires for vesting-locked
else
  printf 'needs-funding\n'
fi

So a vesting multisig with balance=5T, spendable=0 got needs-pubkey, the script tried _mb_multisig_self_send with no funding, ante rejected, target failed.

This is the dominant shape on mainnet — foundation multisigs are continuous-vesting and remain locked for years (advisors_*, seed_sale_2-5, private_sale_3-6, team_2-6, eco_dev_7, …). An operator running the batch driver on launch day would have hit silent failures on all of them.

Fix

  1. Also query bank spendable-balances in _mb_classify_target.
  2. New constant _MB_MIN_SELF_SEND_SPENDABLE_ULUME=10000 (self-send fee 5000 + headroom).
  3. Classifier emits needs-funding whenever spendable < threshold, even if total balance is large → funder path tops the target up before the self-send.
  4. Surface spendable=<ulume> in:
    • operator log line
    • JSONL audit classify event
    • docstring + report subcommand --help

If the spendable-balances query fails (older binary, transient RPC), treat as spendable=0 → safer to over-fund than to silently broadcast a doomed tx.

Verification

Two end-to-end runs on the same rc5 devnet, same mnemonics, same --funder:

Before (PR #163 head)

INFO  === Batch summary ===
INFO    succeeded : 12
INFO    failed    : 19

Failure signature on every locked target: multisig_self_send_failedinsufficient funds.

After (this PR)

INFO  === Batch summary ===
INFO    succeeded : 31
INFO    failed    : 0
INFO    remaining : 0

Chain confirms:

{ "total_migrated": "31", "total_legacy": "21" }

Audit-log evidence for a vesting-locked target:

{"event":"classify","target":"ecosystem_development_7","status":"needs-funding","balance":"11250000000000","spendable":"0"}

Includes the non-sequential signer-order multisigs that prompted PR #176 (team_3 [s3,s2,s1], seed_sale_2 [s1,s3,s2], team_6 [vesting til 2028], etc.) — all addresses reconstructed byte-for-byte and migrated.

Static checks

  • shellcheck -x -e SC1091,SC2034 scripts/migrate-batch.sh → clean
  • bats tests/scripts/migrate-batch.bats → 11/11 pass (report-mode tests unaffected)

Operator impact

  • Behavior strictly safer: a class of targets that previously silently broke now go through the funder path that already exists. No new flags. No flag-default changes.
  • Operators must still pass --funder (with enough balance) when any target needs funding. This was already documented and required for the zero-balance case.
  • --top-up-amount default (200000ulume) is already sized for both cases.

Risk

  • Determinism: pure classifier change; no state mutation.
  • Migration semantics: unchanged — the actual claim-multisig ceremony is identical, only the pre-step (self-send vs. funder-then-self-send) differs.
  • Failure mode if bank spendable-balances is unavailable: we treat as spendable=0 → routes to funder. Funder path requires --funder; absent funder, the target errors out cleanly with the existing needs_funder_not_provided reason (same as the zero-balance path).
  • Backwards compat: report-subcommand text in report/--help updated; JSONL audit adds a new spendable field — additive only.

Rollback

Revert the single commit. No state migration. No state on chain depends on this script.

Related

The classifier in migrate-batch.sh routed any target with TOTAL balance>0 onto
the no-funder code path, even when the balance was vesting-locked and the
spendable portion was zero. Such accounts then failed at broadcast with
"spendable balance 0ulume is smaller than 5000ulume: insufficient funds"
because the ante handler charges fees against spendable, not total.

This is the dominant shape on mainnet: foundation multisigs are continuous-
vesting and remain locked for years (advisors_*, seed_sale_2-5, private_sale_3-6,
team_2-6, eco_dev_7, etc — ~24 of 28 multisigs). An operator running the batch
driver on launch day would have hit silent failures on all of them.

Fix:
* Query `bank spendable-balances` alongside `bank balances` in classify.
* Re-route to `needs-funding` (funder path) when spendable < self-send fee
  threshold (10000ulume), even if total balance is large.
* Surface spendable in the operator log line, the JSONL audit "classify"
  event, and the docstring + report-subcommand --help.

Verified end-to-end on local rc5 devnet against mainnet-shape genesis: prior
run 12/31 ✓, 19/31 ✗ (all vesting-locked). With fix: 31/31 ✓, 0 ✗. Chain
total_migrated=31. Audit log shows spendable=0 + needs-funding for every
vesting-locked target.
@mateeullahmalik mateeullahmalik self-assigned this Jun 23, 2026
Drove the full PR #177 batch end-to-end on a fresh rc5 local devnet with
mainnet-shape genesis, following ONLY the operator-facing docs. The actual
PR-#177 broadcast flow worked perfectly (31/31 ✓, 0 ✗, chain
total_migrated=31), but the docs left several Day-1 questions unanswered.
This commit closes the gaps without changing the migration logic.

scripts/migrate-batch.sh — status subcommand:
* Surface 'spendable=' next to 'balance=' on every status row so an operator
  immediately sees WHY a vesting-locked target (balance=5T, spendable=0)
  is classified needs-funding. The JSONL audit already had this field; the
  human status table did not.
* Add a one-line footer to the status summary explaining that needs-funding
  covers vesting-locked targets too.

scripts/migrate-batch.md:
* New 'Prerequisites' section (lumerad version, bash/jq, RPC endpoint,
  operator keyring location, where to run from) — modeled on
  migration-scripts.md.
* New 'Picking and preparing the funder' section: 3 concrete checks the
  operator must run before kicking off execute (key in operator keyring,
  spendable balance budget, key type supported).
* Rewrite --funder flag row: now correctly says 'pays fees for any target
  classified needs-funding, including vesting-locked accounts whose total
  balance is large but spendable is zero'. The old 'zero-balance targets'
  framing was wrong post-PR #177.
* Rewrite --top-up-amount flag row with the concrete sizing formula
  operators can apply for non-default fees.
* Update status-subcommand descriptions to reflect the spendable-aware
  classifier and the two real shapes of needs-funding.
* Suggested-workflow now dry-runs TWO targets in step 3 — one vested
  (exercises the no-funder self-send path) and one vesting-locked
  (exercises the funder path) — so operators verify both code paths
  before the full batch.
* Suggested-workflow step 5 now uses --log-file and --continue-on-error,
  matching the recommended production shape.

docs/evm-integration/user-guides/migration-scripts.md:
* Add migrate-batch.sh to the 'Start Here' decision table with a link to
  scripts/migrate-batch.md. Previously, operators following the user
  guides were told to 'drop into a loop' over single-account scripts
  and would never discover the actual batch driver.

All static checks green:
  shellcheck -x -e SC1091,SC2034 scripts/migrate-batch.sh — clean
  bats tests/scripts/migrate-batch.bats — 11/11 pass

Live verification on the rebuilt rc5 devnet:
  Status output: shows balance + spendable per row, footer note present.
  Full batch (--log-file --continue-on-error): 31/31 ✓, 0 ✗.
  Chain total_migrated=31. JSONL audit shows clean spendable= for every
  vesting-locked target.
@mateeullahmalik

Copy link
Copy Markdown
Contributor Author

Day-1 operator rehearsal on this branch — full evidence

I rebuilt a fresh lumera-devnet-1 on this branch (rc5 binary, mainnet-shape genesis: testnet-2 bundle + voting=15m, expedited=5m, max-txs=10000, vesting-locked foundation multisigs) and ran the playbook from scripts/migrate-batch.md as if I only had the docs — no script source, no Slack threads, no internal context.

The actual migration flow worked end-to-end: 31/31 succeeded, 0 failed, chain total_migrated=31. Including team_6 (vesting until 2028, non-sequential signer order [s3,s2,s1]) — the exact account class that originally surfaced this bug.

Following the doc strictly surfaced 7 places where the operator had to guess or look at source. The new commit 6faa96aa closes all of them without touching the migration logic.

What the new commit changes

Script (scripts/migrate-batch.sh):

  • status now prints spendable= next to balance= on every row, plus a footer note explaining that needs-funding covers vesting-locked targets. The JSONL audit already had this field; the human-facing status table did not.

Doc (scripts/migrate-batch.md):

  • New Prerequisites section (lumerad version, jq/bash, RPC, keyring, host).
  • New Picking and preparing the funder section — 3 concrete checks operators must do before kicking off execute.
  • Rewrite of --funder row: now says "pays fees for any target classified needs-funding, including vesting-locked accounts whose total balance is large but spendable is zero". The old "zero-balance targets" framing is wrong post-PR evmigration(batch): classify vesting-locked targets as needs-funding #177.
  • Rewrite of --top-up-amount row with a sizing formula operators can apply for non-default fees.
  • status subcommand reference now describes the two real shapes of needs-funding (zero-balance OR vesting-locked).
  • Suggested workflow step 3 now dry-runs two targets — one vested (no-funder path) and one vesting-locked (funder path) — so operators verify both code paths before broadcast.
  • Step 5 example now uses --log-file + --continue-on-error.

Doc (docs/evm-integration/user-guides/migration-scripts.md):

  • migrate-batch.sh now appears in the "Start Here" decision table with a link to scripts/migrate-batch.md. Previously the user-guides told operators to "drop into a loop over single-account scripts" and never mentioned the batch driver.

Evidence from the rehearsal run

Step 2 (status) before any execution — every account class correctly classified, new spendable= field visible:

needs-pubkey  [multisig  ] seed_sale_1                    lumera1...uxq6f8cpe  balance=5000000000000ulume  spendable=5000000000000ulume
needs-funding [multisig  ] seed_sale_2                    lumera1...maq9s      balance=5000000000000ulume  spendable=0ulume
needs-funding [multisig  ] team_3                         lumera1...vzuhxm     balance=8333333330000ulume  spendable=0ulume
needs-funding [multisig  ] team_6                         lumera1...vfna9      balance=8333333330000ulume  spendable=0ulume
needs-pubkey  [multisig  ] ecosystem_development_3        lumera1...uafn0      balance=11250000000000ulume spendable=11250000000000ulume
…
needs-pubkey  [single-sig] community_growth_1             lumera1...m7s7f      balance=12500000000000ulume spendable=12500000000000ulume

Summary: migrated=0 ready=0 needs-pubkey=12 needs-funding=19 unknown=0
Note: needs-funding includes targets where spendable=0 even when balance>0 (vesting-locked).

Step 4 (single-target real broadcast on team_6) — the canonical regression case:

  • Funder paid 200000ulume to team_6 (h=42).
  • 2-of-3 multisig self-send published the legacy pubkey (h=43).
  • claim-multisig ceremony broadcast and included at h=44, new address lumera1dupgu5hv… holds the 8.3T migrated balance.

Step 5 (full batch) — all 31 accounts:

INFO  === Batch summary ===
INFO    succeeded : 31
INFO    failed    : 0
INFO    remaining : 0

Chain confirms:

{ "total_migrated": "31", "total_legacy": "21" }

JSONL auditclassify event for every target carries balance + spendable, e.g.

{"event":"classify","target":"team_3","status":"needs-funding","balance":"8333333330000","spendable":"0"}
{"event":"classify","target":"team_1","status":"needs-pubkey","balance":"8333333340000","spendable":"8333333340000"}

Static checks (re-run after the new commit):

  • shellcheck -x -e SC1091,SC2034 scripts/migrate-batch.sh → clean
  • bats tests/scripts/migrate-batch.bats → 11/11 pass

What this PR is now

A two-commit PR:

  1. ab4cf94e — the original spendable-aware classifier (the code bug fix).
  2. 6faa96aa — the operator-doc + status-output fixes surfaced by actually following the docs as an operator.

Reviewers: the second commit doesn't change migration logic. It's status-output + docs.

migrate-batch.sh is a foundation/operator launch tool — it assumes one
operator holds every signer mnemonic in a single JSON file on one host.
That threat model is fine for the team's coordinated launch ceremony but
is the opposite of what the public CLI user-guide tells normal users
(single key on their box, never share). Putting the script in that
guide's Start Here table is a category error and could be read as
encouraging end users to centralize keys.

Keep the operator-facing playbook (scripts/migrate-batch.md) and the
script-level changes (spendable-aware classifier, status output, JSONL
audit) which are what mainnet operators actually need.
@a-ok123 a-ok123 requested a review from Copilot June 24, 2026 00:03
@mateeullahmalik

Copy link
Copy Markdown
Contributor Author

Fair call — migrate-batch.sh doesn't belong in the public CLI user-guide. It's a foundation-launch tool whose threat model assumes one operator holds every signer mnemonic in one JSON file on one host, which is the opposite of what we tell normal users. Reverted that row in d762cdfc — user-guide is back to master.

Keeping:

  • the code fix (ab4cf94e): vesting-locked multisigs were silently failing in the batch driver because the classifier only looked at TOTAL balance, not spendable. On rc5 mainnet-shape devnet, prior to this it was 12/31 ✓ / 19/31 ✗ (every locked foundation multisig). With the fix: 31/31 ✓. This is the bug from the rebuild rehearsal you asked for.
  • the operator-doc updates in scripts/migrate-batch.md (next to the script, not in public docs): Prerequisites, funder prep, sizing formula, two-target dry-run.
  • the status-output spendable= column so operators can see why a vesting-locked multisig is needs-funding.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes migrate-batch.sh target classification so vesting-locked multisigs (total balance > 0 but spendable = 0) route through the funder path instead of attempting an unfundable self-send that fails at broadcast.

Changes:

  • Extend _mb_classify_target to query bank spendable-balances and include spendable=<ulume> in both operator output and JSONL audit events.
  • Introduce a spendable threshold constant intended to decide between needs-pubkey vs needs-funding.
  • Update operator docs (migrate-batch.md) to explain spendable vs total balance and funder selection.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
scripts/migrate-batch.sh Adds spendable-balance probing and surfaces spendable in status/audit; introduces a spendable threshold for classification.
scripts/migrate-batch.md Documents prerequisites, funder selection, and clarifies status output/meaning with spendable vs total balance.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread scripts/migrate-batch.sh
Comment on lines +203 to +207
# Minimum spendable ulume required for the target to broadcast its own
# self-send (publishes pubkey). Sized to cover the multisig self-send fee
# (5000ulume) with headroom. Targets below this threshold MUST be funded by
# --funder even if their TOTAL balance is large (the vesting-locked case).
_MB_MIN_SELF_SEND_SPENDABLE_ULUME=10000
Comment thread scripts/migrate-batch.sh
Comment on lines 213 to +217
# needs-pubkey — auth account exists (or has balance) but no pubkey
# needs-funding — account not found on auth AND balance==0
# AND has enough SPENDABLE balance to self-fund the
# self-send fee
# needs-funding — pubkey missing AND spendable balance is insufficient
# for the self-send fee. This covers two cases:
Comment thread scripts/migrate-batch.md
Comment on lines +94 to +96
- `bank balances` and `bank spendable-balances` (does the account hold ulume
it can actually spend on fees?)

Comment thread scripts/migrate-batch.md
Comment on lines +103 to +108
- `needs-funding` — pubkey missing AND spendable balance is below the
self-send fee threshold. Covers **two** real cases:
(a) zero total balance (classic fresh foundation
account), and (b) non-zero total balance but
**vesting-locked** so spendable < fee. Both require
`--funder` during `execute`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants