feat(0374): soroban AMM write path (Refs #405) - #441
Merged
Merged
Conversation
…DL docs Write half of the Aquarius work, extracted from the PR #438 branch so the schema can be consolidated while no reader exists. The read half (crates/api, web, libs/api-types) deliberately stays behind on that branch until the wire shape settles (total_shares source, participant_count, activity-ledger semantics, share_percentage denominator — ranked in task 0374). Byte-identical to the #438 states on every path taken: xdr-parser (pool_router, pool_state, scval readers, real-ledger corpus tests), db-clickhouse (init.sql, persist/stage, e2e), indexer soroban arm, backfill-runner sink, the audit-harness crate deletion (sole sqlx consumer — the workspace dep goes with it), deploy/backfill runbooks and architecture docs, ADR 0058. Verified on this branch alone: cargo check --workspace (api at develop state compiles against the new lock), all four crates' test suites green including the 90-ledger real-corpus e2e, extract_openapi output JSON-identical to the committed spec so the api-types freshness gate passes without a regen. Refs #405
… fold home Findings 20-30 of the devils-advocate + ponytail + simplify pass, plus the owner's follow-ups, in one train: - pool_state_changes folds and sorts on (pool_id, plane_id, ledger_sequence): a forged plane entry naming a real pool could previously EVICT the genuine row at the parse-time fold and at the RMT key — the read-side declared-plane filter would then hide the forgery while serving a stale ledger's reserves as current. Forged rows now keep their own key space, die at the read filter, and stay visible to the standing monitor documented in the DDL (uniqExact(plane_id) > 1 per pool — also the alarm for a plane migration, which would otherwise orphan pre-migration history silently). Measured on the 90-ledger raw corpus: zero forged writes exist in the wild; the change is preventive and free while no production DDL exists. - One fold home: the hand-copied last-wins collapse (4 copies) becomes xdr_parser::fold::keep_last_by_key; both soroban state tables now fold in STAGING (fold_pool_state_changes / fold_pool_instance_state), symmetric, and the parser-side pre-folds are gone. The instance fold is load-bearing, not theoretical: 29 of 259 real (pool, ledger) keys in the corpus carry more than one instance image (up to 5 in one ledger) and the table's RMT version ties within a ledger. - write_ledger exhaustively destructures StagedLedger — the other half of commit()'s guard: a future staged row-vec that never reaches a write_rows call refuses to compile instead of dropping rows silently. - pool_state.rs reads through the house scval readers (map_get, address, symbol, typed_str) instead of private copies — the exact 0393 recurrence. - subpool_salt decode deleted: zero consumers, and the schema's own rule for registration provenance is extract-on-demand-never-copy. The corpus knowledge stays in the AddPoolEvent docs. - Smaller: per-event String clone dropped from the hot detect loop (as_deref), one shared parse_reserves for both state arms, AddPoolReject collapsed to NotAddPool | Malformed(&str), contract_payload moved to ids.rs beside the other identity mappings, init.sql overclaim corrected (the router-less UNVERIFIED arm is the documented exception), triplicated fold prose reduced to one home. - pool_state_real_corpus asserted router.is_some(), contradicting the in-tree Router-optional relaxation — env-gated, so CI never saw it red; on the dead-deployment ledger 50,638,875 it failed against real history. Now asserts the plane only, matching the acceptance arm. Verified: 30/30 suites green, cargo check --workspace clean, and the real corpus (90 raw mainnet ledgers, all eras and pool types) passes end to end with the new keys. Refs #405
…onest docs Eight of nine findings valid against current code (the ninth — the corpus router assertion — was already fixed; its module doc line rode along): - The concentrated-instance reserve arm defaulted a missing declared plane to plane_id 0 — a placeholder that would neither match the read filter nor fold with genuine rows. Now refused loudly, same as the sibling instance-state arm. - pool_instance_state DDL comment claimed Router and Plane are both required; Plane is the shape key, Router is optional (five older deployments, 23 pools, accepted UNVERIFIED) — comment, ADR 0058 and indexing-pipeline-overview now all carry the same rule. - database-schema-overview stopped listing subpool_salt/init_args as persisted (extract-on-demand rule), its snapshots section swapped the retired Postgres DDL for the ClickHouse shape, and the tvl/volume/fee_revenue note now says removed-by-0374-DDL instead of retained-for-future-materialization. - The registry-backfill runbook grew the full restore→harvest→generate workflow, including two adjustments the resurrected generator needs (drop share_token_id from its INSERT — the column no longer exists — and refuse duplicate registrations, which the closure check alone cannot see). - The window-closure check reconciles pool-id SETS in both directions via base32Decode (strkey → 32-byte payload; verified against ClickHouse 26.3) instead of comparing cardinalities, which are blind to substitution. - deployment.md's placeholder CREATE TABLE lines now point at init.sql as the verbatim source, with the awk extraction command tested. Refs #405
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
crates/api,web,libs/api-types) stays on the feat(0374): soroban AMM pools — aquarius end to end (Refs #405) #438 branch until the wire shape settles.pool_router(shape-drivenadd_poolregistry),pool_state(plane + instance reserve extraction), typed-JSON readers consolidated inscval.rs; real-mainnet-ledger corpus tests ride along.pool_state_changes+pool_instance_statestaging with per-ledger fold, exhaustiveTableInsertsdestructure in commit, soroban arm wired into the live handler.init.sql+ runbooks: DDL-before-indexer order, three catch-up backfills and the window-closure check documented indocs/deployment.md/docs/backfills.md; ADR 0058 and architecture overviews updated per ADR 0032.audit-harnesscrate (solesqlxconsumer — the workspace dep goes with it). API spec untouched:extract_openapioutput is JSON-identical to the committedopenapi.json.