Repository navigation
Whole-source census: installed Company and GLEIF proof (6,414 Companies, 3,052 GLEIF bindings) - #900
Merged
Merged
Conversation
Operator, 2026-10-10: "merge and take over the goals from grok handover including the checklist". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
tests/support/pg16.py hands out an empty PostgreSQL 16: Docker by default, as CI runs it; with PG16_SERVER=pgserver, a local pgserver on its own socket, deleted afterwards. The Clean MDM and configured bookkeeping fixtures (and the provisioning test) build their addresses from it. Locally (no Docker): 46 bookkeeping and 46 Clean MDM Postgres tests pass with PG16_SERVER=pgserver. Needed for the installed Company proof (whole-source census, taken over from Grok). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
Installed workers and separate verifiers only; the harness provisions the stores, policy, contracts and the disposable Rules pipeline as the G3 acceptance test does. SEC bundles through mdm.merge with a killed-worker recovery; GLEIF Level 1 read with its approved scope, prepared and merged; both again as new runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
The merge worker writes one item in one transaction, so its rows appear only on commit. Killing it "once rows land" killed a worker that had already finished (exit 0 in the SEC-only run, 2026-10-10 08:17 ET). It is now killed while its item is `running`, and the proof fails unless it exited by SIGKILL. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
…very Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
The second GLEIF pass prepared new batches for the first pass's consumer, so its checkpoint 0->1 was refused as stale (run 2026-10-10 09:28 ET, after the first pass recorded 6,414 Companies and 3,052 bound GLEIF records). `mdm.prepare` numbers a unit's consumer from checkpoint 0, so each unit names its own: the second pass re-applies the same records. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
The rerun's first pass held (6,414 Companies; 3,052 GLEIF bound: 2,861 by name and jurisdiction, 191 by name and postcode), then the disk guard stopped it at 870 MB free while the second pass re-read the 13 GB Golden Copy. The second GLEIF pass is now a `gleif-replay` target: the first pass's verified reading, prepared and merged again under its own consumer. Re-reading would only repeat what the `source.read` verifier already recomputed; the second pass tests MDM, not the reader. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
6,414 Companies, 3,052 GLEIF bindings equal to the checkout replay (9216cc18...), an unchanged second pass and recovery, on an empty PostgreSQL 16 with installed workers and a separate verifier. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
- Spec: the installed proof asserts the bindings equal the checkout replay's; the SEC second pass gives each copied batch its own id and consumer, so it is mastered again rather than skipped as already merged; the recovery kill lands while an item is held after an earlier item committed, and the resumed item must be verified on its next attempt. - Standards and GoF: `pg16.server()` yields only once the server answers `SELECT 1` and raises if it never does (both callers' three copies of that wait removed); a container that fails to stop warns; the pgserver folder is removed even when the server fails to start. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
Waiting for an item that is both held and committed is a race: the run of 10:48 ET missed it (the worker finished first). The kill lands on the held item; the resumed item must be verified on its next attempt. The SEC run of 10:47 ET, kept as evidence, killed one after its MDM rows committed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
… contract Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk
paulananth
added a commit
that referenced
this pull request
Oct 10, 2026
…d pg16 server (#903) * Integration tests: RDM, Rules approval and silver writer on the shared pg16 server Each started its own Docker postgres:16-alpine with its own readiness loop. They now use tests/support/pg16.server(), as the clean MDM tests do (#900): Docker in CI, PG16_SERVER=pgserver locally, yielded once it answers SELECT 1, cleaned up either way. The silver writer keeps PG16_ADMIN_URL for a server kept between runs. 23 passed on local PostgreSQL 16. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk * Integration tests: drop the clean MDM docker helper, now unused Review (Standards, Spec, GoF): its last caller was the silver writer test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk --------- Co-authored-by: Aneena Ananth <aneenaananth@MacBook-Air.local> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
paulananth
added a commit
that referenced
this pull request
Oct 14, 2026
- 02: merge version approved and active locally; SEC Company and GLEIF source activation moved to ticket 71 (operator, 2026-10-14). - 03: L8 done by #933; the live SEC qualification moved to ticket 71. - 04: #799 merged; streaming and multi-input steps are tickets 16 and 17. - 05: Company, GLEIF and Person read on the engine, old readers deleted, G3 passed again (installed proof, 2026-10-14). - 06: GLEIF parents and insider links switched on (#775, #964); Person feed 1's switch-on is ticket 11. - 07: the full Person run (76,230 documents, 31,097 people, 45,133 set aside) on the record-by-record Merge Stage: batch times flat, second pass unchanged. The Person proving script adds person-cik only when the policy lacks it. - 08: Phase 2 done (#900, #904, #905, #907). - 01: the ownership switch-on ticked (#964, ade7139). Claude-Session: https://claude.ai/code/session_01AoWChUCsVyST73AjRcdH6D Co-authored-by: Aneena Ananth <aneenaananth@MacBook-Air.local> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this proves
Grok's handoff goal (
.planning/workstreams/whole-source-census/HANDOFF-20261010.md): Company and GLEIF construction on the installed bundle, into an empty PostgreSQL 16, by workers and a verifier that are separate processes and never import the checkout.Result, 2026-10-10 10:36 ET, 33 min 46 s (
evidence/installed-company-proof-20261010.json):company-cik)9216cc18…, equal to the checkout replayWhat is in the PR
.planning/workstreams/whole-source-census/installed_proof.py: the driver. Installs the bundle from this commit withuv tool install, registers, proves, approves and activates the pipeline in a disposable Rules DB, then runs every step withedgar-warehouse workers workand checks it withworkers verifyunder a separate login. SEC: the seven cm27 bundles throughmdm.merge. GLEIF: the 2026-09-11 Golden Copy throughsource.read(approved scope = the 3,755 cohort LEIs),mdm.prepare,mdm.merge. Second pass: SEC bundles with their own batch ids and consumer (mastered again, not skipped), and the verified GLEIF reading prepared and merged again under its own consumer.tests/support/pg16.py: one PostgreSQL 16 for the Postgres acceptance tests: Docker in CI as before, or a localpgserverwithPG16_SERVER=pgserver. It yields only once the server answers a query.test_clean_mdm_postgres.pyandtest_configured_bookkeeping_postgres.pyuse it (92 tests pass locally on pgserver).TICKET.md: the takeover checklist, kept current, and the evidence files.Found on the way
gleif_source.record_evidence) is reachable only through anative_sourcemanifest that nothing writes, but GLEIF relationships and reporting exceptions have no configured reading yet. Recorded on the ticket as L6's parts.Not done here
fdc020f9, 13:57 ET,evidence/installed-company-proof-final-20261010.json) passed with the stronger checks (bindings asserted equal, resumed attempt checked, both second passes mastered again).mdm name-censusand deleting the Python builder: the next PR,claude/name-frequency-assembly, stacked on this one.Review
Three axes (Standards, Spec, GoF) on this diff; every finding fixed or recorded on the ticket.
🤖 Generated with Claude Code
https://claude.ai/code/session_01FFKdv4nqKXKM9d5Cf2hEFk