Mastering writes a batch once: outbox by batch, Company fields once, no repeated write

[paulananth]

Mastering writes a batch once

Ticket: .planning/workstreams/mdm-request-dedup/TICKET.md.

Root cause

Each step that handled a batch kept its own whole copy of it, and a batch with identity work was written twice. On cm27 chunk 1 (919 Companies, a 1.4 MB records file), one batch wrote about 36 MB into the store:

Fact	Copies before	Where
Each source reading	5	source_reading, stage_record.reading, batch.effects, the journal outbox row, the match proposal
Each golden record	7	company.body, company.fields, batch.effects, the journal outbox row twice, the graph and export outbox rows
Each decision	4	decision, batch.effects, the journal outbox row, the proposal

The write itself ran twice: preview_batch (all of write_batch, rolled back), then save_batch.

What

• 014, outbox by batch. A new outbox row keeps only the hash of what it sends. What it sends is the view mdm.outbox_delivery, built from the batch row. It is a view because migrate revokes every function from PUBLIC and a publisher's verifier holds SELECT only. A row stored before keeps and sends its own payload.
• 015, Company fields once. company.fields was body->'fields' again; company_master_field reads body, and the column is dropped.
• The write runs once. MergeStage.apply with identity work assesses once (no preview write, no second computation), commits the proposal before any master write (Q13), then saves the computed batch after re-checking its snapshot and each rule proposal under the lock. assess/apply_assessment keep their preview for a proposal made on its own.

Not done, per the GoF consult (reasons on the ticket): the Stage's copy of the winning reading, per-field policy digests, the publication key per reading.

Evidence

	Before	After
Mastering chunk 1	11.3–11.4 s	6.6 s (−42%)
Store after one batch	36.2 MB	16.5 MB (−54%)
Outbox	5.4 MB	under 0.2 MB
company	5.1 MB	1.9 MB

• New tests: test_clean_outbox_names_batch.py, test_clean_company_fields_once.py, test_clean_assess_and_apply.py. Each migration is tested on rows written by the 001 functions. The built outbox payload and hash equal what the 001 write stored, for journal, export and graph.
• 239 MDM Postgres tests pass (1 xfail).

Full proof (passed)

The installed empty-store proof on the 7 raw-capture cm27 bundles, 12:47–13:14 ET 2026-10-11: 6,414 Companies; GLEIF 3,716 records, 3,052 bound; bindings 9216cc18…, equal to the checkout replay and to #905's proof on the same bundles; recovery on attempt 2; the second pass changed nothing. SEC merge 4.0 to 2.8 min, GLEIF merge 1.9 to 1.3 min against #905's proof. Report .planning/workstreams/mdm-request-dedup/evidence/installed-company-proof-20261011.json.

Three-axis review done; its two faults (an unrelated batch between assessment and save; a refusal at save) fixed and tested in 3024c745. 241 MDM Postgres tests pass.

🤖 Generated with Claude Code