ece94f47 - Align the schema-array and zap-ingest TSDoc with the schema docs and handbook - #112
Draft
TaprootFreak wants to merge 5 commits into
Draft
ece94f47 - Align the schema-array and zap-ingest TSDoc with the schema docs and handbook#112TaprootFreak wants to merge 5 commits into
TaprootFreak wants to merge 5 commits into
Conversation
The MESSAGE_SCHEMA_SQL and CONVERSATION_SCHEMA_SQL comments claimed the arrays "match" docs/schema/message.sql and docs/schema/conversation.sql; those files mirror the DDL and reference the boot repair by comment, while the DO $unwrap$ statement lives only in the arrays. Same wording as the handbook now uses.
Contributor
Author
|
EN: DE: Details
|
TaprootFreak
marked this pull request as ready for review
September 10, 2026 12:02
TaprootFreak
marked this pull request as draft
September 10, 2026 20:46
The schema array in db-change.ts carries the log_db_change trigger body, and that body now stores an unchanged bytea column as a sha256 reference instead of the full value. Its doc comment still described only the one-time view_key rewrite, although the file header and docs/schema/db_change.sql both describe the new behaviour. In zap-index.ts the ingest path returns early for a receipt whose terminal decision this process already persisted, and persistZapIngest suppresses the repeated write. The handbook describes that; the doc comments of indexZapReceipt, indexOpenZapReceipts and ingestOneReceipt did not. They now say where the skip happens and that the memory is process-local, so the first tick after a restart may rewrite one row per known receipt.
A logic review of these comments found that two of them were simply wrong and a third read as unconditional. Fixed against the code and the repository's own tests. indexZapReceipt said that re-validating a known receipt writes no further ingest row. persistZapIngest only skips when the same outcome:reason repeats, so a later different decision does write another row. The test "persists again when a non-terminal decision later becomes indexed" asserts exactly that, two rows. The comment now says so. indexOpenZapReceipts said a steady state writes no ingest rows at all. Only indexed and rejected:duplicate short-circuit; every other rejection reason is re-validated on each tick and writes again when the decision changes. The claim now carries that condition, and ingestOneReceipt names the counter-case too. DB_CHANGE_SCHEMA_SQL described the bytea reference without its conditions. The collapse happens on UPDATE only, and only for a column that is both unchanged and not hashed by db_change_redact, which is what docs/schema/db_change.sql has said all along. The memory's doc comment now also records its asymmetry with deleteById: both store adapters forget a receipt id when the message goes away and would record it again, while this process-local map does not.
Two more review findings, both fair. The claim that the same outcome:reason is persisted at most once per receipt id per store instance was literally false. persistZapIngest sets the memory only after recordZapIngest resolves, startNostrWorker fires each tick with setInterval without awaiting the previous one, and a failed write leaves the memory untouched. Two overlapping ticks or a retry can therefore write a second identical row. The comment now says that the memory is consulted before the write, that this is not a guarantee, and why. The wording about the first tick after a restart had also lost the handbook's "may". It is conditional for good reason: the tick returns early without persisting anything when there are no relay urls or no event ids, and listLatest only covers currently recent notes, so a receipt whose message has aged out is never queried again at all. Both places are conditional again, and the tick's own comment now records that ticks are not serialised.
…mments The handbook still stated the dedupe rule unconditionally: "A repeated identical outcome:reason is not written again." That contradicts the comments this pull request just corrected, and the comments are the side that survived two rounds of checking against the actual concurrency model. The handbook now carries the same bound and the same three reasons. persistZapIngest's own comment had the same problem in the same file: it read as an unconditional skip, three lines below a comment saying that is not a guarantee. It now says the memory is consulted before the write and set only after it resolves, so two overlapping ticks can both pass the check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
EN:
Eight doc comments and one handbook line described behaviour the code does not have, all of them follow-ups to #109 and all comment or prose lines only. Two still claimed that
MESSAGE_SCHEMA_SQLandCONVERSATION_SCHEMA_SQL"match" theirdocs/schemafiles, although those omit theDO $unwrap$boot repair by design. The array indb-change.tscarries thelog_db_changetrigger body that on UPDATE collapses an unchanged, unredacted bytea column to a reference, while its comment named only the one-timeview_keyrewrite. The zap-ingest comments and the handbook now state the same rule with the same bounds: a repeated identicaloutcome:reasonis normally not written again, that is not a guarantee, and a changed decision always writes a new row.DE:
Acht Doc-Kommentare und eine Handbuch-Zeile beschrieben ein Verhalten, das der Code nicht hat, alle als Nachtrag zu #109 und alle nur Kommentar- oder Fliesstext. Zwei behaupteten noch,
MESSAGE_SCHEMA_SQLundCONVERSATION_SCHEMA_SQLwürden ihrendocs/schema-Dateien „entsprechen", obwohl diese denDO $unwrap$-Boot-Repair bewusst auslassen. Das Array indb-change.tsträgt denlog_db_change-Triggerkörper, der beim UPDATE eine unveränderte, nicht redigierte bytea-Spalte auf einen Verweis zusammenzieht, während sein Kommentar nur die einmaligeview_key-Umschreibung nannte. Die Zap-Ingest-Kommentare und das Handbuch nennen jetzt dieselbe Regel mit denselben Grenzen: ein wiederholtes identischesoutcome:reasonwird normalerweise nicht erneut geschrieben, das ist keine Garantie, und eine geänderte Entscheidung schreibt immer eine neue Zeile.Details
What changed
src/lib/message-store.tsMESSAGE_SCHEMA_SQLdocs/schema/message.sql, which omits theDO $unwrap$boot repair by designsrc/lib/conversation-store.tsCONVERSATION_SCHEMA_SQLdocs/schema/conversation.sqlsrc/lib/db-change.tsDB_CHANGE_SCHEMA_SQLview_keyrewrite, and left out that the bytea reference is written on UPDATE only and only for a column that is both unchanged and unredactedsrc/lib/nostr/zap-index.tszapDecisionsdeleteById: both store adapters forget the receipt id when the message goes away, this map does notsrc/lib/nostr/zap-index.tspersistZapIngestsrc/lib/nostr/zap-index.tsindexZapReceiptsrc/lib/nostr/zap-index.tsindexOpenZapReceiptssrc/lib/nostr/zap-index.tsingestOneReceiptdocs/handbook/functions.mdFour commits, because the first three attempts were wrong in different directions
The first version said re-validating a known receipt writes no further ingest row.
persistZapIngestskips only when the sameoutcome:reasonrecurs, so a changed decision does write another row — the repository's own test,persists again when a non-terminal decision later becomes indexed, asserts exactly two rows for that transition.Correcting that overshot into a second false claim: that the same
outcome:reasonis persisted at most once per receipt id per store instance. It is not.persistZapIngestsets the memory only afterrecordZapIngestresolves,startNostrWorkerfires each tick withsetIntervalwithout awaiting the previous one, and a failed write leaves the memory untouched. Nothing in the database enforces it either:recordZapIngestis a plain INSERT and the index onreceipt_idis not unique. Two overlapping ticks or a retry can produce a second identical row.The same round also found that the rewrite had dropped the handbook's "may" about the first tick after a restart. That hedge carries weight: the tick returns early when there are no relay urls or no event ids, and
listLatestonly covers currently recent notes, so a receipt whose message has aged out is never queried again at all.The fourth commit brings the handbook and
persistZapIngestin line, so the file no longer contradicts itself.One asymmetry worth knowing
The decision memory lives in the process, not in the database, so it outlives the state it mirrors. When
deleteByIdremoves a message, both store adapters forget the receipt id and would record it again; this map does not.It is not reachable today, and the reasoning is worth recording: the tick asks relays only for the note event ids returned by
listLatest, which excludes deleted and soft-deleted rows in both adapters, andindexZapReceipthas no caller outside the tick. Two independent reviews tried to refute this and found no path. It would become reachable the moment the queried set included deleted or hidden posts, and then a zap would be skipped silently.Verification on the pushed head
No executable code, schema or test changed in this pull request.