Skip to content

ece94f47 - Align the schema-array and zap-ingest TSDoc with the schema docs and handbook - #112

Draft
TaprootFreak wants to merge 5 commits into
developfrom
ece94f47-schema-array-tsdoc
Draft

ece94f47 - Align the schema-array and zap-ingest TSDoc with the schema docs and handbook#112
TaprootFreak wants to merge 5 commits into
developfrom
ece94f47-schema-array-tsdoc

Conversation

@TaprootFreak

@TaprootFreak TaprootFreak commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

EN:
Eight doc comments and one handbook line described behaviour the code does not have, all of them follow-ups to #109 and all comment or prose lines only. Two still claimed that MESSAGE_SCHEMA_SQL and CONVERSATION_SCHEMA_SQL "match" their docs/schema files, although those omit the DO $unwrap$ boot repair by design. The array in db-change.ts carries the log_db_change trigger body that on UPDATE collapses an unchanged, unredacted bytea column to a reference, while its comment named only the one-time view_key rewrite. The zap-ingest comments and the handbook now state the same rule with the same bounds: a repeated identical outcome:reason is normally not written again, that is not a guarantee, and a changed decision always writes a new row.

DE:
Acht Doc-Kommentare und eine Handbuch-Zeile beschrieben ein Verhalten, das der Code nicht hat, alle als Nachtrag zu #109 und alle nur Kommentar- oder Fliesstext. Zwei behaupteten noch, MESSAGE_SCHEMA_SQL und CONVERSATION_SCHEMA_SQL würden ihren docs/schema-Dateien „entsprechen", obwohl diese den DO $unwrap$-Boot-Repair bewusst auslassen. Das Array in db-change.ts trägt den log_db_change-Triggerkörper, der beim UPDATE eine unveränderte, nicht redigierte bytea-Spalte auf einen Verweis zusammenzieht, während sein Kommentar nur die einmalige view_key-Umschreibung nannte. Die Zap-Ingest-Kommentare und das Handbuch nennen jetzt dieselbe Regel mit denselben Grenzen: ein wiederholtes identisches outcome:reason wird normalerweise nicht erneut geschrieben, das ist keine Garantie, und eine geänderte Entscheidung schreibt immer eine neue Zeile.

Details

What changed

File Symbol Why it was wrong
src/lib/message-store.ts MESSAGE_SCHEMA_SQL Said it matches docs/schema/message.sql, which omits the DO $unwrap$ boot repair by design
src/lib/conversation-store.ts CONVERSATION_SCHEMA_SQL Same, for docs/schema/conversation.sql
src/lib/db-change.ts DB_CHANGE_SCHEMA_SQL Named only the one-time view_key rewrite, and left out that the bytea reference is written on UPDATE only and only for a column that is both unchanged and unredacted
src/lib/nostr/zap-index.ts zapDecisions Did not record the asymmetry with deleteById: both store adapters forget the receipt id when the message goes away, this map does not
src/lib/nostr/zap-index.ts persistZapIngest Read as an unconditional skip, although the memory is set only after the write resolves
src/lib/nostr/zap-index.ts indexZapReceipt Claimed no further row is ever written for a known receipt, then claimed at most one per decision — both false
src/lib/nostr/zap-index.ts indexOpenZapReceipts Claimed a steady state writes nothing at all, and lost the conditional about the first tick after a restart
src/lib/nostr/zap-index.ts ingestOneReceipt Did not say which decisions short-circuit it, nor that every other reason is re-validated
docs/handbook/functions.md Ingest dedupe Stated the rule unconditionally, contradicting the corrected comments

Four commits, because the first three attempts were wrong in different directions

The first version said re-validating a known receipt writes no further ingest row. persistZapIngest skips only when the same outcome:reason recurs, so a changed decision does write another row — the repository's own test, persists again when a non-terminal decision later becomes indexed, asserts exactly two rows for that transition.

Correcting that overshot into a second false claim: that the same outcome:reason is persisted at most once per receipt id per store instance. It is not. persistZapIngest sets the memory only after recordZapIngest resolves, startNostrWorker fires each tick with setInterval without awaiting the previous one, and a failed write leaves the memory untouched. Nothing in the database enforces it either: recordZapIngest is a plain INSERT and the index on receipt_id is not unique. Two overlapping ticks or a retry can produce a second identical row.

The same round also found that the rewrite had dropped the handbook's "may" about the first tick after a restart. That hedge carries weight: the tick returns early when there are no relay urls or no event ids, and listLatest only covers currently recent notes, so a receipt whose message has aged out is never queried again at all.

The fourth commit brings the handbook and persistZapIngest in line, so the file no longer contradicts itself.

One asymmetry worth knowing

The decision memory lives in the process, not in the database, so it outlives the state it mirrors. When deleteById removes a message, both store adapters forget the receipt id and would record it again; this map does not.

It is not reachable today, and the reasoning is worth recording: the tick asks relays only for the note event ids returned by listLatest, which excludes deleted and soft-deleted rows in both adapters, and indexZapReceipt has no caller outside the tick. Two independent reviews tried to refute this and found no path. It would become reachable the moment the queried set included deleted or hidden posts, and then a zap would be skipped silently.

Verification on the pushed head

bun run typecheck        PASS
bun run lint             PASS
bun run handbook:check   PASS
bun run e2e:check        PASS
bun run test             PASS (86 files, 1434 tests)
bun run build            PASS

No executable code, schema or test changed in this pull request.

The MESSAGE_SCHEMA_SQL and CONVERSATION_SCHEMA_SQL comments claimed the
arrays "match" docs/schema/message.sql and docs/schema/conversation.sql;
those files mirror the DDL and reference the boot repair by comment,
while the DO $unwrap$ statement lives only in the arrays. Same wording
as the handbook now uses.
@TaprootFreak

Copy link
Copy Markdown
Contributor Author

EN:
One review round brought both Grok gates (quality and logic) to approved with zero findings on head b55d606, CI is green, and the PR stays in draft only because the mandatory Codex gate is unavailable on this host.

DE:
Eine Review-Runde hat beide Grok-Gates (Qualität und Logik) auf Head b55d606 mit null Funden auf approved gebracht, die CI ist grün, und der PR bleibt nur deshalb Draft, weil das Pflicht-Gate Codex auf diesem Host nicht verfügbar ist.

Details
  • Inner review (spec conformance): approved.
  • Quality gate: approved, 0 findings — TSDoc accurate against docs/schema/message.sql (lines 9-14) and docs/schema/conversation.sql (lines 4-9), consistent with the handbook Returns bullets, commit and PR body conform.
  • Logic gate: approved, 0 findings — comment-only diff, statements verified against the arrays' last statements.
  • Local: typecheck, lint, handbook:check green; CI green.
  • Codex gate: unavailable (CLI not installed on the review host).

@TaprootFreak
TaprootFreak marked this pull request as ready for review September 10, 2026 12:02
@TaprootFreak
TaprootFreak marked this pull request as draft September 10, 2026 20:46
The schema array in db-change.ts carries the log_db_change trigger body, and
that body now stores an unchanged bytea column as a sha256 reference instead of
the full value. Its doc comment still described only the one-time view_key
rewrite, although the file header and docs/schema/db_change.sql both describe
the new behaviour.

In zap-index.ts the ingest path returns early for a receipt whose terminal
decision this process already persisted, and persistZapIngest suppresses the
repeated write. The handbook describes that; the doc comments of
indexZapReceipt, indexOpenZapReceipts and ingestOneReceipt did not. They now
say where the skip happens and that the memory is process-local, so the first
tick after a restart may rewrite one row per known receipt.
@TaprootFreak TaprootFreak changed the title ece94f47 - Align the schema-array TSDoc with the schema docs ece94f47 - Align the schema-array and zap-ingest TSDoc with the schema docs and handbook Sep 10, 2026
A logic review of these comments found that two of them were simply wrong and a
third read as unconditional. Fixed against the code and the repository's own
tests.

indexZapReceipt said that re-validating a known receipt writes no further
ingest row. persistZapIngest only skips when the same outcome:reason repeats, so
a later different decision does write another row. The test "persists again when
a non-terminal decision later becomes indexed" asserts exactly that, two rows.
The comment now says so.

indexOpenZapReceipts said a steady state writes no ingest rows at all. Only
indexed and rejected:duplicate short-circuit; every other rejection reason is
re-validated on each tick and writes again when the decision changes. The claim
now carries that condition, and ingestOneReceipt names the counter-case too.

DB_CHANGE_SCHEMA_SQL described the bytea reference without its conditions. The
collapse happens on UPDATE only, and only for a column that is both unchanged
and not hashed by db_change_redact, which is what docs/schema/db_change.sql has
said all along.

The memory's doc comment now also records its asymmetry with deleteById: both
store adapters forget a receipt id when the message goes away and would record
it again, while this process-local map does not.
Two more review findings, both fair.

The claim that the same outcome:reason is persisted at most once per receipt id
per store instance was literally false. persistZapIngest sets the memory only
after recordZapIngest resolves, startNostrWorker fires each tick with setInterval
without awaiting the previous one, and a failed write leaves the memory
untouched. Two overlapping ticks or a retry can therefore write a second
identical row. The comment now says that the memory is consulted before the
write, that this is not a guarantee, and why.

The wording about the first tick after a restart had also lost the handbook's
"may". It is conditional for good reason: the tick returns early without
persisting anything when there are no relay urls or no event ids, and listLatest
only covers currently recent notes, so a receipt whose message has aged out is
never queried again at all. Both places are conditional again, and the tick's
own comment now records that ticks are not serialised.
…mments

The handbook still stated the dedupe rule unconditionally: "A repeated identical
outcome:reason is not written again." That contradicts the comments this pull
request just corrected, and the comments are the side that survived two rounds
of checking against the actual concurrency model. The handbook now carries the
same bound and the same three reasons.

persistZapIngest's own comment had the same problem in the same file: it read as
an unconditional skip, three lines below a comment saying that is not a
guarantee. It now says the memory is consulted before the write and set only
after it resolves, so two overlapping ticks can both pass the check.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant