Skip to content

fix(lib): stop hydration minting commits — derived props and equal values are not edits - #1260

Open
joepio wants to merge 2 commits into
developfrom
fix/hydration-mints-commits
Open

fix(lib): stop hydration minting commits — derived props and equal values are not edits#1260
joepio wants to merge 2 commits into
developfrom
fix/hydration-mints-commits

Conversation

@joepio

@joepio joepio commented Aug 7, 2026

Copy link
Copy Markdown
Member

The bug

Every reload re-signed and re-POSTed a commit for every hydrated resource (~17 for a one-table drive). This is the "post-reload re-drain storm" the e2e totals budgets are sized around — and it's not just waste: the re-encoded values carry fresh timestamps that can beat a genuinely newer concurrent edit from another device in LWW.

Instrumenting outbox.markDirty and loroSetProperty on a live reload named two writers:

  1. The heal pass in getLoroDoc wrote createdBy into the Loro doc for every resource whose snapshot lacked it — which is all of them: createdBy is not doc state. The server derives it from the genesis certificate's signing key and materializes it into JSON-AD, so "in the cache but not in the doc" is its normal condition. Each heal minted a real local op past the save cursor → subject dirty → drain signs → server acks a spurious full-state commit.
  2. loroSetProperty created an op on every set, equal value or not — Loro's LWW registers don't dedupe, and hydration re-applies a resource's full JSON-AD on every collection re-query, several times per page load.

The fix

  • createdBy joins lastCommit/createdAt in the never-write-to-doc set (deliberately narrower than the cache-preservation list — parent/isA healing is load-bearing, see parse.test's stale-snapshot case) and joins the cache-preservation list so it stays readable across rebuilds.
  • Writes (and deletes of absent keys) that change nothing are skipped. The stringify compare is false-negative-safe: values that aren't comparable fall through to the old behaviour.

Measured

Reload of a one-table drive: 17 re-dirtied subjects → 1 (a genuine heal of a view prop its snapshot really lacks). Each of those 17 was a signed commit, a server round-trip, and an OPFS re-persist with a full index rebuild — per reload, per client.

Verified

  • lib unit tests 225/225 (including the stale-snapshot heal case, which pinned the parent/isA scope decision)
  • e2e locally in an isolated worktree env: tables, sync, offline, aggregates, row-actions, table-templates, dashboard, documents — 35/36 with aggregates failing only under 2-worker load and passing solo (its known baseline flake)

Once this has soaked, the e2e totals budgets (30s, all commented with this as their shrink condition) can come back down.


Note

Medium Risk
Touches core Loro hydration, heal, and write paths; wrong no-op detection or prop classification could drop real edits or break parent/isA healing, though stringify compare is conservative and NEVER_DOC_PROPS deliberately excludes load-bearing heals.

Overview
Fixes the post-reload re-drain storm: every hydrated resource was re-signed and re-POSTed on reload because hydration treated normal cache/doc divergence and redundant sets as real edits.

NEVER_DOC_PROPS (narrower than cache preservation) keeps createdBy out of getLoroDoc seed/heal passes — it is server-derived and “in cache but not in doc” is expected; healing it minted dirty subjects. SERVER_MANAGED_PROPS centralizes the rebuild preserve list and adds createdBy so it stays readable after rebuildCacheFromLoro.

loroSetProperty now skips deletes of absent keys and sets where the value is unchanged (JSON compare, safe fallback on compare failure), because Loro LWW registers an op on every set even when equal.

Measured impact on a one-table drive reload: 17 re-dirtied subjects → 1 (a genuine heal).

Reviewed by Cursor Bugbot for commit 59703d9. Bugbot is set up for automated code reviews on this repo. Configure here.

joepio added 2 commits August 7, 2026 16:24
…lues are not edits

Every reload re-signed and re-POSTed a commit for every hydrated resource
(~17 for one table page), which is where the post-reload drain storm — the
thing the e2e totals budgets are sized around — comes from. Instrumenting
`outbox.markDirty` and `loroSetProperty` on a live reload named two writers:

1. The heal pass in `getLoroDoc` wrote `createdBy` into the Loro doc for
   every resource whose snapshot lacked it — which is ALL of them, because
   `createdBy` is not doc state: the server derives it from the genesis
   certificate's signing key and materializes it into JSON-AD. "In the cache
   but not in the doc" is its normal condition, not missing data. Each heal
   minted a real local op past the save cursor: subject dirty, drain signs,
   server acks a spurious full-state commit — and the re-encoded value
   carries a fresh timestamp that can beat a genuinely newer concurrent edit
   from another device in LWW. `createdBy` now joins `lastCommit`/`createdAt`
   in the never-write-to-doc set (deliberately narrower than the
   cache-preservation list: `parent`/`isA` healing is load-bearing, see
   parse.test's stale-snapshot case) and joins the cache-preservation list so
   it stays readable across rebuilds.

2. `loroSetProperty` created an op on every set, equal value or not — Loro's
   LWW registers don't dedupe — and hydration re-applies a resource's full
   JSON-AD on every collection re-query, several times per page load. Writes
   (and deletes of absent keys) that change nothing are now skipped; the
   stringify compare is false-negative-safe, falling through to the old
   behaviour when values aren't comparable.

Measured on a reload of a one-table drive: 17 re-dirtied subjects → 1 (a
genuine heal of a view prop its snapshot really lacks). Every one of those
was a signed commit, a server round-trip, and an OPFS re-persist with a full
index rebuild — per reload, per client.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

Fix All in Cursor

Bugbot Autofix prepared fixes for both issues found in the latest run.

  • ✅ Fixed: Merge drops derived createdBy
    • Added createdBy to serverManaged arrays in both merge paths to preserve the derived property from remote cache before rebuildCacheFromLoro.
  • ✅ Fixed: Equal set leaves dirty stuck
    • Changed loroSetProperty to return boolean and only set _dirty flag when it actually writes to Loro, preventing stuck unsaved state from equal-value sets.

Create PR

Or push these changes by commenting:

@cursor push 48633fb678

You can send follow-ups to the cloud agent here.

Reviewed by Cursor Bugbot for commit 59703d9. Configure here.

properties.commit.lastCommit,
commits.properties.createdAt,
'https://atomicdata.dev/properties/createdBy',
];

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge drops derived createdBy

Medium Severity

createdBy is now in NEVER_DOC_PROPS, so heal/seed no longer put it into Loro — correct for avoiding spurious commits. lastCommit and createdAt already had the same treatment and are explicitly copied from the remote cache in merge before rebuildCacheFromLoro. createdBy was not added to that copy list, so a merge of a remote that only carries createdBy in JSON-AD can drop it. Legacy resources without a genesis cert then lose getCreatedBy() after ingress merge.

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 59703d9. Configure here.

} catch {
// Not comparable (cyclic, bigint…) — write as before.
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Equal set leaves dirty stuck

Medium Severity

The new equal-value skip in loroSetProperty returns without writing a Loro op, so subscribeLocalUpdates never marks the outbox dirty and drain never runs. Resource.set still sets _dirty afterward. When a caller sets a value that stringifies equal to the current one, hasUnsavedChanges() stays true indefinitely, which blocks hydration gates and can leave the unsaved indicator stuck.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 59703d9. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant