fix(lib): stop hydration minting commits — derived props and equal values are not edits - #1260
fix(lib): stop hydration minting commits — derived props and equal values are not edits#1260joepio wants to merge 2 commits into
Conversation
…lues are not edits Every reload re-signed and re-POSTed a commit for every hydrated resource (~17 for one table page), which is where the post-reload drain storm — the thing the e2e totals budgets are sized around — comes from. Instrumenting `outbox.markDirty` and `loroSetProperty` on a live reload named two writers: 1. The heal pass in `getLoroDoc` wrote `createdBy` into the Loro doc for every resource whose snapshot lacked it — which is ALL of them, because `createdBy` is not doc state: the server derives it from the genesis certificate's signing key and materializes it into JSON-AD. "In the cache but not in the doc" is its normal condition, not missing data. Each heal minted a real local op past the save cursor: subject dirty, drain signs, server acks a spurious full-state commit — and the re-encoded value carries a fresh timestamp that can beat a genuinely newer concurrent edit from another device in LWW. `createdBy` now joins `lastCommit`/`createdAt` in the never-write-to-doc set (deliberately narrower than the cache-preservation list: `parent`/`isA` healing is load-bearing, see parse.test's stale-snapshot case) and joins the cache-preservation list so it stays readable across rebuilds. 2. `loroSetProperty` created an op on every set, equal value or not — Loro's LWW registers don't dedupe — and hydration re-applies a resource's full JSON-AD on every collection re-query, several times per page load. Writes (and deletes of absent keys) that change nothing are now skipped; the stringify compare is false-negative-safe, falling through to the old behaviour when values aren't comparable. Measured on a reload of a one-table drive: 17 re-dirtied subjects → 1 (a genuine heal of a view prop its snapshot really lacks). Every one of those was a signed commit, a server round-trip, and an OPFS re-persist with a full index rebuild — per reload, per client.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Merge drops derived createdBy
- Added createdBy to serverManaged arrays in both merge paths to preserve the derived property from remote cache before rebuildCacheFromLoro.
- ✅ Fixed: Equal set leaves dirty stuck
- Changed loroSetProperty to return boolean and only set _dirty flag when it actually writes to Loro, preventing stuck unsaved state from equal-value sets.
Or push these changes by commenting:
@cursor push 48633fb678
You can send follow-ups to the cloud agent here.
Reviewed by Cursor Bugbot for commit 59703d9. Configure here.
| properties.commit.lastCommit, | ||
| commits.properties.createdAt, | ||
| 'https://atomicdata.dev/properties/createdBy', | ||
| ]; |
There was a problem hiding this comment.
Merge drops derived createdBy
Medium Severity
createdBy is now in NEVER_DOC_PROPS, so heal/seed no longer put it into Loro — correct for avoiding spurious commits. lastCommit and createdAt already had the same treatment and are explicitly copied from the remote cache in merge before rebuildCacheFromLoro. createdBy was not added to that copy list, so a merge of a remote that only carries createdBy in JSON-AD can drop it. Legacy resources without a genesis cert then lose getCreatedBy() after ingress merge.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit 59703d9. Configure here.
| } catch { | ||
| // Not comparable (cyclic, bigint…) — write as before. | ||
| } | ||
| } |
There was a problem hiding this comment.
Equal set leaves dirty stuck
Medium Severity
The new equal-value skip in loroSetProperty returns without writing a Loro op, so subscribeLocalUpdates never marks the outbox dirty and drain never runs. Resource.set still sets _dirty afterward. When a caller sets a value that stringifies equal to the current one, hasUnsavedChanges() stays true indefinitely, which blocks hydration gates and can leave the unsaved indicator stuck.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 59703d9. Configure here.



The bug
Every reload re-signed and re-POSTed a commit for every hydrated resource (~17 for a one-table drive). This is the "post-reload re-drain storm" the e2e totals budgets are sized around — and it's not just waste: the re-encoded values carry fresh timestamps that can beat a genuinely newer concurrent edit from another device in LWW.
Instrumenting
outbox.markDirtyandloroSetPropertyon a live reload named two writers:getLoroDocwrotecreatedByinto the Loro doc for every resource whose snapshot lacked it — which is all of them:createdByis not doc state. The server derives it from the genesis certificate's signing key and materializes it into JSON-AD, so "in the cache but not in the doc" is its normal condition. Each heal minted a real local op past the save cursor → subject dirty → drain signs → server acks a spurious full-state commit.loroSetPropertycreated an op on every set, equal value or not — Loro's LWW registers don't dedupe, and hydration re-applies a resource's full JSON-AD on every collection re-query, several times per page load.The fix
createdByjoinslastCommit/createdAtin the never-write-to-doc set (deliberately narrower than the cache-preservation list —parent/isAhealing is load-bearing, see parse.test's stale-snapshot case) and joins the cache-preservation list so it stays readable across rebuilds.Measured
Reload of a one-table drive: 17 re-dirtied subjects → 1 (a genuine heal of a view prop its snapshot really lacks). Each of those 17 was a signed commit, a server round-trip, and an OPFS re-persist with a full index rebuild — per reload, per client.
Verified
parent/isAscope decision)aggregatesfailing only under 2-worker load and passing solo (its known baseline flake)Once this has soaked, the e2e totals budgets (30s, all commented with this as their shrink condition) can come back down.
Note
Medium Risk
Touches core Loro hydration, heal, and write paths; wrong no-op detection or prop classification could drop real edits or break
parent/isAhealing, though stringify compare is conservative andNEVER_DOC_PROPSdeliberately excludes load-bearing heals.Overview
Fixes the post-reload re-drain storm: every hydrated resource was re-signed and re-POSTed on reload because hydration treated normal cache/doc divergence and redundant sets as real edits.
NEVER_DOC_PROPS(narrower than cache preservation) keepscreatedByout ofgetLoroDocseed/heal passes — it is server-derived and “in cache but not in doc” is expected; healing it minted dirty subjects.SERVER_MANAGED_PROPScentralizes the rebuild preserve list and addscreatedByso it stays readable afterrebuildCacheFromLoro.loroSetPropertynow skips deletes of absent keys and sets where the value is unchanged (JSON compare, safe fallback on compare failure), because Loro LWW registers an op on every set even when equal.Measured impact on a one-table drive reload: 17 re-dirtied subjects → 1 (a genuine heal).
Reviewed by Cursor Bugbot for commit 59703d9. Bugbot is set up for automated code reviews on this repo. Configure here.