Repository navigation
perf(engine): write-allocate the dataset cache and hoist OCC validation out of the write gates - #1291
Conversation
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
ragnorc
left a comment
There was a problem hiding this comment.
Review of head c4a8ede7.
Summary
Why. A write dropped the Lance dataset of the version it had just committed, so the next open of that pin read the manifest again. The write also probed the branch head of __manifest under its gates, inside the latch-held critical section.
What. Two changes:
- Write-allocate.
commit_allputs each committed detached dataset into the table handle cache, under the key that the published registration will carry. - Hoisted probe. A mutation sends its branch-head probe concurrently with its staging puts. Under the gates, it reuses that answer when this handle's installed coordinator still has the incarnation that the probe saw.
Impact. Fewer requests per write and per read-after-write: write_invalidates_table_cache_for_changed_table goes from 4 reads to 0, issue 817's repeated HEAD from 1 to 0, and nine DST scenarios drop. Optimize gains 2 GETs.
Verdict. The write-allocate part is sound. The key is version-exact and the pin is immutable, so a lost publish only leaves an entry that no open can reach. Two items block merge: the branch does not merge with main, and the hoisted probe changes the --if-commit contract when another handle or process publishes during staging.
Blocking
B1. Conflicts with main, and the clean part of the merge does not type-check
- GitHub reports
CONFLICTING.git merge-treeagainstmainde02c281shows text conflicts incrates/omnigraph-dst/cost_table.txt(_verify l.get: #1276 made it 1399, this PR makes it 1385) anddocs/dev/writes.md. crates/omnigraph-core/src/handle_cache.rsmerges without a text conflict, but the result does not type-check. The newTableHandleCache::insertcallsinner.insert(key, dataset)with aDataset. Since #1276,TableHandleCacheInner::inserttakesHeldHandle { dataset, manifest_bytes }, so the byte budget never weighs this entry. (I found this by reading the merged file. I did not compile the merge.)- Because of the conflict, CI on this head ran only
PR Title,RegressionandReview. No build, workspace test, GQT, DST or clippy job ran on this code.
After the rebase, regenerate the cost golden on top of #1276 and explain each changed line again.
Suggested shape after the rebase (it removes liability). #1276 already has one place where a publish changes the branch's working set: evict_superseded_handles in table_ops.rs, after the publish. Admit the new pin in the same place, with one cache method (for example replace(old_key, new_key, dataset)) that builds keys with TableHandleKey::new and measures manifest_bytes the way a miss does. The committed datasets already reach the publish step in CommittedMutation::detached, which is discarded today (detached: _ in exec/mutation.rs:1412 and loader/mod.rs:1504). This gives three things:
- The key comes from the
DatasetUpdatethat is actually published, not from a second derivation instaging.rs, so the two cannot drift. A drift would show up only as a silent miss. - A lost publish leaves no entry that uses count and byte budget.
- "A publish moves the branch from its base pin to its new pin" then has one owner in the cache.
The next operation gets the same benefit.
B2. A conditional update/delete returns ReadSetChanged (409) instead of PreconditionFailed (412) when another handle or process publishes during staging
Contract. skills/omnigraph/references/remote-ops.md says: "Any intervening branch commit makes the condition fail without effects. The CLI exits 4; HTTP returns 412." The existing test conditional_update_and_delete_races_return_precondition_failed_before_effects states that the under-gate check must map update and delete to PreconditionFailed and not leak ReadSetChanged.
Cause. revalidate_write_txn_with (db/omnigraph.rs:2132-2139) trusts the early probe when installed.manifest_incarnation() still matches early.held. That incarnation belongs to this handle. The write gates are shared by every handle on the root in the process (see cross_handle_branch_gate_serializes_post_effect_publish), but the installed coordinator is not shared. So a publish between the probe and the gates, by a second handle on the same root or by another process, does not change this handle's incarnation.
What happens next:
- Revalidation keeps the stale head.
- The caller-head check at
db/omnigraph.rs:2168passes. - The detached commits run.
- The publisher's
ExactGraphHeadcheck refuses withReadSetChanged.
Update and delete are not retryable, so this internal classification reaches the caller. The description says "a same-process publish installs a new incarnation". That is true only for the same handle.
Evidence. I added a temporary copy of the existing test with one change: the winner writes through a second Omnigraph::open on the same root. The writer runs a conditional set_age and is parked at mutation.post_stage_pre_effect_gate. That is after staging and the early probe, and before the gates. The winner inserts a Company, then the writer is released. Command:
cargo test -p omnigraph-engine --locked --features failpoints --test failpoints review_probe -- --nocapture
- Base
d794b725(the PR's six source files reverted in the same worktree):PreconditionFailed { branch: "main", expected: <captured head>, actual: Some(<winner head>) }, which is the documented outcome. - Head
c4a8ede7:Manifest(ManifestError { kind: Conflict, … details: Some(ReadSetChanged { member: "graph_head:main", … }) }).
In the same race, an unconditional write now also fails after its detached commits instead of before them. That is still effect-free for the graph, but it leaves orphaned detached versions for cleanup.
Fix at the owner. The caller's precondition belongs to publication, not to where the probe runs. When caller_expected_graph_head is set, map a graph-head ReadSetChanged from the publisher to PreconditionFailed, with the actual head, as revalidation already does. This holds for every race path. It also closes the shorter cross-process window between the under-gate probe and the CAS that exists on main today. I found that window by reading the code only; I could not reproduce it in-process, because same-process handles share the gates.
Skipping the early probe only for conditional writes would be a symptomatic patch: it leaves that window open.
Regression test. Extend conditional_update_and_delete_races_return_precondition_failed_before_effects with a second-handle winner, for both set_age and remove_person. It must fail at c4a8ede7 and pass after the fix. GQT cannot express this case, because sessions on separate handles are not supported yet (crates/omnigraph-gqt/README.md), so the failpoints test is the cheapest owner.
Optional
- O1. One owner for the freshness decision.
revalidate_write_txn_withadds a third path besidewrite_authority_for_known_branch(branch, fresh), which already computes held, probe andmatches. Pass the early answer into that function as one three-state input (probe now / probed earlier / trust installed). Then one function decides when a head counts as current. With the current shape, five more callers give five copies of the rule. - O2. Make the incarnation root-scoped if in-process multi-handle contention matters. For example, keep a publication counter with the root-scoped branch gate. A same-process publish by any handle then invalidates the early answer before effects, instead of after detached commits.
- O3. State the tradeoff in
docs/dev/writes.md. Under contention from another handle or process, a lost race now costs one detached commit per table and fails after effects, not before them. With one server writer per root this case is probably rare. One sentence makes the tradeoff visible. - O4.
loader/mod.rsnow givesplan_loadthe cache-backed snapshot, not onlyplan_write. That is fine, but the description does not mention it. - O5. Overlap with open PRs. #1275 edits the same
cost_table.txtlines andwrites.md, and it changesoptimize. Re-measure theOptimize +2 GETsline after whichever lands second.
Validation
All runs used an isolated worktree at c4a8ede7, a debug build and a shared target directory.
cargo test -p omnigraph-gqt --locked --test gq_logic_tests -- writer_parked_in_its_early_probe: 2 passed.-- two_table_mutation_fails_before_its_gates: 1 passed (DST seeds 0 and 42, replay matched).cargo test -p omnigraph-gqt --locked --test runner_dispatch -- measure_counts_schema_contract_requests_issue_817: 1 passed.cargo test -p omnigraph-engine --locked --features failpoints --test forbidden_apis: 39 passed.… --test integration warm_read_cost::write_invalidates_table_cache_for_changed_table: passed.… --test failpoints conditional_update_and_delete_races_return_precondition_failed_before_effects(same-handle winner): passed.- The B2 probe as described above: passes at base, fails at head. I removed it afterwards.
git merge-tree --write-treewithmainde02c281: two conflicts, plus the type mismatch from B1 (by inspection).- CI on this head: only 3 checks ran, because of the conflict.
Limits. I did not build or test the merged tree. I did not run the full workspace gate, the DST suite (the cost golden values are the PR's own), clippy, or S3/Azure. I did not measure latency, so the critical-section gain is not verified here. I did not test cross-process races directly.
c4a8ede to
c502dd5
Compare
ragnorc
left a comment
There was a problem hiding this comment.
Review of head 2487c5e0. This follows my review of c4a8ede7.
Summary
Why and impact (unchanged). The PR keeps each Lance dataset a write commits in the table handle cache, and sends the branch-head probe beside the staging puts instead of under the write gates. Fewer object-store requests per write and per read-after-write: nine DST scenarios drop, and Optimize gains 2 GETs.
What changed since c4a8ede7. The commit was rebased onto 0851d796, which includes #1276. It is now c502dd59. Then main de02c281 was merged in as 2487c5e0. The merge needed no manual resolution (git show --remerge-diff is empty). The range-diff touches three places:
TableHandleCache::insertnow builds its key withTableHandleKey::newand stores aHeldHandlewith measuredmanifest_bytes. The byte budget now counts the entry.cost_table.txt_verify l.getis re-measured on top of #1276: 1399 → 1389.docs/dev/writes.mdis rebased onto #1276's eviction paragraph.
revalidate_write_txn_with, early_revalidation and the mutation path are the same as in c4a8ede7.
Verdict. Old B1 is fixed. Old B2 is still there: I reproduced it on this head. A new docs check failure also blocks merge, because of when the PR adds its release note.
Blocking
B1. Old B2 is still there: a conditional update or delete returns ReadSetChanged (409), not PreconditionFailed (412), when another handle publishes during staging
The code did not change, so the cause is the one I described before. revalidate_write_txn_with (crates/omnigraph/src/db/omnigraph.rs) trusts the early probe when installed.manifest_incarnation() still matches early.held. That incarnation belongs to this handle. A publish by a second handle on the same root, or by another process, does not change it. Revalidation then keeps the stale head. The caller-head check passes, the detached commits run, and the publisher's CAS refuses with ReadSetChanged. Update and delete are not retryable, so the 409 reaches the caller. The documented contract is 412 (skills/omnigraph/references/remote-ops.md, and the test conditional_update_and_delete_races_return_precondition_failed_before_effects).
The description still says "the __manifest compare-and-swap still refuses a cross-process publish". That is true, but for a conditional write the refusal has the wrong type.
Evidence on this head. I rebuilt the earlier temporary probe as a copy of the existing test with one change: the winner writes through a second Omnigraph::open on the same root. The writer runs a conditional set_age, then a conditional remove_person, and is parked at mutation.post_stage_pre_effect_gate. That is after staging and the early probe, and before the gates. The winner inserts a Company, then the writer is released.
cargo test -p omnigraph-engine --locked --features failpoints --test failpoints review_probe -- --nocapture
- Base: the PR's six source files set to
mainde02c281in the same worktree. Both cases returnPreconditionFailed { branch: "main", expected: <captured head>, actual: Some(<winner head>) }. 1 passed. - Head
2487c5e0: both cases returnManifest(ManifestError { kind: Conflict, … details: Some(ReadSetChanged { member: "graph_head:main", … }) }). 1 failed.
Fix at the owner (unchanged). The caller's precondition belongs to publication. When caller_expected_graph_head is set, map a graph-head ReadSetChanged from the publisher to PreconditionFailed, with the actual head. This covers every race path. It also closes the shorter cross-process window between the under-gate probe and the CAS that exists on main today. Do not skip the early probe only for conditional writes: that is a symptomatic patch, and it leaves that window open.
Regression test. Extend conditional_update_and_delete_races_return_precondition_failed_before_effects with a winner on a second Omnigraph::open, for both set_age and remove_person. The probe above is that test. It must fail at this head and pass after the fix. GQT cannot express separate handles, so the failpoints test is the cheapest owner.
B2. Check AGENTS.md Links fails because the new release note arrived after the v0.15.0 snapshot
CI on 2487c5e0: python3 scripts/check-docs.py fails with release notes: release notes changed after snapshot generation; regenerate it, and CI Gate fails with it. All other checks pass.
Cause: #1283 committed the v0.15.0 snapshot (docs/releases/v0.15.0.md), which records the digest of each note in the cycle (changelog.d/release.json still names v0.15.0). This PR adds changelog.d/write-keeps-committed-handle.performance.md. I compared the recorded and expected note sets at this head: the only difference is that file (recorded 64, expected 65). check-docs.py passes on main de02c281.
This is not a code defect. It is a choice for the release owner, and the PR cannot merge before one of these happens:
- v0.15.0 is tagged and the next cycle opens (
release.jsonbumped,docs/dev/documentation.mdstep 8). The note then belongs to the next release. No change to this PR is needed beyond a rerun. - The note ships in v0.15.0. Then regenerate the snapshot with
python3 scripts/release_notes.py snapshot --target HEAD --date <date> --replaceand commit it with this PR.
Every other open PR that adds a note will hit the same check until one of these happens.
Optional
- O1. Admission happens before publication, and eviction after it.
commit_allinserts the new pin before the publisher runs. #1276'sevict_superseded_handlesremoves the base pin only after publication. The two compose correctly: the eviction key is the exact base key, so it never removes the new pin. But a lost publish now leaves an entry that uses count and byte budget until LRU removes it. It is harmless for correctness. The key is built from the same values as the adjacentDatasetUpdate, so the drift risk I raised before is now small. My earlier suggestion still applies if you want one owner: admit the pin next toevict_superseded_handles, from the publishedDatasetUpdateandCommittedMutation::detached. If you keep the current shape, say indocs/dev/writes.mdthat the writer admits the committed pin at commit time. The section describes only the eviction half today, and the description sayswrites.mdcarries the contract. - O2. Old O1 to O4 (one owner for the freshness decision, root-scoped incarnation, the tradeoff sentence in
writes.md, theplan_loadsnapshot change in the description) are not addressed. They still apply. The fix for B1 removes most of the reason for the O3 sentence, but a lost race still costs one detached commit per table. - O3. Overlap with open PRs.
git merge-treewith #1275 (0c9a0a18), #1282 (565978da) and #1224 (ce5c89ed) is clean. #1275 still changesOptimizecosts andwrites.md, so re-measure the cost golden after whichever lands second.
Validation
- Isolated worktree at
2487c5e0. Debug build with a shared target directory. - Range-diff
c4a8ede7→c502dd59, andgit show --remerge-diff 2487c5e0(empty). - B1 probe (
cargo test -p omnigraph-engine --locked --features failpoints --test failpoints review_probe -- --nocapture): passes at basede02c281(PR source files reverted), fails at2487c5e0for bothset_ageandremove_person. python3 scripts/check-docs.pywith the pinned docs packages: fails at2487c5e0with the release-note error, passes atde02c281. A digest comparison ofdocs/releases/v0.15.0.mdagainst HEAD shows only the new note as different.- CI on this head: 26 pass, 4 skipped, 2 fail (
Check AGENTS.md Links,CI Gate). Test Workspace, Lint (clippy), the GQT jobs, DST pinned suite, S3/Azure and storage-upgrade jobs pass. git merge-tree --write-treewith #1275, #1282 and #1224 heads: no conflicts.
Limits. I did not run the full workspace gate, DST, GQT, clippy or S3/Azure locally; CI covers them on this head. I did not measure latency. I did not test cross-process races directly; the second-handle probe covers the same cause. I removed the temporary probe afterwards.
2487c5e to
51f9c4a
Compare
ragnorc
left a comment
There was a problem hiding this comment.
Review of head 51f9c4a9. This follows my reviews of c4a8ede7 and 2487c5e0. It covers only what changed since 2487c5e0.
Summary
What changed. The branch was rebased onto main 2d8effab. The perf commit is now 9ae5b473, and git range-diff shows it is the same patch as c502dd59. The main delta since the old merge (de02c281..2d8effab, #1293) changes only release CI, install docs and release notes. It touches none of this PR's files. git diff 2487c5e0 51f9c4a9 -- crates/ Cargo.toml Cargo.lock is empty, so the engine code is the same as the code I reviewed. A new commit, 51f9c4a9, deletes changelog.d/write-keeps-committed-handle.performance.md.
Verdict. Old B2 (the docs check) now passes. Old B1 is still there, because the code did not change. Withdrawing the note causes a new PR Title failure and leaves a release-note gap that needs a decision before merge.
Blocking
B1. Old B1 is still there: a conditional update or delete returns ReadSetChanged (409), not PreconditionFailed (412), when another handle publishes during staging
revalidate_write_txn_with and the early probe are the same as at 2487c5e0, so my reproduction and the cause are unchanged. Please see B1 in the review of 2487c5e0. The fix is also unchanged: when caller_expected_graph_head is set, map a graph-head ReadSetChanged from the publisher to PreconditionFailed with the actual head. Add the regression test as a second-Omnigraph::open winner in conditional_update_and_delete_races_return_precondition_failed_before_effects, for both set_age and remove_person.
I did not run the probe again. The crates/ tree is the same as the tree on which it failed, so the result cannot be different.
Release note: decide before merge
51f9c4a9 fixes the old docs failure. But PR Title now fails at this head: "a perf pull request adds a changelog.d note, or carries the skip-changelog label" (run). PR Title is not a required check, so it does not stop the merge. That is the risk:
- The
skip-changeloglabel is not correct here.docs/dev/documentation.mdkeeps it for typed changes with "nothing users can see". This change removes requests on every write and read-after-write, and it adds 2 GETs toOptimize. Users can see both. - v0.15.0 is not tagged yet, and
release.jsonstill namesv0.15.0. If the v0.15.0 tag lands on a commit that includes this PR, the change ships in v0.15.0 with no note. A note that a later PR adds would then appear in the wrong release.
Two clean paths:
- Merge after v0.15.0 is tagged and the next cycle opens. Revert
51f9c4a9at that time, so the note goes into the next release. This is what the commit title says. - Ship in v0.15.0. Restore the note and regenerate the snapshot with
python3 scripts/release_notes.py snapshot --target HEAD --date <date> --replacein this PR.
Do not merge with the note withdrawn and the label added.
Optional
- O1. Old optional items. O1 to O3 of the
2487c5e0review are not addressed. They still apply. - O2. New overlap with #1272 (
d3336e7b, patch an update's assigned columns). It changesexec/staging.rs,exec/mutation.rs,loader/mod.rs,forbidden_apis.rs,docs/dev/writes.mdandcost_table.txt.git merge-treewith this head is clean. Its newPendingMode::ColumnPatchis staged throughprepare_pending_tableand committed by the sameStagedMutation::commit_all, so this PR's handle admission and early probe cover it with no extra work. The DST cost golden needs a new measurement after the second of #1272, #1275 and #1299 lands. All three changecost_table.txtor write costs.
Validation
git range-diff de02c281..2487c5e0 2d8effab..51f9c4a9: the perf commit is unchanged (=). The only new commit is the note deletion.git diff 2487c5e0 51f9c4a9 -- crates/ Cargo.toml Cargo.lock: empty. The full tree diff is #1293 plus the note deletion.git merge-tree --write-treewith the current heads of every open PR that shares a file: clean with #1299, #1282, #1275, #1272 and #1235. #1217, #1184, #1175 and #818 conflict, but each of them already conflicts withmainalone, so this PR does not cause it.- CI on
51f9c4a9:Check AGENTS.md LinksandCI Gatenow pass. Test Workspace, Lint (clippy), the GQT jobs, the DST pinned suite, S3/Azure and storage-upgrade jobs pass.PR Titlefails as described above. Release tag jobs are skipped.
Limits. I did not build or run tests locally for this head, because its engine code is the same as the code I tested at 2487c5e0. I did not measure latency. I did not run #1272 and this PR together; the overlap note above comes from code reading and merge-tree.
51f9c4a to
8f6f6c4
Compare
ragnorc
left a comment
There was a problem hiding this comment.
Review of head 8f6f6c43. This follows my review of 51f9c4a9. It covers only what changed since then.
Summary
What changed. The branch was rebased onto main 373834b4, which is the v0.15.0 tag. git range-diff shows that the two earlier commits are the same patches (9ae5b473 = ccbe8f0e, 51f9c4a9 = d7f64038). One new commit, 8f6f6c43, does two things:
- It restores
changelog.d/write-keeps-committed-handle.performance.md. The bytes are the same as the note thatd7f64038removed (blob02ebb252). - It opens the next cycle in
changelog.d/release.json:versionv0.15.1,baserefs/tags/v0.15.0.
Why this matters. v0.15.0 is now tagged and published. The note can no longer go into v0.15.0, so it must go into the next cycle. This is path 1 from my last review. PR Title now passes.
Verdict. The release-note item from my last review is closed. B1 is still open. I reproduced it again on this head, on the new main base.
Blocking
B1. Still open: a conditional update or delete returns ReadSetChanged (409), not PreconditionFailed (412), when another handle publishes during staging
The PR's own code did not change. The cause and the fix are the same as in my reviews of 2487c5e0 and 51f9c4a9: when caller_expected_graph_head is set, map a graph-head ReadSetChanged from the publisher to PreconditionFailed, with the actual head. Add the second-handle winner to conditional_update_and_delete_races_return_precondition_failed_before_effects for set_age and remove_person.
main changed under the PR, including the update path (#1272). So I ran the probe again on this head instead of relying on the earlier result.
Evidence on this head. The probe is a copy of conditional_update_and_delete_races_return_precondition_failed_before_effects with one change: the winner writes through a second Omnigraph::open on the same root. The writer runs a conditional set_age, then a conditional remove_person, and is parked at mutation.post_stage_pre_effect_gate. The winner inserts a Company, then the writer is released.
cargo test --workspace --exclude omnigraph-gqt --exclude omnigraph-gqt-served --exclude omnigraph-dst --locked --features omnigraph-engine/failpoints,omnigraph-cluster/failpoints --test failpoints review_probe -- --nocapture
- Base: the PR's six source files (
handle_cache.rs,db/mod.rs,db/omnigraph.rs,exec/mutation.rs,exec/staging.rs,loader/mod.rs) set tomain373834b4in the same worktree. Both cases returnPreconditionFailed { branch: "main", expected: <captured head>, actual: Some(<winner head>) }. 1 passed. - Head
8f6f6c43: both cases returnManifest(ManifestError { kind: Conflict, … details: Some(ReadSetChanged { member: "graph_head:main", … }) }). 1 failed.
So the result does not depend on #1272 or the other main changes. GQT cannot express two handles on one root, so the failpoints test stays the cheapest owner for the regression.
Optional
- O1. Open the v0.15.1 cycle once, with the package version.
docs/dev/documentation.mdstep 8 opens a cycle with two edits:release.jsonand the workspace package version. Every crate is still0.15.0. Edge builds stay refused until the package version is above the latest stable release (scripts/edge_release.py check-version). This PR, #1316, #1317 and #1318 each carry the samerelease.jsonhunk and none bumps the crates. The identical hunks merge cleanly (git merge-treewith #1318 is clean), so this does not block. But a small cycle-opening PR that does both edits gives the release one owner. It also lets each feature PR keep only its note. This is a decision for the release owner, not a defect of this PR. - O2. #1272 now on
main. My last review predicted that #1272'sPendingMode::ColumnPatchwould get the write-allocate path at no extra cost. That is true on this head:ColumnPatchbecomesPreparedPendingTable::Keyed(crates/omnigraph/src/exec/staging.rs:959-977) and commits throughStagedMutation::commit_all, which inserts the committed dataset into the handle cache (staging.rs:1300-1317). The DST pinned suite passes with the PR'scost_table.txt, so the cost golden needs no new measurement for #1272. - O3. Old optional items. The old O1 to O4 (one owner for the freshness decision, root-scoped incarnation, the tradeoff sentence in
writes.md, theplan_loadsnapshot note) are not addressed. They still apply.
Validation
git range-diff 2d8effab..51f9c4a9 373834b4..8f6f6c43: both old commits=; one new commit.git show 8f6f6c43:release.json(2 lines) and the restored note (1 line). No code.- Isolated worktree at
8f6f6c43, pinned docs packages:python3 scripts/release_notes.py preview --target HEAD:# Omnigraph v0.15.1,previousv0.15.0, base373834b4, one selected note (write-keeps-committed-handle.performance.md) under Performance.python3 scripts/check-docs.py:Documentation OK (476 Markdown files checked).python3 scripts/test_release_notes.py: 96 tests, OK.
- B1 probe: see B1 above.
git merge-tree --write-treeagainst the open PRs that share a file: clean with #1318, #1316, #1317, #1282 and #1275. #1235, #1217, #1184 and #1175 conflict, but each of them already conflicts withmainalone.- CI on
8f6f6c43(run): Test Workspace, Lint (clippy), the GQT jobs, the DST pinned suite, S3/Azure, storage-upgrade andPR Titlepass.Dependency Guard (cargo deny)andCI Gatefail. The cause is three newrmcp 3.3.0advisories (RUSTSEC-2026-0457, RUSTSEC-2026-0461, RUSTSEC-2026-0462). This PR does not changeCargo.lock, and #1318 fails the same way. The lastmainrun (373834b4, 06:47 UTC) passed before these advisories arrived. A fix belongs in a separate dependency change.
Limits. I did not run the full workspace gate, DST, GQT, clippy or S3/Azure locally; CI covers them on this head. I did not measure latency. I removed the temporary probe afterwards.
ragnorc
left a comment
There was a problem hiding this comment.
Review of head 565d116a. This follows my review of 8f6f6c43. It covers only what changed since then.
Summary
What changed. One new commit, 565d116a ("review"), on top of 8f6f6c43. No rebase: main is still 373834b4. It does two things:
- B1 fix.
commit_updates_on_branch_with_expected(crates/omnigraph/src/db/omnigraph/table_ops.rs) now passes the publisher's error throughcaller_precondition_at_publishon both publish paths (bound branch and other branch). When the write carries the caller's--if-commithead and the publisher refuses withReadSetChangedongraph_head:<branch>, the error becomesPreconditionFailed { branch, expected, actual }.actualis the head the publisher read. The failpoints testconditional_update_and_delete_races_return_precondition_failed_before_effectsgains the two second-handle cases that my probe used.docs/dev/writes.mdstates the contract and the cost of the lost race. - Dependency fix.
rmcpgoes from=3.3.0to=3.9.0for RUSTSEC-2026-0457, -0461 and -0462.ServerInfobecomesServerConfig(the alias was deprecated in rmcp 3.4.0). A new note,changelog.d/rmcp-advisories.fixed.md, records it.
Why this matters. B1 was the only blocking item. A conditional update or delete now gets the same terminal 412 whether the race is caught under the gates or at the manifest CAS. The rmcp bump turns Dependency Guard (cargo deny) and CI Gate green.
Verdict. Ready to merge. B1 is fixed by design at the publish call, and I proved the regression: it fails without the fix for the predicted reason and passes on this head. No blocking items remain. The optional items below are cheap, and O1 (land the rmcp bump separately) helps every other open PR.
Blocking
None.
Optional
- O1. Land the
rmcpbump on its own, first. The advisories affectmainand every open PR, not only this one. #1318 already failscargo deny. #1316, #1317, #1311, #1282 and #1275 passed only because their last runs came before the advisories. Each of them fails on its next push untilmainhas the bump. If the bump stays here, it reachesmainonly when this perf change merges, and the squash subjectperf(engine): …hides a security fix. A separatefix(deps)PR made from these four hunks (Cargo.lock,omnigraph-mcp/Cargo.toml,omnigraph-mcp/src/lib.rs, the note) fixesmainnow and gives the fix one owner. Then rebase this PR on it. - O2. The test's doc comment now describes only half of the cases. It says the "authoritative under-gate check" maps both shapes. In the second-handle cases, the refusal comes from the publisher's CAS, after one detached commit per table. The final assertion compares the published pin and the linear
latest_version_id. A detached commit moves neither, so "must fail before any Person effect" means no graph-visible effect in those cases. Please change the comment to name both refusal points, aswrites.mdnow does. - O3. Only the bound-branch path is tested. The mapping is also added to the non-bound path (
table_ops.rs, thecommit_updates_on_branch_with_expectedbranch for a branch the handle is not bound to). No test reaches it with a second-handle winner. One more loop row, a conditionalset_ageon a named branch with the winner on a second handle, would cover it at almost no cost. - O4. Old optional items. O1 to O4 from my review of
51f9c4a9(one owner for the freshness decision, root-scoped incarnation, the tradeoff sentence inwrites.md, theplan_loadsnapshot note) still apply. The newwrites.mdparagraph covers part of the third one.
Tradeoffs and liability
- The B1 fix is at the right layer. The caller's precondition is checked in two places: under the gates (
revalidate_write_txn_with) and, after the early probe, implicitly by the publisher'sExactGraphHeadCAS. The fix makes the second place give the first place's answer. It does not add a probe, a lock or a retry. It is one function in the module that owns the publish call. It adds one coupling: it matches the publisher's member stringgraph_head:<branch>(GraphHeadExpectation::read_set_member,omnigraph-catalog/src/publisher.rs). The under-gate check and the merge path already build the same string, so this is the third copy of that format. A typed member (for example aReadSetMember::GraphHead { branch }inManifestConflictDetails) would make the match hold by construction. That is optional and belongs with a change that touches the conflict type anyway. - The mapping is sound. At prepare time, the write refuses unless the effective head equals the caller's expected head (
exec/mutation.rs). So when the publisher later reports that the graph head moved, the caller's expected head no longer holds either. APreconditionFailedis then always correct, and it is terminal, which matches the update/delete rule that a moved head is never replayed. - The cost of the hoisted probe is now written down. A publish from another handle or process between the early probe and the gates is refused at the CAS, after the detached commits, not before them. The collector reclaims those commits. That is within invariant 5 (pre-publication effects stay unreachable), and
writes.mdnow says it. rmcp3.9.0. I compared the two crate sources.StreamableHttpServerConfig::default()has the sameallowed_hosts; Omnigraph still setslegacy_session_mode,json_response,max_request_body_bytes, hosts and origins itself; the newvalidate_empty_origin_allowlistdefaults tofalse.ProtocolVersion::KNOWN_VERSIONSis the same set.LATESTmoves from2025-11-25to2026-07-28, but initialize negotiation still answers a supported client version with that version and otherwise falls back to the newest version that has initialize,2025-11-25. So the note's "The MCP endpoint is unchanged" holds for initialize clients. The server MCP tests and the CLI relay tests pass in CI on this head.
Validation
git show --stat 565d116a: 7 files, +80/−23.git merge-basewithorigin/mainis still373834b4.- B1 regression, after the fix. Isolated worktree at
565d116a:
cargo dev-test --test failpoints conditional_update_and_delete_races_return_precondition_failed_before_effects -- --nocapture
(the.cargo/config.tomlalias:--workspace --exclude omnigraph-gqt --exclude omnigraph-gqt-served --exclude omnigraph-dst --locked --features omnigraph-engine/failpoints,omnigraph-cluster/failpoints). Result: 1 passed. All four rows (set_age,remove_person, each on the same handle and on a second handle) returnPreconditionFailedwith the winner's head. - B1 regression, before the fix. The same worktree and command, with only
crates/omnigraph/src/db/omnigraph/table_ops.rsset to8f6f6c43(the new test kept). Result: 1 failed, atfailpoints.rs:2000:conditional set_age race (winner on a second handle: true) must be PreconditionFailed, got: write authority 'graph_head:main' changed during preparation (expected hb1….2…, current hb1….3…). This is the predicted reason: the publisher'sReadSetChangedreached the caller. The two same-handle rows passed before reaching it. I restored the file afterwards; the worktree is clean at565d116a. - GQT cannot express this case: its
--- concurrentwriters share one engine handle. So the failpoints test stays the right owner, as in my last review. rmcp: readrmcp-3.9.0/CHANGELOG.md(3.4.0 to 3.9.0) and comparedStreamableHttpServerConfig::default(),ProtocolVersionandnegotiate_protocol_versionin the 3.3.0 and 3.9.0 crate sources. See the last tradeoff item.git merge-tree --write-treewith #1318, #1317, #1316, #1311, #1302, #1282 and #1275: all clean. #1184 conflicts, but it already conflicts withmainalone.- CI on
565d116a(run): every check passes, includingDependency Guard (cargo deny),CI Gate,Test Workspace,Lint (clippy), the GQT jobs, the DST pinned suite, RustFS S3 (default and failpoints), Azurite, storage upgrade andPR Title.
Limits. I ran only the focused failpoints test locally. CI covers the workspace gate, clippy, GQT, DST and the S3/Azure suites on this head. I did not run the MCP endpoint against a live client, and I did not measure latency. The new commit does not touch a Lance API, so no new Lance page applies beyond my earlier reviews.
Main's ModernRelay#1291 keeps the dataset a write commits in the table handle cache, so a write that follows another on the same handle opens no manifest. The keyed-write scenario runs insert, update, merge-present and merge-new in turn on one reopened handle: the insert starts cold and is unchanged; the three later writes drop l.get (update 11 -> 8, merge present 9 -> 8, merge new 7/5 -> 4 at 2/16 fragments).
…nRelay#1318) * perf(mutation): an update narrows a keyed match through the id index A `<key> = v` conjunct also selects the derived `id = v`, so an update's match scan adds that conjunct when the pinned table has an id BTREE, as a node delete already does. The scan then reads the index instead of the key column of every fragment. * perf(storage): an all-new or all-present upsert skips the full-table join Lance's insert-capable merge-insert route joins the source against every id of the table. With an id BTREE, the exact-id probe a strict insert runs decides whether an upsert's rows are all new or all present; the first stages through the absence-proven insert writer, the second as a known-present update. A mixed batch, or a table without the BTREE, keeps the join. * test(dst): pin one-row keyed write costs at two table sizes KeyedInsert, KeyedUpdate, MergeLoadPresent and MergeLoadNew on an indexed table of 2 and 16 fragments. On main their l.get rows were 10/25, 13/41, 10/24 and 10/24; now 9/10, 11/11, 9/9 and 5/5. InsertLegacy l.get moves 15 -> 13: its keyed insert takes the absence-proven writer. * perf(storage): bound the upsert probe and the known-present switch by index coverage The exact-id probe plans its IN list once per fragment the id BTREE does not cover, so it runs only while those copies hold no more ids than one keyed batch (ModernRelay#1225). An all-present upsert's indexed merge reads each uncovered fragment once more, where the join reads every fragment once, so it switches only while the BTREE covers more fragments than it leaves. key_column_uncovered_fragments counts the tail under the coverage rule key_column_coverage already applies; both read one helper. * test(dst): MergeLoadNewFragments2 reads 7 after the coverage-bounded switch * docs(writes): the upsert probe and the keyed update conjunct * test(write_cost): a keyed insert on the indexed standard graph stages the fenced insert * docs(changelog): shorten the keyed-writes note * release(notes): select new notes for v0.15.1 now v0.15.0 is tagged * test(dst): re-measure keyed write costs after merging main Main's ModernRelay#1291 keeps the dataset a write commits in the table handle cache, so a write that follows another on the same handle opens no manifest. The keyed-write scenario runs insert, update, merge-present and merge-new in turn on one reopened handle: the insert starts cold and is unchanged; the three later writes drop l.get (update 11 -> 8, merge present 9 -> 8, merge new 7/5 -> 4 at 2/16 fragments). * test(dst): say why the keyed-write rows now match at both sizes * fix(writes): keep the id join when the uncovered tail's data files exceed a byte budget Lance 13's indexed merge route scans every fragment the id BTREE leaves uncovered for every source column, so the all-present switch's read grew with the row width. It now also requires the tail's recorded data-file bytes to be known and at most KNOWN_PRESENT_TAIL_MAX_BYTES (1 MiB); the fragment rule stays, since it bounds the tail's requests. * refactor(writes): one exact-id scan for both id probes; pin the probe's IN-list bound existing_id_count and first_existing_id built the same filter and scan; exact_id_scan now owns it. A staged test pins the ModernRelay#1225 bound at its edge: uncovered fragments x ids = 8,192 probes, one id more keeps the join. writes.md and the changelog note state the byte budget. * test(dst): build the keyed-write scenario's indexes with ensure_indices_as Main (ModernRelay#1275) replaced Session::ensure_indices with ensure_indices_as; the scenario this pull request adds still called the old name. * docs(changelog): keep the keyed-writes note within the 200-character limit writes.md carries the byte budget the note no longer states. * docs(changelog): say the keyed-write index route holds while the unindexed tail is small The note said a one-row keyed insert, update by key or merge load always reads through the id index. For a row that already exists, the indexed route is taken only while the uncovered tail has fewer fragments than the index covers and at most 1 MiB of data files; otherwise the join still reads the id column of every fragment. Reword it to state that condition within the 200-character limit.
The version-keyed dataset cache is read-allocate only, so a writer drops the Lance dataset for the version it committed and the next open re-reads the manifest (a HEAD and a GET, twice past one block); the OCC validation probe of the branch head runs inside the latch-held critical section, after the gate-free staging puts.
Write-allocate:
commit_allinserts the committed dataset under the key the publish registers (path, branch, version, detached version, e_tag), an immutable pin dropped by branch control or refresh. Validation hoisted: the probe is issued concurrently with the staging puts and reused under the gates while the installed coordinator's manifest incarnation still matches it; the__manifestcompare-and-swap still refuses a cross-process publish.write_invalidates_table_cache_for_changed_table4 reads → 0,measure_counts_schema_contract_requests_issue_8171 HEAD → 0;crates/omnigraph-dst/cost_table.txt: nine scenarios drop,Optimize+2 GETs (the held dataset's first index-section read).For review: the probe is trusted on the installed coordinator's manifest incarnation (version, branch id, e_tag or timestamp); wrong if a store returns neither e_tag nor timestamp and two images share a version. A writer that loses the head between probe and gates re-probes under them: two requests more.
loadkeeps its probe under the gates. Staging's.dataset()chokepoint count 1 → 2 (a clone, no write).docs/dev/writes.mdand the release note carry the contract; no flag, endpoint or format change.