From fc4822d26b437555cad22be076acb7aa331584ae Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Mon, 28 Sep 2026 15:31:37 +0200 Subject: [PATCH 1/7] rfc: RFC 0047, search plan truth on the planned engine --- docs/rfcs/0047-search-plan-truth.md | 455 ++++++++++++++++++++++++++++ docs/rfcs/README.md | 1 + 2 files changed, 456 insertions(+) create mode 100644 docs/rfcs/0047-search-plan-truth.md diff --git a/docs/rfcs/0047-search-plan-truth.md b/docs/rfcs/0047-search-plan-truth.md new file mode 100644 index 000000000..6eca2b75e --- /dev/null +++ b/docs/rfcs/0047-search-plan-truth.md @@ -0,0 +1,455 @@ +--- +rfc: "0047" +title: "Search plan truth: loud search failures and one total order" +track: public +status: draft +implementation: not-started +authors: + - Ragnor Comerford (@ragnorc) +created: 2026-09-01 +updated: 2026-09-28 +discussion: null +supersedes: [] +superseded_by: [] +blocked_on: [] +--- + +# RFC 0047: Search plan truth: loud search failures and one total order + +Every code reference is at `main` `b14c22c5`. Behaviour marked as observed was +reproduced there, or at `80406fee` where noted, with a logic-test probe on +engine v2 or through the CLI. + +## Summary + +This is the correctness slice of search. It removes every search shape that +returns a silent wrong answer, gives every search order one total order, and +makes every refused query describe itself in the same four fields. It lands on +engine v2, the planned read route, and on the compiler both engines share; +engine v1 stays frozen and refuses the shapes it would answer wrongly. + +1. **Diagnostics.** Every refused query carries a stable code, where the + failure is, what was expected, and one fix. This covers parse and type + refusals, planner refusals and the engine v1 door, and a refusal by design + is a bad request on every path. +2. **Full-text search on an unindexed property.** A property the schema does + not declare `@index` is refused at compile time (`T27`); a declared index + that is not built is refused at planning (`FullTextIndexRequired`). Neither + falls back to Lance's case-sensitive flat scan. +3. **Ranking a traversal-introduced binding.** Declaration order stops + mattering: the ranked binding roots its component, so a query ranks the + binding it names whichever binding it declared first. +4. **One total order.** Every search order sorts rows by score, then the + query's remaining order keys, then binding identity, and `limit` cuts rows. + The fused `rrf()` score becomes a projectable column. A search-ordered + aggregate orders its groups by the remaining keys. +5. **Read descriptors.** A read reports which retrievals ran and what each + projected rank column means, derived from the plan that ran. +6. **The served read floor** is attributed to its phases and reported. + +Boundaries that do not change: no storage format change, no change to BM25 or +vector scoring, the deprecated `POST /read` envelope stays byte-stable, engine +v1's executor keeps its bytes, and GQ gains no syntax. + +Related decisions: [Shared expression model](2026-09-24-shared-expression-model.md) +owns the query surface this RFC works within; RFC 0048 +([Search contracts and retrieval algebra, PR #PR0048](https://github.com/ModernRelay/omnigraph/pull/PR0048)) +owns the retrieval contract beyond this slice; the analyzed lexical search RFC +([PR #PRLEX](https://github.com/ModernRelay/omnigraph/pull/PRLEX)) owns analyzers +and exact lexical matching; [Self-contained server testing with GQT and DST](2026-09-26-self-contained-server-testing.md) +owns the read envelope's result column types, which item 5 extends beside. + +## Motivation + +Each item below was reproduced; the issue carries the reproduction. + +- **Refusals an agent cannot repair.** An in-context measurement (a general + agent writing GQ from a schema and a one-page card) ended 45 of 45 tasks + correct, but 8 of 45 first attempts were refused, all for one shape: `query + name {` without the empty parameter list. The parser answered `parse error + --> 1:1 … expected query_file` inside terminal colour codes and a backtrace + hint, naming neither the missing `(` nor the fix. +- **A refusal by design reported as a crash.** On engine v2 the planner + refuses a query shape as the caller's error (`PlanError::Unsupported`), but + the ordinary path reports it as an internal error, HTTP 500, wrapped in JSON + (#786). +- **Confident false negatives from full-text search.** On both engines, + `search()` on a property without an index runs Lance's flat scan with a bare, + case-sensitive tokenizer: `"deep"` matched only "deep dive" and `"Deep"` + only "Deep Learning" (#747). A real report of an absent entity came from + this. +- **Ranking that depends on declaration order.** Ranking a binding reached by + a traversal fails when that binding is declared after the one it is reached + from: engine v1 fails with `search-ordered query produced rows without its + 'd._score' ranking column`, engine v2 refuses it (#789). It was reported twice + from real graph work, once for passages scoped by a matter and once for the + neighbours of one key-selected node. +- **Orders that ignore the query.** `rrf()` drops the order keys written after + it (#787), and a search-ordered aggregate applies no order, so `limit` keeps + an arbitrary set of groups (#788). The logic-test runner already refuses + ordered expectations for both shapes (`ordered_refusal` in + `crates/omnigraph-gqt/src/lib.rs`). +- **Reads that do not describe themselves.** A read carries rows and a graph + commit, but not which retrieval produced its order, whether that retrieval + is exact or approximate, or what a projected score column measures. + +Two fixes already landed on engine v2 and are not repeated here: the planner +states the retrieval once as typed plan nodes (`optimizer.rs` `search_node`), +and `search()` on a traversal destination filters that destination (#750, +fixed by #760). + +## User and operational behavior + +### Diagnostics + +Every refusal of a query carries four things: a stable code; where the failure +is, as a line and column for a parse refusal or as the stage and expression +for a later one; what was expected or violated; and one concrete fix that names +the construct to use. A refusal with no fix names the decision instead. The +reader is often an agent that treats an error as the documentation it acts on, +so one repair turn is the norm and a blind retry the exception. The contract +does not recognize other languages' idioms. + +```text +error[Q002]: parse error: expected `(`: a query declares its parameters even when it has none + --> line 1, column 11 + fix: query name() +``` + +Codes are grouped by who refuses: `Q…` the parser, `T…` the type checker, +`P…` the planner, `V…` the engine v1 door. A code's meaning is frozen once +published; its message may improve. The numbers are assigned in the one +catalogue by the change that adds each refusal. + +- **HTTP.** A refused query is a `400` whose error body carries an additive + `diagnostic` object (`code`, `position` or `stage` and `expression`, + `expected`, `fix`). A refusal by design is never a `500`; a planner defect + still is. +- **CLI.** The human formats print the form above on stderr with no colour + codes and no backtrace. `--json`, `--format json` and `--format jsonl` + print the API's error body on stdout, pretty or as one line. +- **Stored queries.** `queries validate --json` and `cluster plan --json` + carry the same object beside their messages. +- **Engine v1 door.** A refusal of a query v1 cannot answer correctly keeps + the existing message, which names the construct and both switches (`set + engine = v2;`, `OMNIGRAPH_ENGINE=v2`); its `fix` names the one that applies: + the process setting for a stored query, which carries no settings, and the + query prefix otherwise. + +### Full-text search needs its index + +- `T27`: a full-text predicate or ranking (`search`, `fuzzy`, `match_text`, + `bm25`) on a String property that the schema does not declare `@index`. The + fix is to declare `@index` and build it (`omnigraph optimize`). The check + reads only the schema, never physical index state. +- `FullTextIndexRequired`: the property is declared but its index has no + built segments at the query's snapshot. It is refused at planning, before + any scan, as HTTP `409` with a typed detail, the shape RFC 0043's + `FullTextIndexRebuildRequired` already has; both can fire on one graph. + Rows written after the last build keep today's behavior: Lance scans them + with the index's analyzer. + +The analyzer-equivalent exact scan that later lets an unbuilt index answer +instead of refusing belongs to the analyzed lexical search RFC. + +### Ranking a traversal-introduced binding + +A search order may rank any binding of the pattern, whichever binding the +`match` block declares first: + +```gq +query passages($matter: String, $q: String) { + match { + $r: SourceRevision { matter_number: $matter } + $p: Passage + $p passageOfRevision $r + } + return { $r.path, $p.locator } + order { bm25($p.text, $q) } + limit 5 +} +``` + +The ranked binding is where the component's scan starts. When the other end +of the component is far more selective (one node selected by its key), engine +v2 may instead start there and rank only the rows the traversal reaches; the +answer is the same, only the cost differs. One shape stays refused on both +engines as a type error: `rrf()` arms that rank two different bindings of one +traversal-connected component, since one scan cannot start at both. + +### One total order + +Every search order is total and cuts rows: + +| Order | Sort | Cut | +|---|---|---| +| `nearest(…)` | distance ascending, then the remaining keys, then every binding's id | `limit` rows | +| `bm25(…)` | score descending, then the remaining keys, then every binding's id | `limit` rows | +| `rrf(…)` | fused score descending, then the remaining keys, then every binding's id | `limit` rows | + +`nearest` and `bm25` already behave this way on engine v2. For `rrf`, the +change is visible only where keys follow `rrf()` or where one entity has +several rows; today the top `limit` entities are chosen first and their rows +cut again. + +`return { $d.slug, rrf(bm25($d.title, $q), nearest($d.embedding, $v)) as score }` +projects the fused score the order used, under the rule `nearest` and `bm25` +already follow (`T33`): the projected expression repeats the leading order +key. The refusal of a projected `rrf()` (`T37`) is retired. + +In an aggregate query, the leading search function selects the population the +aggregate reads (the matches, or the `nearest` window) and does not order the +groups; the remaining order keys order the groups, bound against `return` as +the shared expression model specifies, and the group keys break remaining +ties, so `limit` is deterministic. With no key after the search function the +groups are unordered, as today, and a test cannot expect an order. + +### Read descriptors + +The canonical read envelope (`POST /query`, stored-query invocation, CLI +`--json` and the `jsonl` metadata record) gains two additive arrays: + +```json +"retrievals": [{ "binding": "d", "property": "embedding", "kind": "nearest", + "recall": "approximate", + "embedding_coverage": { "state": "unknown", "reason": "not_computed" } }], +"metrics": [{ "column": "score", "kind": "distance", "source": "nearest", + "binding": "d", "property": "embedding", "descending": false, + "recall": "approximate" }] +``` + +- `recall` reports the source's contract, not what one execution did: a + `nearest` is `approximate` even when a run happened to scan exactly, so a + client never relies on a guarantee that disappears when an index is built. +- `embedding_coverage` is `known` with counts only when the run already + established them, and `unknown` otherwise. Exact counts on request belong to + RFC 0048's result metadata contract. +- The deprecated `POST /read` envelope carries neither array. + +The extension is one additive change to the envelope, made together with the +result column types that the self-contained server testing RFC needs. + +### Served read floor + +A served read reports its time per phase (authentication, target resolution, +compilation, planning, execution, serialization) in an additive `usage` +object, so a slow read is attributed without instrumentation (#752). + +### Operators + +`T27` refuses queries the compiler accepts today, and stored queries are +recompiled when a server starts, where a failure quarantines the graph. Run +`omnigraph queries validate` and `omnigraph cluster plan` before upgrading; a +refused stored query is a pre-upgrade finding, never a boot failure. The v1 +door refusals appear at invocation, not at boot; the release note lists them. + +## Design + +### Where each item lands + +| Item | Compiler (both engines) | Engine v2 (planner and operators) | Engine v1 door (`exec/query_doors.rs`) | +|---|---|---|---| +| Diagnostics | `QueryDiagnostic`, one code catalogue | `PlanError::Unsupported` carries a diagnostic; the ordinary path maps it to a bad request | `V1Refusal` carries a diagnostic | +| Full-text index | `T27` | index presence as a planning fact | refuses an unbuilt index with the same `FullTextIndexRequired` | +| Ranking a destination | lowering roots the component at the ranked binding; type error for `rrf` arms on two bindings of one component | optional reversal when the other end is selective | none needed: the lowering fixes v1 too | +| Total order | `T33` admits a projected `rrf()`; `T37` retired | fused score column, `Sort` over the fusion and over a search-ordered aggregate | refuses keys after `rrf()` and keys after a search order in an aggregate | +| Descriptors | — | derived from the physical plan's ranked scans and fusion | none; v1 reads carry no descriptors | +| Read floor | — | — | — (server instrumentation) | + +Nothing changes a byte of engine v1's frozen files. The v1 door is the one +place v1 learns a new rule, following the rule the shared expression model +set: when v1 cannot run a query and v2 can, the error says so; when neither +can, the ordinary error stands alone. + +### Diagnostics + +`QueryDiagnostic { kind, code, message, position, stage, fix }` lives in the +compiler with its code catalogue. `CompilerError::Query` carries it; its +display is the legacy one-line form (`parse error: …`, `type error: T33: …`), +so existing assertions and logic-test error needles keep holding. The planner +builds the same type for its refusals, with `stage: plan` and the expression +it refused; `plan_source::plan_query` maps `Unrouted::UnsupportedQuery` to +that diagnostic instead of `no_plan`, which stays for genuine defects. The +server's `ErrorOutput` gains the optional `diagnostic` detail, so every +existing error body is byte-identical. A declaration without its parameter +list needs a grammar recognizer (`missing_param_list`), because the parser +records attempts per rule, never per token, so the missing `(` is not an +attempt it can name. + +### Full-text index presence as a planning fact + +`PlanSource` gains one fact: whether a property's full-text index has built +segments at the pinned dataset version. The planner reads it while resolving +a ranked scan or a search predicate and refuses when it is absent. The read +goes through the recording wrapper, so the fact is part of the plan's +`Assumptions` and a replay against another snapshot is refused, as for every +other planner input. Index coverage of newer rows is not consulted: an +uncovered tail is not a refusal. + +The `T27` rule is the catalog predicate index reconciliation already uses (a +non-enum, single-column String `@index`), moved into the compiler so the type +checker and reconciliation cannot drift. + +### The ranked binding roots its component + +`scan_root` in `crates/omnigraph-compiler/src/ir/lower.rs` picks each +component's scan: today the first-declared binding, or a searched binding +inside a correlated block. It gains one rule: at the top level, a component +that holds the binding the leading order key ranks (both arms' binding, for an +`rrf()` over one binding) roots there. The traversal is lowered from that +root; the lowering already expands in either direction. Both engines rank a +root scan, so no engine change is needed for correctness. Engine v2's +`nearest_prefilter_gate` applies as it does to any ranked root with +traversals leaving it. + +A key-selected other end makes the ranked root expensive: the ranked scan +starts from the whole type, and the `nearest` overfetch ladder can end in an +exact pass over it. The later performance step lets the v2 planner reverse a +ranked component when its cost model estimates the other end as selective, +running the traversal first and ranking the reached rows with their ids as +the scan's prefilter; the planner's current refusal of a ranked dependent scan +(`optimizer.rs` `rank`) becomes that plan. + +### Total order + +The planner plans a `Sort` over a `RankFuse` exactly as over a ranked scan: +`sort_keys` stops returning `None` for a fusion. `RankFuseExec` stops +truncating to the limit and emits the fused score as a `Float64` column +`._rrf`; the `Sort` orders by it, then the remaining keys, then the +declared tie-break ids, and `Limit` cuts rows. The `Sort` with a `fetch` is +already a streaming top-k, so memory stays bounded by `fetch` rows plus one +batch above the fusion's own output. Winner selection inside the fusion no +longer decides the cut, so fused ties need no plateau rule. + +For an aggregate under a search order, `sort_keys` returns the remaining keys +instead of `None`, bound against `return`, with the group keys as the declared +tie-break. The runner's `ordered_refusal` drops its `rrf` rule and admits an +aggregate order whose keys make it total. + +### Read descriptors + +Each ranked `Scan` (`RankedAccess`) and each `RankFuse` of the physical plan +yields one retrieval; each projected column that reads `_distance`, `_score` +or `_rrf` yields one metric. Both are computed from the plan that ran, so they +cannot disagree with the order the rows have. `recall` is a function of the +retrieval kind: `nearest` is approximate, `bm25` exact, `rrf` approximate when +either arm is. + +## Invariants + +- **Integrity failures are loud (8).** Strengthened: every silent wrong answer + this RFC names becomes a correct answer or a typed refusal on both engines. +- **Query semantics are typed structures (9).** Strengthened: refusals carry + a typed diagnostic instead of a string with a code prefix; the root choice is + a lowering rule, not a textual accident. +- **Physical acceleration is derived (7).** `FullTextIndexRequired` refuses; + it never returns a different answer. An index's coverage of newer rows is + never a refusal. The refusal follows RFC 0043's accepted fence shape and is + lifted by the lexical RFC's exact scan. +- **Bounded, observable resource use (11).** The fusion's `Sort` is a + streaming top-k; descriptors add no scan; the read floor is reported. +- **One source of truth (12).** Descriptors derive from the plan; no + retrieval description is kept beside it. The index fact is a recorded plan + input. +- Deny-list: no side channel for discarded rank (the fused score is a + column), no string-built predicates, no logical precondition on index + coverage. + +## Compatibility and reversibility + +- **Wire:** `diagnostic`, `retrievals`, `metrics` and `usage` are additive and + omitted when absent. `POST /read` is untouched. OpenAPI regenerates. +- **Language:** `T27` and the `rrf()` arms type error refuse queries the + compiler accepts today; both previously produced wrong or failing results. + The v1 door refuses four shapes v1 answered wrongly; each refusal names the + switch to engine v2. +- **Order:** results change only where an order was not total: ties inside a + fused score, keys after `rrf()`, several rows per entity under `rrf()`, and + search-ordered aggregates. +- **Reverting** needs no storage work: every field is additive, the root + rule is a lowering choice, and the refusals can be relaxed, at the cost of + the silent wrong answers they remove. + +## Alternatives + +- **Refuse a ranked traversal destination at compile time** (this RFC's first + draft, `T26`). Rejected: declaration order is not semantics, the fix it + offered ("declare it first") is a workaround the language can apply itself, + and engine v2 already filters a destination correctly. +- **Choose the top `limit` entities of a fusion, then cut rows.** Rejected: a + second cut rule beside the one every other search order uses, and it needs a + tie-plateau budget that the row cut does not. +- **Warn instead of ordering a search-ordered aggregate.** Rejected: the keys + the query wrote can be applied, so a warning would describe a defect instead + of removing it. +- **Record the retrieval in the `QueryIR`.** Rejected: engine v2's planner + already states it once as typed plan nodes; a second record would be a + second source of truth. +- **Fix engine v1's executor.** Rejected: v1 is frozen, and its defects are + fixed on v2; the v1 door refuses what v1 would answer wrongly. +- **Leave v1's silent wrong answers until v2 is the default.** Rejected: v1 + stays the default for at least one release, and the door check costs one + walk of the compiled query. + +## Evidence and tests + +Each change lands with its regression at the owner the +[testing map](../dev/testing.md) names: + +- compiler parser, type-check and lowering tests for the diagnostics, `T27`, + the `rrf()` arms error and the root rule; +- `crates/omnigraph-gqt/cases/v2/` cases for every behavior visible in rows or + errors, `--- expect plan` where the plan is the claim (the ranked root, the + fusion `Sort`, the aggregate `Sort`), named for the issue each closes; +- `crates/omnigraph-planner/tests/` for the planning fact and the diagnostic + of a refusal; +- server `data_routes` and `openapi`, CLI `cli_queries` and `parity_matrix` + for the error body and the envelope. + +The frozen `tests/search.rs` cannot grow, so no retrieval golden lives there. +The four probe results in the motivation are reproductions, not tests; each +issue restates its shape for the case that will own it. + +## Rollout + +Each step is shippable alone, lands on engine v2 and the shared compiler, +and closes its issues. + +| Step | Delivers | Closes | +|---|---|---| +| 1 | Diagnostics contract for parse and type refusals (PR #759) | — | +| 2 | Planner and v1 door refusals carry diagnostics; a refusal by design is a bad request | #786 | +| 3 | `T27` and `FullTextIndexRequired`; the v1 door refuses an unbuilt index | #747 | +| 4 | The ranked binding roots its component; the `rrf()` arms type error | #789 | +| 5 | One total order: fused score column, `Sort` over fusion and search-ordered aggregates, `T37` retired; the v1 door refuses both shapes | #787, #788 | +| 6 | Read descriptors | — | +| 7 | Served read floor | #752 | +| 8 | Engine v2 reverses a ranked component whose other end is selective | — (performance) | + +Steps 2 and 3 to 5 extend the v1 door beyond the new-feature rule it has +today; that extension and the refusal-as-bad-request rule need the engine +owner's agreement before this RFC is accepted. `implementation` becomes +`in-progress` when step 1 lands and `complete` when step 7 does; step 8 is a +performance follow-up. + +## Unresolved questions + +1. Does the engine owner agree that the v1 door refuses known v1 defects + (steps 3 to 5), not only features v1 lacks, and that planner refusals by + design become bad requests carrying diagnostics (step 2)? + +## Decision log + +- 2026-09-01 — first draft opened in PR #606, written against engine v1. +- 2026-09-19 — accepted as drafted in PR #606. +- 2026-09-28 — rewritten against engine v2 (`main` `b14c22c5`) and returned + to draft pending the unresolved question. Removed, because engine v2 already + does it or the freeze forbids it: the `QueryIR` retrieval field, the + compile-time refusal of a ranked or searched traversal destination (`T26`), + standalone retrieval goldens in `tests/search.rs`, the `explain` route, and + a warnings channel threaded through the v1 executor. Changed: a ranked + binding roots its component, `rrf()` cuts rows, a search-ordered aggregate + applies its remaining keys, descriptors derive from the plan, the + full-text refusal is a planning fact, and the v1 door refuses the shapes v1 + answers wrongly. The drafts in PR #606 are superseded as a whole; this file + replaces them. diff --git a/docs/rfcs/README.md b/docs/rfcs/README.md index f406881ab..e7e493531 100644 --- a/docs/rfcs/README.md +++ b/docs/rfcs/README.md @@ -198,6 +198,7 @@ then dated RFCs by date. | [0044](0044-edge-keys.md) | Edge keys: derived edge identity | maintainer | draft | in-progress | | [0045](0045-gq-logic-tests.md) | GQ logic tests | maintainer | draft | partial | | [0046](0046-index-status.md) | Read-only index status | maintainer | draft | not-started | +| [0047](0047-search-plan-truth.md) | Search plan truth: loud search failures and one total order | public | draft | not-started | | [0049](0049-control-plane-seams.md) | Control-plane seams: observe, readiness witness, bounded shutdown | maintainer | accepted | partial | | [0051](0051-json-output-via-arrow.md) | JSON output via Arrow | maintainer | draft | partial | | [0052](0052-managed-control-plane-cli.md) | Managed control-plane CLI | maintainer | accepted | complete | From 13ebfa5a85c4f0ebcb1e4873a76d4f11ddd1ef37 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Mon, 28 Sep 2026 15:46:09 +0200 Subject: [PATCH 2/7] rfc: link RFC 0047 to its discussion and sibling PRs --- docs/rfcs/0047-search-plan-truth.md | 6 +++--- docs/rfcs/README.md | 3 ++- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/rfcs/0047-search-plan-truth.md b/docs/rfcs/0047-search-plan-truth.md index 6eca2b75e..d0bdf6cca 100644 --- a/docs/rfcs/0047-search-plan-truth.md +++ b/docs/rfcs/0047-search-plan-truth.md @@ -8,7 +8,7 @@ authors: - Ragnor Comerford (@ragnorc) created: 2026-09-01 updated: 2026-09-28 -discussion: null +discussion: "https://github.com/ModernRelay/omnigraph/pull/791" supersedes: [] superseded_by: [] blocked_on: [] @@ -53,9 +53,9 @@ v1's executor keeps its bytes, and GQ gains no syntax. Related decisions: [Shared expression model](2026-09-24-shared-expression-model.md) owns the query surface this RFC works within; RFC 0048 -([Search contracts and retrieval algebra, PR #PR0048](https://github.com/ModernRelay/omnigraph/pull/PR0048)) +([Search contracts and retrieval algebra, PR #793](https://github.com/ModernRelay/omnigraph/pull/793)) owns the retrieval contract beyond this slice; the analyzed lexical search RFC -([PR #PRLEX](https://github.com/ModernRelay/omnigraph/pull/PRLEX)) owns analyzers +([PR #792](https://github.com/ModernRelay/omnigraph/pull/792)) owns analyzers and exact lexical matching; [Self-contained server testing with GQT and DST](2026-09-26-self-contained-server-testing.md) owns the read envelope's result column types, which item 5 extends beside. diff --git a/docs/rfcs/README.md b/docs/rfcs/README.md index e7e493531..9d1e71ed6 100644 --- a/docs/rfcs/README.md +++ b/docs/rfcs/README.md @@ -48,7 +48,8 @@ RFCs 0001 to 0068 use `NNNN-kebab-title.md`, the heading closed at 0068: no new number is allocated, and `scripts/check-docs.py` rejects any numbered filename outside the allocated and reserved numbers it lists. Numbers reserved by PRs that were open -when the namespace closed (0047 and 0048 by PR #606; 0050 by the +when the namespace closed (0047 and 0048 by PR #606, carried on by PR #791 +and PR #793; 0050 by the `rfc/0050-engine-crate-topology` branch; 0056 by PR #670; 0059 by PR #675; 0060 by PR #677; 0067 and 0068 by PR #725) may still land under their reserved numbers. Every other gap From aa356603fe0e806e2b9abfef0d95823d3d1ca490 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Tue, 29 Sep 2026 23:31:57 +0200 Subject: [PATCH 3/7] rfc: RFC 0047 on engine v2 as the only engine --- docs/rfcs/0047-search-plan-truth.md | 100 ++++++++++++---------------- 1 file changed, 44 insertions(+), 56 deletions(-) diff --git a/docs/rfcs/0047-search-plan-truth.md b/docs/rfcs/0047-search-plan-truth.md index d0bdf6cca..b22df3918 100644 --- a/docs/rfcs/0047-search-plan-truth.md +++ b/docs/rfcs/0047-search-plan-truth.md @@ -7,7 +7,7 @@ implementation: not-started authors: - Ragnor Comerford (@ragnorc) created: 2026-09-01 -updated: 2026-09-28 +updated: 2026-09-29 discussion: "https://github.com/ModernRelay/omnigraph/pull/791" supersedes: [] superseded_by: [] @@ -16,22 +16,20 @@ blocked_on: [] # RFC 0047: Search plan truth: loud search failures and one total order -Every code reference is at `main` `b14c22c5`. Behaviour marked as observed was -reproduced there, or at `80406fee` where noted, with a logic-test probe on -engine v2 or through the CLI. +Every code reference is at `main` `baf10c94`. Behaviour marked as observed was +reproduced at `b14c22c5` with a logic-test probe on engine v2 or through the +CLI, and the code it rests on is unchanged at `baf10c94`. ## Summary This is the correctness slice of search. It removes every search shape that returns a silent wrong answer, gives every search order one total order, and makes every refused query describe itself in the same four fields. It lands on -engine v2, the planned read route, and on the compiler both engines share; -engine v1 stays frozen and refuses the shapes it would answer wrongly. +engine v2, the only query engine since v0.12.0 (PR #795), and on the compiler. 1. **Diagnostics.** Every refused query carries a stable code, where the - failure is, what was expected, and one fix. This covers parse and type - refusals, planner refusals and the engine v1 door, and a refusal by design - is a bad request on every path. + failure is, what was expected, and one fix. This covers parse, type and + planner refusals, and a refusal by design is a bad request on every path. 2. **Full-text search on an unindexed property.** A property the schema does not declare `@index` is refused at compile time (`T27`); a declared index that is not built is refused at planning (`FullTextIndexRequired`). Neither @@ -48,8 +46,8 @@ engine v1 stays frozen and refuses the shapes it would answer wrongly. 6. **The served read floor** is attributed to its phases and reported. Boundaries that do not change: no storage format change, no change to BM25 or -vector scoring, the deprecated `POST /read` envelope stays byte-stable, engine -v1's executor keeps its bytes, and GQ gains no syntax. +vector scoring, the deprecated `POST /read` envelope stays byte-stable, the +frozen reference engine keeps its bytes, and GQ gains no syntax. Related decisions: [Shared expression model](2026-09-24-shared-expression-model.md) owns the query surface this RFC works within; RFC 0048 @@ -73,15 +71,16 @@ Each item below was reproduced; the issue carries the reproduction. refuses a query shape as the caller's error (`PlanError::Unsupported`), but the ordinary path reports it as an internal error, HTTP 500, wrapped in JSON (#786). -- **Confident false negatives from full-text search.** On both engines, +- **Confident false negatives from full-text search.** On engine v2, `search()` on a property without an index runs Lance's flat scan with a bare, case-sensitive tokenizer: `"deep"` matched only "deep dive" and `"Deep"` only "Deep Learning" (#747). A real report of an absent entity came from this. - **Ranking that depends on declaration order.** Ranking a binding reached by a traversal fails when that binding is declared after the one it is reached - from: engine v1 fails with `search-ordered query produced rows without its - 'd._score' ranking column`, engine v2 refuses it (#789). It was reported twice + from: engine v2 refuses it (#789), and engine v1 failed with + `search-ordered query produced rows without its 'd._score' ranking column` + before v2 replaced it. It was reported twice from real graph work, once for passages scoped by a matter and once for the neighbours of one key-selected node. - **Orders that ignore the query.** `rrf()` drops the order keys written after @@ -117,7 +116,7 @@ error[Q002]: parse error: expected `(`: a query declares its parameters even whe ``` Codes are grouped by who refuses: `Q…` the parser, `T…` the type checker, -`P…` the planner, `V…` the engine v1 door. A code's meaning is frozen once +`P…` the planner. A code's meaning is frozen once published; its message may improve. The numbers are assigned in the one catalogue by the change that adds each refusal. @@ -130,11 +129,6 @@ catalogue by the change that adds each refusal. print the API's error body on stdout, pretty or as one line. - **Stored queries.** `queries validate --json` and `cluster plan --json` carry the same object beside their messages. -- **Engine v1 door.** A refusal of a query v1 cannot answer correctly keeps - the existing message, which names the construct and both switches (`set - engine = v2;`, `OMNIGRAPH_ENGINE=v2`); its `fix` names the one that applies: - the process setting for a stored query, which carries no settings, and the - query prefix otherwise. ### Full-text search needs its index @@ -173,8 +167,8 @@ query passages($matter: String, $q: String) { The ranked binding is where the component's scan starts. When the other end of the component is far more selective (one node selected by its key), engine v2 may instead start there and rank only the rows the traversal reaches; the -answer is the same, only the cost differs. One shape stays refused on both -engines as a type error: `rrf()` arms that rank two different bindings of one +answer is the same, only the cost differs. One shape stays refused, as a +type error: `rrf()` arms that rank two different bindings of one traversal-connected component, since one scan cannot start at both. ### One total order @@ -240,26 +234,25 @@ object, so a slow read is attributed without instrumentation (#752). `T27` refuses queries the compiler accepts today, and stored queries are recompiled when a server starts, where a failure quarantines the graph. Run `omnigraph queries validate` and `omnigraph cluster plan` before upgrading; a -refused stored query is a pre-upgrade finding, never a boot failure. The v1 -door refusals appear at invocation, not at boot; the release note lists them. +refused stored query is a pre-upgrade finding, never a boot failure. ## Design ### Where each item lands -| Item | Compiler (both engines) | Engine v2 (planner and operators) | Engine v1 door (`exec/query_doors.rs`) | -|---|---|---|---| -| Diagnostics | `QueryDiagnostic`, one code catalogue | `PlanError::Unsupported` carries a diagnostic; the ordinary path maps it to a bad request | `V1Refusal` carries a diagnostic | -| Full-text index | `T27` | index presence as a planning fact | refuses an unbuilt index with the same `FullTextIndexRequired` | -| Ranking a destination | lowering roots the component at the ranked binding; type error for `rrf` arms on two bindings of one component | optional reversal when the other end is selective | none needed: the lowering fixes v1 too | -| Total order | `T33` admits a projected `rrf()`; `T37` retired | fused score column, `Sort` over the fusion and over a search-ordered aggregate | refuses keys after `rrf()` and keys after a search order in an aggregate | -| Descriptors | — | derived from the physical plan's ranked scans and fusion | none; v1 reads carry no descriptors | -| Read floor | — | — | — (server instrumentation) | - -Nothing changes a byte of engine v1's frozen files. The v1 door is the one -place v1 learns a new rule, following the rule the shared expression model -set: when v1 cannot run a query and v2 can, the error says so; when neither -can, the ordinary error stands alone. +| Item | Compiler | Engine v2 (planner and operators) | +|---|---|---| +| Diagnostics | `QueryDiagnostic`, one code catalogue | `PlanError::Unsupported` carries a diagnostic; the ordinary path maps it to a bad request | +| Full-text index | `T27` | index presence as a planning fact | +| Ranking a destination | lowering roots the component at the ranked binding; type error for `rrf` arms on two bindings of one component | optional reversal when the other end is selective | +| Total order | `T33` admits a projected `rrf()`; `T37` retired | fused score column, `Sort` over the fusion and over a search-ordered aggregate | +| Descriptors | — | derived from the physical plan's ranked scans and fusion | +| Read floor | — | — (server instrumentation) | + +Engine v1 is the frozen reference engine (`crates/omnigraph-reference-engine`), +reached only through a logic-test step's `--- expect same as v1`; this RFC +changes none of its bytes. A case for a shape the reference answers wrongly +does not use that comparison. ### Diagnostics @@ -297,7 +290,7 @@ component's scan: today the first-declared binding, or a searched binding inside a correlated block. It gains one rule: at the top level, a component that holds the binding the leading order key ranks (both arms' binding, for an `rrf()` over one binding) roots there. The traversal is lowered from that -root; the lowering already expands in either direction. Both engines rank a +root; the lowering already expands in either direction. Engine v2 ranks a root scan, so no engine change is needed for correctness. Engine v2's `nearest_prefilter_gate` applies as it does to any ranked root with traversals leaving it. @@ -338,7 +331,7 @@ either arm is. ## Invariants - **Integrity failures are loud (8).** Strengthened: every silent wrong answer - this RFC names becomes a correct answer or a typed refusal on both engines. + this RFC names becomes a correct answer or a typed refusal. - **Query semantics are typed structures (9).** Strengthened: refusals carry a typed diagnostic instead of a string with a code prefix; the root choice is a lowering rule, not a textual accident. @@ -361,8 +354,6 @@ either arm is. omitted when absent. `POST /read` is untouched. OpenAPI regenerates. - **Language:** `T27` and the `rrf()` arms type error refuse queries the compiler accepts today; both previously produced wrong or failing results. - The v1 door refuses four shapes v1 answered wrongly; each refusal names the - switch to engine v2. - **Order:** results change only where an order was not total: ties inside a fused score, keys after `rrf()`, several rows per entity under `rrf()`, and search-ordered aggregates. @@ -385,11 +376,6 @@ either arm is. - **Record the retrieval in the `QueryIR`.** Rejected: engine v2's planner already states it once as typed plan nodes; a second record would be a second source of truth. -- **Fix engine v1's executor.** Rejected: v1 is frozen, and its defects are - fixed on v2; the v1 door refuses what v1 would answer wrongly. -- **Leave v1's silent wrong answers until v2 is the default.** Rejected: v1 - stays the default for at least one release, and the door check costs one - walk of the compiled query. ## Evidence and tests @@ -406,7 +392,6 @@ Each change lands with its regression at the owner the - server `data_routes` and `openapi`, CLI `cli_queries` and `parity_matrix` for the error body and the envelope. -The frozen `tests/search.rs` cannot grow, so no retrieval golden lives there. The four probe results in the motivation are reproductions, not tests; each issue restates its shape for the case that will own it. @@ -418,25 +403,21 @@ and closes its issues. | Step | Delivers | Closes | |---|---|---| | 1 | Diagnostics contract for parse and type refusals (PR #759) | — | -| 2 | Planner and v1 door refusals carry diagnostics; a refusal by design is a bad request | #786 | -| 3 | `T27` and `FullTextIndexRequired`; the v1 door refuses an unbuilt index | #747 | +| 2 | Planner refusals carry diagnostics; a refusal by design is a bad request | #786 | +| 3 | `T27` and `FullTextIndexRequired` | #747 | | 4 | The ranked binding roots its component; the `rrf()` arms type error | #789 | -| 5 | One total order: fused score column, `Sort` over fusion and search-ordered aggregates, `T37` retired; the v1 door refuses both shapes | #787, #788 | +| 5 | One total order: fused score column, `Sort` over fusion and search-ordered aggregates, `T37` retired | #787, #788 | | 6 | Read descriptors | — | | 7 | Served read floor | #752 | | 8 | Engine v2 reverses a ranked component whose other end is selective | — (performance) | -Steps 2 and 3 to 5 extend the v1 door beyond the new-feature rule it has -today; that extension and the refusal-as-bad-request rule need the engine -owner's agreement before this RFC is accepted. `implementation` becomes +`implementation` becomes `in-progress` when step 1 lands and `complete` when step 7 does; step 8 is a performance follow-up. ## Unresolved questions -1. Does the engine owner agree that the v1 door refuses known v1 defects - (steps 3 to 5), not only features v1 lacks, and that planner refusals by - design become bad requests carrying diagnostics (step 2)? +None. ## Decision log @@ -453,3 +434,10 @@ performance follow-up. full-text refusal is a planning fact, and the v1 door refuses the shapes v1 answers wrongly. The drafts in PR #606 are superseded as a whole; this file replaces them. +- 2026-09-29 — engine v2 became the only query engine in v0.12.0 (PR #795), + and engine v1 the frozen reference engine. Removed: the engine v1 door + refusals of steps 2, 3 and 5, the `V…` code group, the two alternatives + about engine v1, the note about the frozen `tests/search.rs`, and the + unresolved question, which asked the engine owner to agree to those + refusals. The planner-refusal rule of step 2 stays. Every defect this RFC + names was checked again in code at `baf10c94`. From 95469eae39120bd8fac6d0f4d111ffe8809aa508 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Wed, 30 Sep 2026 14:21:48 +0200 Subject: [PATCH 4/7] rfc: RFC 0047 step 1 shipped in PR #759 --- docs/rfcs/0047-search-plan-truth.md | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/docs/rfcs/0047-search-plan-truth.md b/docs/rfcs/0047-search-plan-truth.md index b22df3918..8b2a573a8 100644 --- a/docs/rfcs/0047-search-plan-truth.md +++ b/docs/rfcs/0047-search-plan-truth.md @@ -3,11 +3,11 @@ rfc: "0047" title: "Search plan truth: loud search failures and one total order" track: public status: draft -implementation: not-started +implementation: in-progress authors: - Ragnor Comerford (@ragnorc) created: 2026-09-01 -updated: 2026-09-29 +updated: 2026-09-30 discussion: "https://github.com/ModernRelay/omnigraph/pull/791" supersedes: [] superseded_by: [] @@ -122,7 +122,10 @@ catalogue by the change that adds each refusal. - **HTTP.** A refused query is a `400` whose error body carries an additive `diagnostic` object (`code`, `position` or `stage` and `expression`, - `expected`, `fix`). A refusal by design is never a `500`; a planner defect + `expected`, `fix`, and an optional `suggestion`: byte-range edits of the + original source with an `applicability` of `machine_applicable` or + `needs_review`, so an agent can apply a repair without parsing prose). A + refusal by design is never a `500`; a planner defect still is. - **CLI.** The human formats print the form above on stderr with no colour codes and no backtrace. `--json`, `--format json` and `--format jsonl` @@ -402,7 +405,7 @@ and closes its issues. | Step | Delivers | Closes | |---|---|---| -| 1 | Diagnostics contract for parse and type refusals (PR #759) | — | +| 1 | Diagnostics contract for parse and type refusals (PR #759, merged 2026-09-30) | — | | 2 | Planner refusals carry diagnostics; a refusal by design is a bad request | #786 | | 3 | `T27` and `FullTextIndexRequired` | #747 | | 4 | The ranked binding roots its component; the `rrf()` arms type error | #789 | @@ -441,3 +444,7 @@ None. unresolved question, which asked the engine owner to agree to those refusals. The planner-refusal rule of step 2 stays. Every defect this RFC names was checked again in code at `baf10c94`. +- 2026-09-30 — step 1 shipped in PR #759, and `implementation` moved to + `in-progress`. The maintainer's review added the optional `suggestion` to + the diagnostic: source edits a caller can apply mechanically, marked + machine-applicable only after the edited text parses. From 1e7c9ab02c4ee72b6a658b6b4e736206fbd4db86 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Wed, 30 Sep 2026 14:22:02 +0200 Subject: [PATCH 5/7] rfc: RFC 0047 registry row in progress --- docs/rfcs/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/rfcs/README.md b/docs/rfcs/README.md index 960199ec8..c774166ab 100644 --- a/docs/rfcs/README.md +++ b/docs/rfcs/README.md @@ -208,7 +208,7 @@ then dated RFCs by date. | [0044](0044-edge-keys.md) | Edge keys: derived edge identity | maintainer | draft | in-progress | | [0045](0045-gq-logic-tests.md) | GQ logic tests | maintainer | draft | partial | | [0046](0046-index-status.md) | Read-only index status | maintainer | draft | not-started | -| [0047](0047-search-plan-truth.md) | Search plan truth: loud search failures and one total order | public | draft | not-started | +| [0047](0047-search-plan-truth.md) | Search plan truth: loud search failures and one total order | public | draft | in-progress | | [0049](0049-control-plane-seams.md) | Control-plane seams: observe, readiness witness, bounded shutdown | maintainer | accepted | partial | | [0051](0051-json-output-via-arrow.md) | JSON output via Arrow | maintainer | draft | partial | | [0052](0052-managed-control-plane-cli.md) | Managed control-plane CLI | maintainer | accepted | complete | From fcc3bdbfc99a729c5ebe3b944b69e7b13b898727 Mon Sep 17 00:00:00 2001 From: azim afroozeh <13484327+azimafroozeh@users.noreply.github.com> Date: Wed, 30 Sep 2026 23:20:35 +0200 Subject: [PATCH 6/7] review --- docs/rfcs/0047-search-plan-truth.md | 503 ++++++++++++++++++++++++---- docs/rfcs/README.md | 2 +- 2 files changed, 447 insertions(+), 58 deletions(-) diff --git a/docs/rfcs/0047-search-plan-truth.md b/docs/rfcs/0047-search-plan-truth.md index 8b2a573a8..2c2045970 100644 --- a/docs/rfcs/0047-search-plan-truth.md +++ b/docs/rfcs/0047-search-plan-truth.md @@ -16,20 +16,21 @@ blocked_on: [] # RFC 0047: Search plan truth: loud search failures and one total order -Every code reference is at `main` `baf10c94`. Behaviour marked as observed was -reproduced at `b14c22c5` with a logic-test probe on engine v2 or through the -CLI, and the code it rests on is unchanged at `baf10c94`. +Existing-behavior code references are at `main` `baf10c94`. Behaviour marked +as observed was reproduced at `b14c22c5` with a logic-test probe on engine v2 +or through the CLI, and the code it rests on is unchanged at `baf10c94`. ## Summary -This is the correctness slice of search. It removes every search shape that -returns a silent wrong answer, gives every search order one total order, and -makes every refused query describe itself in the same four fields. It lands on +This is the correctness slice of search. It fixes the silent wrong answers +described below, gives every search order one total order within its retrieved +population, and makes every refused query describe itself in the same four fields. It lands on engine v2, the only query engine since v0.12.0 (PR #795), and on the compiler. 1. **Diagnostics.** Every refused query carries a stable code, where the failure is, what was expected, and one fix. This covers parse, type and - planner refusals, and a refusal by design is a bad request on every path. + planner refusals. Query-shape refusals are bad requests; unavailable index + prerequisites retain their distinct conflict response. 2. **Full-text search on an unindexed property.** A property the schema does not declare `@index` is refused at compile time (`T27`); a declared index that is not built is refused at planning (`FullTextIndexRequired`). Neither @@ -37,13 +38,17 @@ engine v2, the only query engine since v0.12.0 (PR #795), and on the compiler. 3. **Ranking a traversal-introduced binding.** Declaration order stops mattering: the ranked binding roots its component, so a query ranks the binding it names whichever binding it declared first. -4. **One total order.** Every search order sorts rows by score, then the +4. **One total order.** Every search order sorts its retrieved rows by score, then the query's remaining order keys, then binding identity, and `limit` cuts rows. The fused `rrf()` score becomes a projectable column. A search-ordered aggregate orders its groups by the remaining keys. 5. **Read descriptors.** A read reports which retrievals ran and what each projected rank column means, derived from the plan that ran. 6. **The served read floor** is attributed to its phases and reported. +7. **Plan validation.** One shared component checks the query's search + requirements across lowering and optimization. It checks the invariants + listed in this RFC for all its search shapes, with stronger rewrite + validation for a defined exact-search subset. Boundaries that do not change: no storage format change, no change to BM25 or vector scoring, the deprecated `POST /read` envelope stays byte-stable, the @@ -97,6 +102,12 @@ states the retrieval once as typed plan nodes (`optimizer.rs` `search_node`), and `search()` on a traversal destination filters that destination (#750, fixed by #760). +Describing the selected physical plan does not establish that it implements +the query. If lowering drops a search expression or optimization drops an +order key, a descriptor derived from that plan can describe the wrong plan +consistently. Validation needs the original typed query requirements as well +as facts derived from the selected operators. + ## User and operational behavior ### Diagnostics @@ -120,13 +131,14 @@ Codes are grouped by who refuses: `Q…` the parser, `T…` the type checker, published; its message may improve. The numbers are assigned in the one catalogue by the change that adds each refusal. -- **HTTP.** A refused query is a `400` whose error body carries an additive +- **HTTP.** A parse, type or planner query-shape refusal is a `400` whose error body carries an additive `diagnostic` object (`code`, `position` or `stage` and `expression`, `expected`, `fix`, and an optional `suggestion`: byte-range edits of the original source with an `applicability` of `machine_applicable` or `needs_review`, so an agent can apply a repair without parsing prose). A - refusal by design is never a `500`; a planner defect - still is. + query-shape refusal is never a `500`; a planner defect still is. + `FullTextIndexRequired` is the index-prerequisite exception below: it + returns `409`, as does `FullTextIndexRebuildRequired`. - **CLI.** The human formats print the form above on stderr with no colour codes and no backtrace. `--json`, `--format json` and `--format jsonl` print the API's error body on stdout, pretty or as one line. @@ -167,16 +179,19 @@ query passages($matter: String, $q: String) { } ``` -The ranked binding is where the component's scan starts. When the other end -of the component is far more selective (one node selected by its key), engine -v2 may instead start there and rank only the rows the traversal reaches; the -answer is the same, only the cost differs. One shape stays refused, as a +The ranked binding is where the component's scan starts. A later optimization +may start at a selective other end and use traversal eligibility during +retrieval, but only under the mode-specific conditions in +[The ranked binding roots its component](#the-ranked-binding-roots-its-component). +An approximate recall label alone does not authorize changing a candidate +window or a scoring population. One shape stays refused, as a type error: `rrf()` arms that rank two different bindings of one traversal-connected component, since one scan cannot start at both. ### One total order -Every search order is total and cuts rows: +Every search order is total over the rows produced by its retrieval population. +The final `limit` cuts those rows after eligibility and ordering: | Order | Sort | Cut | |---|---|---| @@ -184,10 +199,29 @@ Every search order is total and cuts rows: | `bm25(…)` | score descending, then the remaining keys, then every binding's id | `limit` rows | | `rrf(…)` | fused score descending, then the remaining keys, then every binding's id | `limit` rows | -`nearest` and `bm25` already behave this way on engine v2. For `rrf`, the -change is visible only where keys follow `rrf()` or where one entity has -several rows; today the top `limit` entities are chosen first and their rows -cut again. +The implicit binding ids are appended once per in-scope binding in lexicographic +UTF-8 binding-name order, preserving the current name-sorted convention. Derive that scope from +the checked query, not physical traversal order. Each id is ascending and nulls +first. Written keys retain their declared direction; +ascending keys place nulls first and descending keys place nulls last. Every +key uses the typed `SortExec` comparison implemented by Arrow +`lexsort_to_indices`, including its string and floating-point ordering. These +choices form the complete comparator that validation checks. + +Nearest candidate membership is approximate, including ties at the candidate +boundary, even when execution uses a flat exact distance scan. A candidate +window cap is distinct from the final row limit. For equal-distance candidates +titled A, B and C and a candidate cap of two, retrieval may supply B,C. A +subsequent title key orders B before C; it does not recover A. This RFC does +not require retrieving every equal-distance candidate outside that window. +An overfetch policy may widen the window as specified below, but a full window +need not widen merely to resolve a secondary-key tie. + +BM25 supplies its full matching population before eligibility, ordering and +the final row limit. RRF fuses the populations its arms supply; a nearest arm +may have its own candidate cap, while a BM25 arm remains uncapped. Those arm +caps are not a second limit on fused entities. `RankFuse` no longer selects +the top `limit` entities before the final row order and cut. `return { $d.slug, rrf(bm25($d.title, $q), nearest($d.embedding, $v)) as score }` projects the fused score the order used, under the rule `nearest` and `bm25` @@ -195,11 +229,17 @@ already follow (`T33`): the projected expression repeats the leading order key. The refusal of a projected `rrf()` (`T37`) is retired. In an aggregate query, the leading search function selects the population the -aggregate reads (the matches, or the `nearest` window) and does not order the -groups; the remaining order keys order the groups, bound against `return` as -the shared expression model specifies, and the group keys break remaining -ties, so `limit` is deterministic. With no key after the search function the -groups are unordered, as today, and a test cannot expect an order. +aggregate reads (the matches, or the nearest window) and does not order the +groups. The remaining order keys order groups, bound against `return` as the +shared expression model specifies. Append all grouping expressions in their +first-occurrence order in the checked `return` declaration, each ascending +and nulls first, using the same typed comparator defined above. The sequence +comes from the query, never the optimizer's grouping-column layout. These +implicit keys break remaining ties before `limit` cuts groups. For groups +`(category=null, count=1)` and `(category="A", count=1)`, ordering by count +descending with limit one therefore selects the null category. With no key +after the search function the groups remain unordered, as today, and a test +cannot expect an order. ### Read descriptors @@ -207,14 +247,35 @@ The canonical read envelope (`POST /query`, stored-query invocation, CLI `--json` and the `jsonl` metadata record) gains two additive arrays: ```json -"retrievals": [{ "binding": "d", "property": "embedding", "kind": "nearest", +"retrievals": [{ "id": "r0", "binding": "d", "property": "embedding", "kind": "nearest", "recall": "approximate", "embedding_coverage": { "state": "unknown", "reason": "not_computed" } }], -"metrics": [{ "column": "score", "kind": "distance", "source": "nearest", +"metrics": [{ "column": "score", "kind": "distance", "retrieval_id": "r0", "source": "nearest", "binding": "d", "property": "embedding", "descending": false, "recall": "approximate" }] ``` +Retrieval ids identify producers within one accepted plan. Each fusion names +its arms in query argument order. For +`rrf(bm25($d.title, $q1), bm25($d.title, $q2))`, the shape is: + +```json +"retrievals": [ + { "id": "r0", "kind": "rrf", "arms": ["r1", "r2"], "recall": "exact" }, + { "id": "r1", "kind": "bm25", "binding": "d", "property": "title", "recall": "exact" }, + { "id": "r2", "kind": "bm25", "binding": "d", "property": "title", "recall": "exact" } +], +"metrics": [{ "column": "score", "kind": "rrf", "retrieval_id": "r0", + "source": "rrf", "descending": true, "recall": "exact" }] +``` + +An RRF descriptor uses `arms` instead of a singular `binding`/`property`. +Its metric likewise refers to the fusion, not to one arm. Distinct producers +keep distinct ids even when kind, binding and property match; parameter values +are not exposed. Multiple projected columns reading one producer reference +the same id. The arrays describe retrieval and score meaning, not the scope +of the internal rewrite-validation guarantee. + - `recall` reports the source's contract, not what one execution did: a `nearest` is `approximate` even when a run happened to scan exactly, so a client never relies on a guarantee that disappears when an index is built. @@ -246,17 +307,257 @@ refused stored query is a pre-upgrade finding, never a boot failure. | Item | Compiler | Engine v2 (planner and operators) | |---|---|---| | Diagnostics | `QueryDiagnostic`, one code catalogue | `PlanError::Unsupported` carries a diagnostic; the ordinary path maps it to a bad request | +| Plan validation | checked declaration, type context and shared expression semantics | one planner-owned validator for lowering, rewrites and physical requirements; engine adapter preserves the accepted plan at execution | | Full-text index | `T27` | index presence as a planning fact | | Ranking a destination | lowering roots the component at the ranked binding; type error for `rrf` arms on two bindings of one component | optional reversal when the other end is selective | | Total order | `T33` admits a projected `rrf()`; `T37` retired | fused score column, `Sort` over the fusion and over a search-ordered aggregate | -| Descriptors | — | derived from the physical plan's ranked scans and fusion | -| Read floor | — | — (server instrumentation) | +| Descriptors | None | derived from the physical plan's ranked scans and fusion | +| Read floor | None | Server instrumentation | Engine v1 is the frozen reference engine (`crates/omnigraph-reference-engine`), reached only through a logic-test step's `--- expect same as v1`; this RFC changes none of its bytes. A case for a shape the reference answers wrongly does not use that comparison. +### Validation across planning stages + +One validation module in `omnigraph-planner` owns acceptance across stages. +It reuses the compiler's type rules and the planner's property derivation. +Execution-specific checks belong to an engine adapter; the planner gains no +Lance or execution-engine dependency. The validation behavior below is +proposed; it does not describe shipped checks. A term in bold italics is +defined where it first appears. + +#### Requirements come from the checked query + +Before lowering, derive immutable requirements from the checked declaration, +its type context and the accepted catalog. Keep their typed expressions and +resolved identities available through planning. Deriving them only from +`QueryIR` would miss meaning lost while creating that IR. The checked query +remains authoritative; the requirements are a derived validation input and +never a separately editable retrieval description. + +Planning entry points must therefore retain the checked declaration and +context alongside the IR; an entry point receiving only `QueryIR` cannot +check the earlier translation. + +| Requirement | Derived from | Required check | +|---|---|---| +| Search identity | search expression, binding, property and typed arguments | lowering and physical selection retain the same target and arguments | +| Eligibility and scoring population | predicates and the language's search/scoring rules | retain typed predicate references and corpus identity; the exact subset additionally checks every transformation by the rules below | +| Score meaning | rank expression and projected score references | each score reference resolves to the intended producer, binding and retrieval kind | +| Ordering | user order keys plus this RFC's tie-break rules | the selected operators provide the full global order, including direction, null placement and comparison semantics | +| Row cut | limit and the query stage it applies to | final truncation follows eligibility and the required order; keep nearest candidate caps and RRF arm caps distinct from this final cut; aggregate limits cut groups, ordinary limits cut rows | +| Approximation | the retrieval kind's declared contract | preserve the exact/approximate classification; an exact execution of `nearest` does not change its declared contract | + +For a search-ordered aggregate, derive population requirements before grouping +and order/cut requirements after grouping. With no remaining order key, group +order stays unspecified as defined above. Full-text index readiness comes +from the recorded snapshot facts in [Full-text index presence as a planning +fact](#full-text-index-presence-as-a-planning-fact), not from query syntax. + +#### Checks have explicit boundaries + +1. Check `QueryIR` and the initial logical plan against the checked query's + requirements using defined lowering patterns. A shared compiler predicate + still owns query legality; the validator does not duplicate type checking. +2. After optimization, recompute provided properties from operator definitions + and children. Extend `Properties` with typed order keys and score origins + needed for these checks. An order-preserving filter inherits its input + order, a Sort establishes its comparator's order, and a row limit preserves + that order. + These rules apply at the relevant input/output, including hidden score + columns; partition-local order does not establish a global order. +3. Check required properties and recorded prerequisites before accepting the + plan. Cost estimates select among accepted candidates and cannot discharge + a missing requirement. + +Property checks do not by themselves prove row-selection equivalence. A plan +with a dropped filter can have the same schema, ordering and row count. The +stronger check below covers transformations within its stated subset. + +#### Checked rewrites for exact search + +Subset membership is a syntax-and-type check over the checked declaration. +The initial fragment admits one binding of one concrete node type, one pinned +dataset scan, an optional leading BM25 order over one String property and a +literal or bound String argument, eligibility filtering, a total sort, a +nonnegative bound row limit and a final projection. Every expression must +belong to the following closed list; anything else uses invariant checks only. + +| Position | Admitted forms | +|---|---| +| Eligibility operands | resolved scalar properties, typed literals and already bound parameters, all Boolean, integer or String, including their nullable forms | +| Eligibility predicates | same-type equality/inequality and ordered integer/String comparisons, null tests, and Boolean `and`, `or`, `not`; use the compiler's typed null semantics and keep only rows where the predicate is true | +| Sort keys | direct scalar properties, the BM25 score when present, and the implicit binding id; every type must have the typed total comparator defined in [One total order](#one-total-order) | +| Projection | direct scalar properties and references to the selected BM25 score, with aliases; no computed expressions | + +No implicit cast, arithmetic, division, function call other than the selected +BM25, clock/random read, subquery, list operation, traversal, join, optional +match, aggregation, distinct, offset, nearest or RRF belongs to this fragment. +There is no inference from a general expression's claimed determinism. A +parameter is resolved once; rewriting cannot reread the clock or substitute a +new value. The fragment extends only through a versioned rule addition. + +The notation below describes proposed typed structures, not new GQ syntax or +existing Rust APIs. `X` denotes a particular typed subtree, not any plan with +allegedly equal rows. `a` contains the resolved search target and bound query +value; `p` is a typed eligibility predicate; `K` is the complete comparator; +`V` is the visible projection; `n` is the bound final limit. `C` identifies +the accepted schema type and property identities, table incarnation, pinned +dataset, and the analyzer/scoring configuration and pinned statistics used by +BM25. The planner records the facts forming `C`; replay verifies them against +the accepted view. A changed eligible-id mask is not the same `C`. + +First reconstruct the canonical logical form from those checked inputs: +`Project(V, Limit(n, Sort(K, Filter(p, S))))`, where `p=true` when absent and +`S` is `Scan(C)` or `BM25(a, Scan(C))`. Match the compiler IR and initial +logical plan to this form using the rules below, including typed column/alias +maps. This also checks any pre-planner constant substitution against the +retained bound values. A declaration, expression or hidden score lost during +lowering therefore cannot be accepted merely because the remaining plan is +well typed. + +A ***rewrite trace*** is a checked derivation from that form to the selected +physical plan. It records rule ids, subtree references and typed substitutions. +The checker reconstructs each successor itself and compares its typed +structure to the proposed successor. Referenced nodes must already exist in +the checked derivation. Node identity is established from validated contents; +an optimizer-supplied id or hash is insufficient. The final reconstruction +must equal the candidate, including all operator arguments, hidden-column +maps and bound values. Checking asks for identity and finite pattern matches, +never an execution-time proof that arbitrary inputs produce the same rows. + +The initial rule catalogue is closed: + +| Rule and typed pattern | Reconstructed successor and mechanical checks | +|---|---| +| Lower `Scan(C)` | A physical unranked Scan of the identical pinned source and typed columns, without an absorbed filter or row cap. | +| Lower `BM25(a, Scan(C))` | A physical ranked Scan with BM25 access, identical `a` and `C`, an uncapped matching population, no eligibility prefilter, and a score-column map to this query producer. | +| Lower `Filter(p,X)`, `Sort(K,X)`, `Limit(n,X)` or `Project(V,X)` | The corresponding physical operator over the already checked child, copying the typed predicate, complete comparator, count or projection map respectively. No cast, predicate split, reordering or row cap is added. | +| Place a projection before filtering/sorting | `Project(V, Limit(n, Sort(K, Filter(p,X))))` may become `Project(V, Limit(n, Sort(K, Filter(p, Project(H,X)))))`. `H` is the typed union of all columns referenced by `V`, `K` and `p`, including score and id carriers. Only direct column references are admitted; the outer visible projection and its output schema remain unchanged. | +| Absorb an unranked scan filter | `Filter(p, Scan(C))` may become `Scan(C, filter=T(p))` only through the admitted typed translation `T` below. The scan has no ranking or row cap; `C` and its output multiplicity are unchanged. | +| Use ordered top-k | `Limit(n, Sort(K, fetch=f, X))` becomes `Sort(K, fetch=min(n,f), X)`, taking `min(n,None)=n`. The same checked child `X`, typed comparator `K` and hidden-column map are retained. The Sort specification emits that ordered prefix. | +| Remove a redundant Sort | `Sort(K, fetch=None, X)` becomes `X` only when independently derived global ordering has the full typed `K` as a prefix. With `fetch=f`, the successor is `Limit(f,X)`. Direction, null placement, comparison semantics, tie keys and output column mapping must all match. | + +`T` is a finite, versioned table from the admitted predicate's typed operators +to typed scan-filter operators. Literal/parameter and property leaves retain +their exact type, value and resolved identity; a comparison keeps its opcode +and typed comparison semantics; null tests and Boolean nodes keep their +three-valued truth tables. A recursive match must succeed at every node, +with no text serialization, coercion or unsupported operator. Each table +entry names source and target implementations of the same specified semantics +and has null/boundary conformance tests. If an entry is absent, the absorption +rule is unavailable and the standalone Filter remains. This is an enumerated +implementation assumption, not a runtime test of arbitrary predicate +equivalence. For BM25 the initial rules always keep eligibility above search; +they never absorb it into the scoring scan. + +Projection carriers are internal columns, not returned fields. The outer +visible projection may be implemented by a checked output-schema mapping, +including the removal of carriers by `SortExec`; the checker must verify +that mapping rather than treating hidden columns as visible output. There is +no rule to move a limit across a filter, change `C`, replace BM25 with another +search implementation, or fuse arbitrary searches. An optional optimization +without a rule retains its validated predecessor. + +The trusted implementation assumptions are the specifications of pinned +Scan, BM25 matching/scoring, typed predicate evaluation, Filter, Project, +global Sort/ordered top-k and Limit, plus the admitted scan-filter mappings. +Tests must exercise each admitted implementation law when its rule is enabled. +Given those assumptions, accepted derivations in this fragment preserve visible +rows, multiplicities, score values and total order for all data satisfying +the recorded facts. The checker verifies the derivation; it does not prove +the implementations themselves correct. + +ANN, RRF, traversal, aggregation and nonmember expressions receive checks of +binding/argument identity, retained predicate references, score origin, +declared approximation, required ordering, cut placement and prerequisites, +with their execution regressions. These checks do not establish population +equivalence. Membership and the accepted scope are recorded in internal +inspection/replay evidence, separately from retrieval `recall`. + +#### Evidence has bounded size and checking work + +Store the derivation as a directed acyclic graph of shared typed nodes and +references, not a complete plan copy per optimizer step. Reject cycles, +forward/invalid references and duplicate conflicting definitions. Before +decoding saved evidence, check its serialized byte length. Charge decoded +node allocations, expression bytes, rule applications and visited nodes +during decoding, reconstruction and property derivation, before exceeding +their configured limits. References do not bypass those charges. + +The implementation must set finite limits for evidence bytes, retained nodes, +rule applications and checking work before enabling acceptance. Those limits +are explicit validation inputs, captured for inspection. The same query, +bound values, facts, evidence, rule/semantics versions and limits produce the +same accept, invalid or budget-exhausted result; costs do not decide validity. +Wall-clock timeout and cancellation remain separate resource outcomes. +Exhaustion is not evidence that a query or plan is invalid. + +If optional rewriting exceeds its evidence budget, use the already validated +predecessor and its evidence. If baseline validation or replay exceeds the +budget, return a typed resource-limit outcome naming the exhausted limit; +never execute unchecked, relabel the query unsupported, or report a validator +defect solely because a budget ran out. Account evidence memory to the query's +memory budget and discard unused optimizer derivations. + +#### Execution and replay use the same acceptance path + +Only the validator constructs an accepted-plan wrapper, a type whose private +constructor prevents unchecked plans from entering execution. Normal +execution and replay consume that wrapper; deserialization cannot create it. +Changing a plan requires validation again. Its acceptance includes every +adaptive search policy encoded in the plan, not only one predicted execution. + +For each such policy, validate its mode, typed arguments, eligible-set source, +candidate/arm caps, probe and overfetch bounds, ordered finite transitions, +branch guards, and fallback/termination outcomes. Examples are the nearest +prefilter/postfilter/proven-empty choices, overfetch rungs and exact fallback, +and RRF's BM25 prefilter gate. A branch guard may require a pinned coverage +fact or a runtime eligibility count. The plan must name that evidence and the +comparison to perform; a false or unavailable fact selects only a declared +fallback or refusal. Validation checks every reachable alternative against +its mode-specific prerequisites. It does not infer equal ANN candidate sets +from the alternatives' common approximate label. + +Execution may select branches and advance rungs within that accepted policy. +It must record the branch, supporting facts and rung used for inspection and +replay, and cannot invent a new prefilter, cap, search mode or transition. +Replay retains the same policy, its parameters and relevant snapshot pins. +Data-dependent branch evidence must be re-established from those pins, not +trusted because it was serialized; any recorded branch observation must agree +with the re-established evidence. This preserves the existing replay scope +without asserting general equivalence between approximate search policies. + +Replay carries original query inputs and the evidence for its accepted scope, +including the exact-subset derivation when applicable. Its envelope identifies +the evidence format, rule catalogue and compiler/operator semantics versions. +Compatibility is explicit; a version change does not reinterpret an old rule +id. Replay reruns compiler legality and type resolution against the accepted +catalog and captured bound parameters, then derives requirements again. A +serialized type context or binding identity cannot validate itself. Relevant +schema identities and snapshot facts remain fixed through execution. + +| Replay or validation outcome | Required action | +|---|---| +| Evidence is absent, obsolete, or uses unsupported format/rule/semantics versions | Return a typed replan-required outcome naming the incompatible version or missing evidence. A legacy bare `BoundPlan` needs the caller to resupply the original query and inputs; the artifact alone cannot be migrated. | +| Relevant catalog or snapshot facts differ | Return an incompatible-facts outcome naming the changed prerequisite; replan against a newly accepted view. Do not silently combine old evidence with new facts. | +| Supplied evidence is malformed or fails a supported rule | Return invalid-evidence with the failing node/rule and reason. It is not a compiler defect merely because an external artifact is invalid. | +| The planner produces invalid evidence, or a supported baseline lacks its required check | Report an internal compiler/planner defect. Do not turn the validator gap into a new language refusal. | +| A configured validation limit, timeout or cancellation ends checking | Return the resource outcome defined above; no accepted wrapper is constructed. | + +Existing query-legality refusals and unavailable index prerequisites retain +their distinct outcomes. Cross-version executable-plan portability is not +promised; compatible query inputs can always be submitted for fresh planning. + +Physical-to-executor lowering remains a trusted implementation boundary, +guarded by the existing operator-lowering and replay test owners. The engine +adapter implements the selected operators and their accepted policies; it +must introduce no search, order or cut choice outside them. Any later +execution-plan rewrite needs its own checked rule before extending the +guarantee to that boundary. + ### Diagnostics `QueryDiagnostic { kind, code, message, position, stage, fix }` lives in the @@ -301,10 +602,17 @@ traversals leaving it. A key-selected other end makes the ranked root expensive: the ranked scan starts from the whole type, and the `nearest` overfetch ladder can end in an exact pass over it. The later performance step lets the v2 planner reverse a -ranked component when its cost model estimates the other end as selective, -running the traversal first and ranking the reached rows with their ids as -the scan's prefilter; the planner's current refusal of a ranked dependent scan -(`optimizer.rs` `rank`) becomes that plan. +ranked component only when both its cost estimate and a mode-specific rule +admit it. For uncapped BM25, the traversal eligibility set must cover every +surviving row, preserve multiplicity, and leave the scoring corpus and score +values unchanged. A prefilter that changes tail scoring is not admitted. +For nearest, traversal-first execution must preserve the declared eligible +population and candidate-window policy; relabeling a changed population +`approximate` is insufficient. A constitutive nearest arm window in RRF +cannot be replaced by a window ranked over traversal survivors. Without a +rule establishing these conditions, retain the ranked-root plan. Step 8 +supplies the additional rules and regressions before replacing the current +ranked-dependent-scan refusal (`optimizer.rs` `rank`). ### Total order @@ -315,26 +623,38 @@ truncating to the limit and emits the fused score as a `Float64` column declared tie-break ids, and `Limit` cuts rows. The `Sort` with a `fetch` is already a streaming top-k, so memory stays bounded by `fetch` rows plus one batch above the fusion's own output. Winner selection inside the fusion no -longer decides the cut, so fused ties need no plateau rule. +longer decides the final row cut, so fused ties need no plateau rule over +the arm populations supplied. This does not expand nearest arm windows. For an aggregate under a search order, `sort_keys` returns the remaining keys -instead of `None`, bound against `return`, with the group keys as the declared -tie-break. The runner's `ordered_refusal` drops its `rrf` rule and admits an +instead of `None`, bound against `return`, with the complete group-key +comparator defined in [One total order](#one-total-order). The runner's +`ordered_refusal` drops its `rrf` rule and admits an aggregate order whose keys make it total. ### Read descriptors -Each ranked `Scan` (`RankedAccess`) and each `RankFuse` of the physical plan -yields one retrieval; each projected column that reads `_distance`, `_score` -or `_rrf` yields one metric. Both are computed from the plan that ran, so they -cannot disagree with the order the rows have. `recall` is a function of the -retrieval kind: `nearest` is approximate, `bm25` exact, `rrf` approximate when +Each ranked `Scan` (`RankedAccess`), each retrieval arm stored inside a +`RankFuse`, and the fusion itself yields a retrieval descriptor. Assign +plan-local ids in deterministic preorder of the accepted physical plan, +visiting fusion arms in query argument order. A retrieval embedded in an arm +is identified by its fusion node and arm position, even when displayed fields +match another arm. Revisiting the same producer reuses its id, including when +multiple projections share it. The same accepted plan retains these ids on +replay; a replan need not. Each projected column reading `_distance`, `_score` +or `_rrf` yields a metric with its producer's `retrieval_id`. Both arrays +are computed from the plan that ran, so they +describe the selected retrieval and score producers. The shared validator +separately checks the plan against the query's requirements within its stated +scope. `recall` is a function of the retrieval kind: `nearest` is approximate, +`bm25` exact, `rrf` approximate when either arm is. ## Invariants - **Integrity failures are loud (8).** Strengthened: every silent wrong answer this RFC names becomes a correct answer or a typed refusal. + Validator-detected compiler or optimizer defects remain internal failures. - **Query semantics are typed structures (9).** Strengthened: refusals carry a typed diagnostic instead of a string with a code prefix; the root choice is a lowering rule, not a textual accident. @@ -344,9 +664,12 @@ either arm is. lifted by the lexical RFC's exact scan. - **Bounded, observable resource use (11).** The fusion's `Sort` is a streaming top-k; descriptors add no scan; the read floor is reported. + Validation evidence and checking work have explicit limits and typed + exhaustion outcomes, including decoding saved evidence. - **One source of truth (12).** Descriptors derive from the plan; no retrieval description is kept beside it. The index fact is a recorded plan - input. + input. Validation requirements derive from the checked query and remain + independent of edits to the candidate plan. - Deny-list: no side channel for discarded rank (the fused score is a column), no string-built predicates, no logical precondition on index coverage. @@ -363,6 +686,12 @@ either arm is. - **Reverting** needs no storage work: every field is additive, the root rule is a lowering choice, and the refusals can be relaxed, at the cost of the silent wrong answers they remove. +- **Plan replay:** the internal serialized plan gains validation inputs and + versioned evidence under the [acceptance and replay contract](#execution-and-replay-use-the-same-acceptance-path). + Older plans require the caller's original query and inputs for fresh + planning; they cannot bypass validation or be promoted by deserialization. + This does not change the graph storage format or the deprecated read + envelope. A validator coverage gap does not define a new language refusal. ## Alternatives @@ -378,7 +707,23 @@ either arm is. of removing it. - **Record the retrieval in the `QueryIR`.** Rejected: engine v2's planner already states it once as typed plan nodes; a second record would be a - second source of truth. + second source of truth. Derived query requirements serve validation only; + they do not supply retrieval descriptors. +- **Validate only the final physical plan.** Rejected: deleting a search or + filter can leave a structurally valid, ordered plan. Requirements retained + from before lowering expose the missing obligation; checked rewrites cover + population preservation within the exact subset. +- **Keep only local assertions and regression tests.** These remain necessary, + but do not supply a shared acceptance path for normal planning and replay. + A final Sort cannot recover rows discarded by an invalid earlier cut. +- **Require general query equivalence before shipping.** Rejected for this + slice: ANN, fusion, traversal and aggregation need their own semantics and + transformation rules. Their enumerated invariant checks and regressions + remain required while the first rewrite checker covers the exact subset. +- **Retain a complete optimizer history.** Rejected: repeated intermediate + plans add size and replay work without strengthening the scoped check. + Shared typed nodes and a compact derivation carry only the rules needed to + reconstruct the candidate. ## Evidence and tests @@ -386,37 +731,81 @@ Each change lands with its regression at the owner the [testing map](../dev/testing.md) names: - compiler parser, type-check and lowering tests for the diagnostics, `T27`, - the `rrf()` arms error and the root rule; + the `rrf()` arms error, the root rule and requirement derivation before + lowering; - `crates/omnigraph-gqt/cases/v2/` cases for every behavior visible in rows or errors, `--- expect plan` where the plan is the claim (the ranked root, the fusion `Sort`, the aggregate `Sort`), named for the issue each closes; - `crates/omnigraph-planner/tests/` for the planning fact and the diagnostic - of a refusal; + of a refusal; extend `query_plan.rs`, `bound_plan.rs` and `lower_walk.rs` + for validation and replay evidence, alongside the optimizer's rule tests; - server `data_routes` and `openapi`, CLI `cli_queries` and `parity_matrix` for the error body and the envelope. +Validation tests must accept every admitted rewrite and reject a changed +search target, dropped predicate, changed scoring corpus, incomplete +comparator, premature row cut, removed Sort `fetch`, stale prerequisite, or +trace ending at a different candidate. Use tied scores, several rows per entity, filtered-out +high scores and group-order cases so schema or row-count checks cannot pass +in place of the required semantic check. A predicate/population rejection +claimed from rewrite validation must use the exact supported subset. + +Membership tests cover every admitted expression form and the excluded +families. Rule tests cover baseline lowering, bound-value substitution, +projection carriers, nullable scan-filter translation, Sort `fetch` and every +typed comparator field. Adaptive-policy tests alter a guard, prefilter source, +rung, cap or fallback and require rejection; exercise each permitted branch +and its recorded evidence through replay. + +Use the A/B/C nearest boundary-tie example to assert the order within the +retrieved window without demanding A from a B,C window. Separately assert +BM25's uncapped matching population and RRF arm caps versus its final row +limit. Cover nullable aggregate group keys and multiple group/binding keys +at a limit boundary. Envelope tests use two BM25 arms over the same property, +shared score projections and ordered arm references, without parameter values. + +Extend `engine_v2_plan_replay.rs` to require the same checks on replay and +assert the distinct outcomes for missing/obsolete versions, incompatible +facts, malformed evidence, internal defects and validation exhaustion. +Test oversized evidence before decoding, cycles, invalid references and +each deterministic budget boundary. An over-budget optional rewrite must +retain its validated predecessor; an over-budget baseline must not execute. +Add an instrumented planning/replay case recording evidence bytes, retained +nodes, rule applications and node visits as query/rewrite size grows, with +elapsed time and peak memory reported as measurements. Existing +in-source `engine/report/tests.rs` tests own physical-node-to-executor +correspondence. ANN/RRF/traversal/aggregate +regressions verify their declared behavior without claiming the exact +subset's equivalence guarantee. No new test may compare a known-wrong shape +against the frozen reference engine. + The four probe results in the motivation are reproductions, not tests; each issue restates its shape for the case that will own it. ## Rollout -Each step is shippable alone, lands on engine v2 and the shared compiler, -and closes its issues. +Each step lands on engine v2 and the shared compiler. Steps 1 and 2 can ship +independently. Step 2a establishes validation before steps 3-6 add and enforce +their respective requirements; none is declared complete without its checks +and regressions. Step 7 can ship independently. | Step | Delivers | Closes | |---|---|---| -| 1 | Diagnostics contract for parse and type refusals (PR #759, merged 2026-09-30) | — | +| 1 | Diagnostics contract for parse and type refusals (PR #759, merged 2026-09-30) | None | | 2 | Planner refusals carry diagnostics; a refusal by design is a bad request | #786 | +| 2a | Shared validation requirements and acceptance path; finite exact-subset rules, adaptive-policy checks, bounded and versioned replay evidence | supports steps 3-6 | | 3 | `T27` and `FullTextIndexRequired` | #747 | | 4 | The ranked binding roots its component; the `rrf()` arms type error | #789 | | 5 | One total order: fused score column, `Sort` over fusion and search-ordered aggregates, `T37` retired | #787, #788 | -| 6 | Read descriptors | — | +| 6 | Read descriptors with producer ids and fusion-arm references | None | | 7 | Served read floor | #752 | -| 8 | Engine v2 reverses a ranked component whose other end is selective | — (performance) | +| 8 | Engine v2 reverses a ranked component when selectivity and its mode-specific validation rule admit it | None (performance) | `implementation` becomes -`in-progress` when step 1 lands and `complete` when step 7 does; step 8 is a -performance follow-up. +`in-progress` when step 1 lands and `complete` when steps 1-7, including 2a, +are complete; step 8 is a performance follow-up. Step 8 must add validation +rules for its traversal reversal before enabling that transformation; it +does not inherit the exact single-scan subset's equivalence claim. ## Unresolved questions @@ -424,9 +813,9 @@ None. ## Decision log -- 2026-09-01 — first draft opened in PR #606, written against engine v1. -- 2026-09-19 — accepted as drafted in PR #606. -- 2026-09-28 — rewritten against engine v2 (`main` `b14c22c5`) and returned +- 2026-09-01: first draft opened in PR #606, written against engine v1. +- 2026-09-19: accepted as drafted in PR #606. +- 2026-09-28: rewritten against engine v2 (`main` `b14c22c5`) and returned to draft pending the unresolved question. Removed, because engine v2 already does it or the freeze forbids it: the `QueryIR` retrieval field, the compile-time refusal of a ranked or searched traversal destination (`T26`), @@ -437,14 +826,14 @@ None. full-text refusal is a planning fact, and the v1 door refuses the shapes v1 answers wrongly. The drafts in PR #606 are superseded as a whole; this file replaces them. -- 2026-09-29 — engine v2 became the only query engine in v0.12.0 (PR #795), +- 2026-09-29: engine v2 became the only query engine in v0.12.0 (PR #795), and engine v1 the frozen reference engine. Removed: the engine v1 door refusals of steps 2, 3 and 5, the `V…` code group, the two alternatives about engine v1, the note about the frozen `tests/search.rs`, and the unresolved question, which asked the engine owner to agree to those refusals. The planner-refusal rule of step 2 stays. Every defect this RFC names was checked again in code at `baf10c94`. -- 2026-09-30 — step 1 shipped in PR #759, and `implementation` moved to +- 2026-09-30: step 1 shipped in PR #759, and `implementation` moved to `in-progress`. The maintainer's review added the optional `suggestion` to the diagnostic: source edits a caller can apply mechanically, marked machine-applicable only after the edited text parses. diff --git a/docs/rfcs/README.md b/docs/rfcs/README.md index c774166ab..e11f309bf 100644 --- a/docs/rfcs/README.md +++ b/docs/rfcs/README.md @@ -208,7 +208,7 @@ then dated RFCs by date. | [0044](0044-edge-keys.md) | Edge keys: derived edge identity | maintainer | draft | in-progress | | [0045](0045-gq-logic-tests.md) | GQ logic tests | maintainer | draft | partial | | [0046](0046-index-status.md) | Read-only index status | maintainer | draft | not-started | -| [0047](0047-search-plan-truth.md) | Search plan truth: loud search failures and one total order | public | draft | in-progress | +| [0047](0047-search-plan-truth.md) | Search plan validation and result guarantees | public | draft | in-progress | | [0049](0049-control-plane-seams.md) | Control-plane seams: observe, readiness witness, bounded shutdown | maintainer | accepted | partial | | [0051](0051-json-output-via-arrow.md) | JSON output via Arrow | maintainer | draft | partial | | [0052](0052-managed-control-plane-cli.md) | Managed control-plane CLI | maintainer | accepted | complete | From 77a99537f6c36a73a34aead9af33db01e7cf7a37 Mon Sep 17 00:00:00 2001 From: azim afroozeh <13484327+azimafroozeh@users.noreply.github.com> Date: Wed, 30 Sep 2026 23:29:49 +0200 Subject: [PATCH 7/7] hm --- docs/rfcs/0047-search-plan-truth.md | 100 +++++++++++++++++++++++++++- 1 file changed, 97 insertions(+), 3 deletions(-) diff --git a/docs/rfcs/0047-search-plan-truth.md b/docs/rfcs/0047-search-plan-truth.md index 2c2045970..35a8ea311 100644 --- a/docs/rfcs/0047-search-plan-truth.md +++ b/docs/rfcs/0047-search-plan-truth.md @@ -1,6 +1,6 @@ --- rfc: "0047" -title: "Search plan truth: loud search failures and one total order" +title: "Search plan validation and result guarantees" track: public status: draft implementation: in-progress @@ -14,7 +14,7 @@ superseded_by: [] blocked_on: [] --- -# RFC 0047: Search plan truth: loud search failures and one total order +# RFC 0047: Search plan validation and result guarantees Existing-behavior code references are at `main` `baf10c94`. Behaviour marked as observed was reproduced at `b14c22c5` with a logic-test probe on engine v2 @@ -328,6 +328,55 @@ Lance or execution-engine dependency. The validation behavior below is proposed; it does not describe shipped checks. A term in bold italics is defined where it first appears. +#### Query and explain share acceptance + +Ordinary execution, plain `EXPLAIN` and inspected execution call the same +planning and validation path for the underlying query. Explain is a rendering +of an accepted plan, not an alternate planner or a stronger validation mode. +For identical checked-query inputs, resolved parameter values, schema and +snapshot facts, planning statistics, settings, rule/semantics versions and +validation limits, they produce the same accepted plan and validation scope, +or the same deterministic planning/validation failure. Failure preserves its diagnostic +class, code and reason; source positions still refer to the submitted source. +Wall-clock timeout, cancellation and failure to obtain planning inputs are +separate attempt outcomes, outside this deterministic equality. +Bind volatile inputs such as `now()` once for this comparison. Two separate +requests may capture different inputs and are not covered by that equality. + +Validation failure returns an error before execution or a successful explain +document. After acceptance, plain explain returns the plan without executing +its operators. It validates adaptive-policy definitions and prerequisites +known at planning, but does not run their data-dependent guards or claim a +runtime branch was taken. Execution still evaluates those guards and can fail +on I/O, memory, cancellation or other execution conditions that plain explain +does not exercise. Successful explain therefore establishes planning +acceptance, not successful execution. + +Successful explain carries a compact summary derived from the accepted-plan +wrapper, for example `"validation": { "scope": "exact_subset" }`. The scope +is `exact_subset` when the closed fragment and its derivation were checked, +or `invariants_only` for the explicitly narrower checks. No optimizer or +renderer may manufacture or upgrade this scope. Acceptance is implicit in a +successful document; a second `accepted` flag adds no guarantee. The full +derivation remains internal evidence for validation, replay and diagnostics, +not a required field of ordinary read responses. + +Inspected execution renders this summary from the same accepted plan that +produced its rows, without a second planning run. GQT asserts the scope +through the following new `expect plan` form, alongside existing row and +shape expectations: + +```text +--- expect plan +validation scope exact_subset +``` + +`validation scope invariants_only` selects the other scope. Missing evidence +or a different scope fails the assertion. Failed planning uses the existing +`expect error:` mechanism instead; it produces no successful plan summary. +These assertions inspect the validator's result and do not implement another +validator in GQT. + #### Requirements come from the checked query Before lowering, derive immutable requirements from the checked declaration, @@ -558,6 +607,32 @@ must introduce no search, order or cut choice outside them. Any later execution-plan rewrite needs its own checked rule before extending the guarantee to that boundary. +#### Which checks a query performs + +| Work | Fresh query planning and plain explain | Other owner or stage | +|---|---|---| +| Compiler legality, parameter/type resolution and query-derived requirements | Required through the shared compiler and validation path | Do not repeat compiler rules in another checker | +| Lowering checks, physical requirements, recorded index/snapshot prerequisites and adaptive-policy definitions | Required before accepting a new plan | Runtime evaluates the declared data-dependent guards when reached | +| Exact-subset derivation checking | Required for that subset under this RFC, including ordinary production planning | Nonmember queries receive invariant checks only | +| Saved-evidence decoding, format/version compatibility and reconstruction of saved inputs | Not needed for a fresh in-memory plan | Required when accepting a serialized plan for replay | +| Rendering the scope, full trace or other explain diagnostics | Not needed to execute an ordinary query | Explain, inspection and replay export render from accepted evidence | +| Expected-row comparisons, malformed-plan injections, reference comparisons and overhead measurements | Not part of serving a query | GQT and the existing Rust test owners | + +The derivation checker is additional protection against compiler/optimizer +defects; it is not needed merely to evaluate the operators. Requiring it in +production for the supported fragment is a deliberate assurance and cost +choice in this RFC. Moving it to tests or a debug mode would weaken that +production guarantee and require an explicit change to the acceptance +contract, shared by query and explain. Explain cannot silently enable checks +that ordinary planning skips. + +Validate a newly built plan once before using its accepted wrapper. Execution +does not recheck the entire derivation for each row, batch or policy rung; +it checks the declared runtime guards and resource limits where required. +A fresh plan needs no serialization-and-replay round trip to establish +acceptance. Mutation of the plan or acceptance of a saved plan still requires +validation as specified above. + ### Diagnostics `QueryDiagnostic { kind, code, message, position, stage, fix }` lives in the @@ -678,6 +753,9 @@ either arm is. - **Wire:** `diagnostic`, `retrievals`, `metrics` and `usage` are additive and omitted when absent. `POST /read` is untouched. OpenAPI regenerates. +- **Explain:** the additive validation-scope summary comes from the same + acceptance path as execution. It does not add a validation field to an + ordinary read response or expose the complete derivation by default. - **Language:** `T27` and the `rrf()` arms type error refuse queries the compiler accepts today; both previously produced wrong or failing results. - **Order:** results change only where an order was not total: ties inside a @@ -757,6 +835,20 @@ typed comparator field. Adaptive-policy tests alter a guard, prefilter source, rung, cap or fallback and require rejection; exercise each permitted branch and its recorded evidence through replay. +Add parity tests for ordinary planning, plain explain and inspected execution +using one captured set of query inputs, including resolved `now()`, snapshot +facts, statistics, settings and validation limits. Require the same accepted +plan/scope on success and the same diagnostic class/code/reason on a +deterministic planning or validation failure, including validation-budget +exhaustion. Timeout and cancellation do not require equal outcomes across +attempts. Assert that plain explain runs no query operators; +an execution-only failure may occur after successful explain. GQT cases +assert `validation scope exact_subset` or `validation scope invariants_only` +on the explain document from their own run. Parser/runner tests reject a +missing or mismatched scope; error cases retain ordinary error expectations. +Ordinary execution must pass the acceptance gate even when no explain output +is requested. Keep malformed evidence and bypass tests in the Rust owners. + Use the A/B/C nearest boundary-tie example to assert the order within the retrieved window without demanding A from a B,C window. Separately assert BM25's uncapped matching population and RRF arm caps versus its final row @@ -793,7 +885,7 @@ and regressions. Step 7 can ship independently. |---|---|---| | 1 | Diagnostics contract for parse and type refusals (PR #759, merged 2026-09-30) | None | | 2 | Planner refusals carry diagnostics; a refusal by design is a bad request | #786 | -| 2a | Shared validation requirements and acceptance path; finite exact-subset rules, adaptive-policy checks, bounded and versioned replay evidence | supports steps 3-6 | +| 2a | Shared query/explain validation and acceptance; exact-subset rules, adaptive-policy checks, bounded/versioned replay evidence, GQT scope assertions | supports steps 3-6 | | 3 | `T27` and `FullTextIndexRequired` | #747 | | 4 | The ranked binding roots its component; the `rrf()` arms type error | #789 | | 5 | One total order: fused score column, `Sort` over fusion and search-ordered aggregates, `T37` retired | #787, #788 | @@ -837,3 +929,5 @@ None. `in-progress`. The maintainer's review added the optional `suggestion` to the diagnostic: source edits a caller can apply mechanically, marked machine-applicable only after the edited text parses. +- 2026-09-30: retitled to "Search plan validation and result guarantees" to + reflect shared validation and the explicit scope of search result guarantees.