From 2152bef8d1a7f085c6686099fe6b6a6933d7330e Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Fri, 16 Oct 2026 12:13:56 -0700 Subject: [PATCH 1/5] rfc: schema namespaces and multi-file schemas A draft RFC for authoring one graph's schema as several .pg files, composed into one canonical source before hashing, and for namespace blocks whose canonical names (crm::Contact) are part of the name and never of identity. It records how the design composes with the polymorphic types draft. --- ...chema-namespaces-and-multi-file-schemas.md | 626 ++++++++++++++++++ docs/rfcs/README.md | 1 + 2 files changed, 627 insertions(+) create mode 100644 docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md diff --git a/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md b/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md new file mode 100644 index 000000000..130e868db --- /dev/null +++ b/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md @@ -0,0 +1,626 @@ +--- +rfc: "2026-10-15-schema-namespaces-and-multi-file-schemas" +title: "Schema namespaces and multi-file schemas" +track: maintainer +status: draft +implementation: not-started +authors: + - ragnorc +created: 2026-10-15 +updated: 2026-10-16 +discussion: null +supersedes: [] +superseded_by: [] +blocked_on: [] +--- + +# RFC: Schema namespaces and multi-file schemas + +## Summary + +This RFC makes two decisions. They can ship separately, and the second depends +on the first. + +**Multi-file schemas.** A graph's schema may be written as several `.pg` files. +The files carry no meaning: there is no `import`, and a file's name or directory +never decides a type's name. Composition is declared once, in `cluster.yaml` or +on the command line, and the loader joins the files in a fixed order into one +canonical source text. Plan, apply, the stored contract, the contract digest and +the cluster ledger all see that one text, as they see one file today. No storage +format, IR, ledger or wire shape changes, and a schema given as one file keeps +byte-identical source and digests. Errors name the file and line. + +**Namespaces.** A declaration may sit in a namespace block, +`namespace crm { node Contact { … } }`, which gives it the canonical name +`crm::Contact`. + +- The namespace is part of the name and never part of identity. `StableTypeId` + stays the identity. +- Moving a type into, out of or between namespaces is a rename. It goes through + the existing `RenameType` and metadata-only `RenameTable` path, and it must be + declared with `@rename_from`, never inferred. +- Queries write qualified names, or make a namespace's names available + unqualified with `use crm;`. +- An unqualified name resolves only in the root namespace and the namespaces a + query `use`s, with no search order. Two matches are a type error, never a + silent choice. +- Every wire surface carries the canonical string (`"type": "crm::Contact"`) in + the field that carries a type name today, with no new field. +- A graph that declares no namespace keeps byte-identical IR, `schema_ir_hash`, + table keys and wire output. + +What does not change: data paths, Lance schemas and index names, the +one-publication schema apply, its main-only restriction, and identity by stable +id. + +## Motivation + +- **Schemas outgrow one file.** One production knowledge graph declares 76 + node types and 426 edge types in a single `.pg` file. Templates that people + copy into a graph arrive as files of their own, and several teams may own + parts of one graph. +- **A schema is one file today, everywhere.** + - `parse_schema(input: &str)` parses one string + (`crates/omnigraph-compiler/src/schema/parser.rs`). + - Every production call site passes one file: `init`, `schema plan`, + `schema apply`, `lint`, and cluster `validate`/`plan`/`apply`. + - `GraphConfig.schema` is one `PathBuf` (`crates/omnigraph-cluster/src/types.rs`). + - Queries can already be a file, a directory or a list (`QueriesDecl` in + `crates/omnigraph-cluster/src/config.rs`). +- **Names are flat.** `type_name = @{ ASCII_ALPHA_UPPER ~ (ASCII_ALPHANUMERIC | "_")* }` + in both grammars, so two templates that each declare `Event` or `OwnedBy` + cannot coexist. Edge names are also unique after case folding across the + whole graph (`normalize_edge_name` in `catalog/mod.rs`). +- **Workarounds don't scale.** A prefix convention such as `Crm_Contact` works + today and costs nothing. It gives no scope to a query, to a future per-type + policy rule, or to an author reading a 400-type schema. + +An issue is not enough: both halves add a grammar surface, and namespaces add a +spelling that is persisted in `__manifest` table keys and in immutable +`__history` extents. + +## User and operational behavior + +### Multi-file schemas + +```yaml +graphs: + main: + schema: schema/ # every *.pg file directly inside, in path order + crm: + schema: [schema/core.pg, schema/crm/] # files and directories, in this order +``` + +```text +omnigraph schema plan --schema schema/ graph.omni +``` + +Discovery copies the `queries:` rules exactly, so one rule covers both. + +- **What a path may name.** A single path may be a file or a directory, and a + list may mix both. + - A directory contributes its top-level `*.pg` files, with no recursion. + - Each directory's files are sorted by path. A list keeps its own order. +- **Refusals and caps.** + - Symlinks and `..` components are refused (`check_config_path`). + - Reads go through `SourceCapture`, with the same per-file and total byte caps + and the same discovery cap as queries. + - A directory with no `.pg` file is an error, not a warning, because a graph + has to have a schema. +- **Composition.** + - The composed source is each file's bytes in order. Each file is preceded by + a banner line `// file: ` and ends with exactly one newline. + - `` is relative to the declared directory, or to the `cluster.yaml` + directory for list entries. + - A schema declared as one file is composed as that file's bytes with no + banner, so every existing graph keeps its digest. +- **What sees the composed text.** It is what `schema plan`, `schema apply`, + the contract's `source`, `source_hash`, the `schema.` resource digest + and `schema show` see. +- **Moving text between files changes the contract.** Moving a declaration from + one file to another changes the composed bytes. That publishes a new contract + with no schema steps, exactly as editing a comment does today. +- **Errors name the file.** + - A syntax error reports `schema/crm.pg:12:5`. + - A semantic error reports the file of the declaration it concerns. + - A name declared twice is an error naming both files. + +### Namespaces in `.pg` + +```pg +namespace core { + interface Named { name: String } + node Project { slug: String @key name: String } +} + +namespace crm { + node Contact implements core::Named { + email: String @key + name: String + } + node Deal { title: String @key } + edge OwnedBy: Deal -> Contact // crm::Deal -> crm::Contact + edge WorksOn: Contact -> core::Project +} + +node Note { body: String } // the root namespace: today's schemas +``` + +- **Paths.** A namespace path is one or more lowercase segments + `[a-z][a-z0-9_]*` joined by `::`, as in `namespace acme::crm { … }`. The path + is flat: `acme::crm` is one namespace, and `acme` gains nothing from it. +- **Blocks.** + - Blocks do not nest. + - Blocks with the same path merge, across files too. + - A file may hold several blocks. +- **The root namespace.** A declaration outside every block is in the root + namespace, where every existing schema already sits. +- **Reserved names.** `omnigraph`, `std` and segments that start with `_` are + reserved for future built-ins. +- **Kinds.** Node and edge names stay separate per kind, as today, inside each + namespace. The interface rule of + [Polymorphic types](https://github.com/ModernRelay/omnigraph/pull/884) (a new + schema may not give an interface the name of a node or edge type) applies to + canonical names, so `crm::Event` as an interface and `core::Event` as a node + may coexist. +- **Edge names.** They are unique per namespace after folding the case of the + final segment, so `crm::OwnedBy` and `hr::OwnedBy` coexist, while + `crm::OwnedBy` and `crm::Ownedby` collide. +- **Resolving names inside a schema.** + - An unqualified reference in namespace `N` resolves to `N::X` or to the root + `X`. If both exist it is an error naming both; if neither exists the type is + unknown. + - A qualified reference resolves exactly. A reference at root level resolves + only in the root. + - Adding a type can therefore make a schema ambiguous. It can never silently + change what an existing reference means. + +### Namespaces in `.gq` + +```gq +use crm; + +query open_deals() { + match { + $d: Deal + $d ownedBy $c + $c worksOn $p + $p: core::Project + } + return { $d.title, $c.email, $p.slug } +} +``` + +- **`use` lines.** They come at the top of a query text and apply to every query + in it. A stored query's payload is its whole file, so the lines are stored + with it. +- **Name resolution.** + - A qualified name resolves exactly. + - An unqualified name must match exactly one type among the root namespace and + the `use`d namespaces. More than one match is a type error listing the + candidates; none is unknown. + - There is no search order, no session-level default and no environment + setting. +- **Where every form applies.** Bindings (`$c: crm::Contact`), traversals + (`$d crm::ownedBy $c`, alternations, undirected and bound edges), `insert`, + `update`, `delete` and `load csv … into node:crm::Contact` all take qualified + names. +- **Telling a binding from an edge.** The parser decides by the case of the + final segment, as it decides by the case of the whole name today. +- **Engine catalogue types.** The built-in types of the + [Engine catalogue](2026-10-16-engine-catalogue.md) (`@Operator`, `@Setting` + and the rest) keep their `@` spelling and belong to no namespace. A qualified + name never carries `@`, so `crm::@Operator` is a syntax error, and `use` never + affects how an `@` name resolves. +- **Breakage surfaces at plan time.** Stored queries are type-checked at + `cluster plan`, at apply and at boot. A schema change that makes a stored + query ambiguous is reported at plan time and never changes the query's meaning + at run time. + +### Wire and tools + +- **Canonical names everywhere.** Every external surface uses the canonical name + in the field it uses today: + - load and export envelopes (`{"type": "crm::Contact"}`, + `{"edge": "crm::OwnedBy"}`); + - change-feed `type=` filters, blob selectors and `export --type`; + - `$x.@type` values and every HTTP and MCP `type_name` field. +- **No `use` outside queries.** Load envelopes and filters have no `use`. They + take canonical names: exact for nodes, and with the final segment + case-folded for edges, as today. +- **Table keys.** + - The persisted `__manifest` table key becomes `node:crm::Contact`. + - Every reader strips the `node:`/`edge:` prefix and treats the rest as opaque. + - The key stays one component of the `lance-namespace` table id; + `table_id_to_key` keeps refusing multi-component ids. +- **Parsers to fix.** + - The CLI `embed --select Type:field=value` selector, which splits on the + first `:` (`crates/omnigraph-cli/src/embed.rs`). + - The GQT `expect plan` claim parser, which does the same + (`crates/omnigraph-gqt-core/src/plan.rs`). + - The GQT `expect shape` type-name check (`crates/omnigraph-gqt-core/src/shape.rs`). +- **Policy.** + - A namespace confers no permission, as `implements` confers none under + Polymorphic types. Otherwise a `schema_apply` holder could widen an + actor's view by moving a type into a namespace. + - Per-type visibility + ([Actor views and subgraph sharing](https://github.com/ModernRelay/omnigraph/pull/951)) + stays decided per concrete type. A rule written over a namespace pattern + (`crm::*`) would be sugar, expanded to `StableTypeId`s when the policy is + applied, and schema plan would report any move that changes the set. + - Type names stay Cedar entity ids (strings) and never become Cedar entity + types, because Cedar already uses `::` for those (`Omnigraph::…`). + +### Moving existing types + +```pg +namespace crm { + node Contact @rename_from("Contact") { … } // was the root Contact +} +``` + +- **What a move does.** + - It plans as `RenameType`, and apply publishes a metadata-only `RenameTable` + in the same manifest commit. + - Data paths (`nodes/{type_id:016x}-{incarnation:016x}`), Lance field names + and index names do not move. + - Old commits keep `node:Contact` in their immutable history extents. + Historical reads rebind by identity (`bind_catalog_aliases`), so they keep + working. +- **Prerequisite: refuse dropping populated types.** A move without + `@rename_from` plans today as `DropType` plus `AddType`, and a populated drop + is accepted silently (issue #984). With namespaces that mistake becomes easy, + so populated drops must fail closed, owned by #984, before namespaces ship. +- **Interfaces wait.** Interfaces cannot be renamed today, so existing + interfaces cannot move until + [Polymorphic types](https://github.com/ModernRelay/omnigraph/pull/884) lands + interface `@rename_from`. New interfaces may be declared in namespaces from + the start. + +## Design + +### Multi-file: compose before hashing + +Downstream of the loaders, everything identifies a schema by +`sha256(source bytes)`: +- the contract digest's `source_hash`; +- the cluster resource digest `schema.`; +- the invariant `resources[schema] == contract.source_hash` in + `crates/omnigraph-cluster/src/deployment.rs`; +- drift detection in `observe` and at server boot. + +Composing the files into one canonical text before that hash keeps every one of +those consumers unchanged. Keeping file boundaries after apply would instead need +a new `SchemaContractRow` field, a storage stamp bump with an upgrade route, a +new archive magic for `__history/schemas/` and a new `schema_content_hash`. File +layout is presentation, so this RFC does not pay that cost. The user's repository +remains the authority for files; the graph remains the authority for the accepted +text. + +**One parse path.** The loader parses the composed text with the same function +that validates a stored contract on every open (`validate_schema_contract_row` +in `crates/omnigraph/src/db/schema_state.rs`). Applied and reopened, it +therefore gives the same answer. A source map from composed byte ranges to +`(file, line)` turns spans into file positions, the approach Gel's CLI takes. + +Today only syntax errors carry a span (`ParseDiagnostic` in +`crates/omnigraph-compiler/src/error.rs`). Declarations gain a span, skipped by +serde, so catalog and identity errors can name the declaration's file. + +**One loader.** A single compiler-crate function resolves a `SchemaDecl` into +the composed text and its source map. The CLI (`init`, `schema plan`, +`schema apply`, `lint`) and the cluster config share it. +- `GraphConfig.schema` becomes an untagged `SchemaDecl { Path, Many }`, + mirroring `QueriesDecl`. +- The single-path form serializes as today, so `config_digest` and + `validate_bundle`'s canonical-serialization check are unchanged. +- Bundles keep carrying one composed source, so remote `cluster plan` and + `cluster apply` need no wire change. + +### Namespaces: part of the name, not of identity + +Both places a namespace could live were weighed against the code: + +- **As a separate IR field (rejected).** + - The resolver matches accepted declarations by `(kind, name)` and then by + hint (`resolve` in `crates/omnigraph-compiler/src/catalog/schema_ir.rs`). + It would match `crm::Contact` against `Contact` by name and keep the id, + but the planner would see a change it cannot classify and refuse it + (`schema_plan.rs`, the unclassified-change refusal). + - IR uniqueness and sort order are by name. Two namespaces declaring `Contact` + would need new uniqueness and sort keys, a namespace copy in every + `TypeRefIR`, `PropertyRefIR` and `SystemFieldRefIR`, a new planner step and + a feature name. + - Every wire surface would still need one string per type, so the field would + add structure that no consumer reads. +- **As part of the canonical name (chosen).** + - `NodeIR.name` holds `crm::Contact`. + - Uniqueness, sort order, references, the manifest's `table_key`, the rename + machinery and every wire field work as they do now. + - The namespace of a name is everything before its last `::`, so it is + derived, never stored. A namespace has no object of its own: it exists while + a declaration names it, and there is nothing to create or drop. Gel's + module-rename and empty-module problems have no counterpart. + +**Feature name.** The IR's `required_features` derives `namespaces` whenever a +declaration name contains `::`. An older binary then refuses the graph by +unknown feature name, a clear reason, instead of failing to re-parse the stored +source or reporting a hash mismatch. A graph that declares no namespace derives +no feature, so its IR and hash stay byte-identical, the same posture as +`system-columns` and `edge-keys`. + +**Delimiter `::`.** +- `.` is property access in `.gq` (`$p.name`) and the nested-field path + separator in Lance, which also refuses `.` in top-level field names + (`lance-core` `Schema::validate`). Protobuf's dotted partial names produced + its first-component lookup trap, and Postgres needs `OPERATOR(schema.op)` to + disambiguate. +- `/` collides with `//` comments and with URL paths. +- `__` is the system-column prefix. +- `:` alone already marks bindings, parameters, property matches and `node:` + load targets. +- `::` appears in neither grammar today. It is what Gel, the closest prior art, + uses, as do Rust and Cedar, and `node:crm::Contact` still parses with every + existing `strip_prefix`. + +**Resolution without a search path.** Postgres' `search_path` lets a type added +early on the path capture names that resolved elsewhere (CVE-2018-1058). GraphQL +declined namespaces partly because "adding a field can break queries". Gel +resolves within one active module and never searches others. + +The rule here goes one step stricter. A name either resolves to exactly one +declaration among the root and the `use`d namespaces, or it is an error. Every +name is bound at compile time to a `StableTypeId` in the typed IR, so a later +schema change can produce a loud plan-time error but never a different binding. + +**Files and namespaces stay independent.** Protobuf made packages independent of +files and the ecosystem pushed them back together with lint (Buf +`PACKAGE_DIRECTORY_MATCH`), while Go and TypeScript fix "directory or file = +module". Here, names are an API that is persisted in table keys and history, so +tying them to file layout would turn moving a file into a migration. Gel shows +that declared namespaces with concatenated files hold up. An optional lint can +warn when a namespace spans directories or a file holds several namespaces. + +Block syntax is also what keeps concatenation safe. A file-scoped header +(`namespace crm;` at the top) would leak into the next file once files are +joined, unless the loader rewrote the text, and then the stored source would no +longer be the files' bytes. + +### Interaction with polymorphic types + +[Polymorphic types](https://github.com/ModernRelay/omnigraph/pull/884) (draft) +makes interfaces and inline unions bindable, traversable and usable as edge +endpoints. The two RFCs are independent; neither requires the other to ship. +Where they meet, this is how they compose. + +- **One rule for type references.** Every place that RFC accepts a type name + takes a qualified one, resolved by the rules above: + - edge endpoints and unions (`edge HeldBy: Holding -> crm::Contact | hr::Employee`); + - `implements` lists and interface inheritance (`interface crm::Party implements core::Entity`); + - abstract bindings and unions in queries (`$x: core::Entity`, `$x: crm::Contact | Note`); + - type tests (`$x is crm::Contact`); + - typed endpoints (`to: crm::Contact($id)`); + - edge alternations (`(crm::ownedBy | hr::ownedBy)`). + + In a union, each member resolves on its own. The grammar has one + `type_ref = (segment "::")* type_name` rule that both RFCs use, so they never + grow two spellings. +- **Interfaces span namespaces.** An interface's member set is its concrete + implementors in any namespace. `$x: core::Named` reads `crm::Contact` and + `hr::Employee` alike. Namespaces never filter membership; a query narrows + with type tests or a union. +- **Moves keep ids; names change.** + - Moving a type is a rename, so it keeps its `StableTypeId`, the type half of + every packed endpoint under the integer identity format that RFC builds on. + No edge row changes. + - `$x.@type`, `$e.@src_type` and `$e.@dst_type` return the new qualified + name, while `$x.@type_id` stays the same. Clients that cache or route by + type should key on `@type_id`, as that RFC advises. + - Ordering ties break on the packed reference, not on the name, so a move + does not reorder results. +- **Interface moves wait for interface renames.** Moving an existing + interface into a namespace needs interface `@rename_from`, which that RFC + adds. New interfaces may be declared in namespaces from the start. +- **Registry pinning covers re-resolution.** That RFC records, for each stored + query, every binding name's kind and `StableTypeId`, and refuses when a name + later resolves to a different kind or id. Together with the + exactly-one-match rule: + - A move keeps every stored query that still resolves the name, for example + through `use crm;`, because the id is unchanged. + - A query that named the old unqualified `Contact` without `use crm;` fails + to typecheck at plan time. + - A new root `Contact` added in the same apply makes `use crm;` queries + ambiguous, and registry pinning refuses queries whose `Contact` would now + bind a different id. + + No path changes a stored query's meaning silently. +- **Released names.** That RFC refuses reusing a name released by a rename or + drop as a different kind in the same apply. A move into a namespace releases + the root name, so the same rule applies to it. +- **Catalog route.** The structured route that RFC adds lists canonical names + next to each `type_id`. A shell groups types by namespace by splitting on the + last `::`; the route needs no namespace object, because namespaces have none. +- **Change-feed filters.** That RFC accepts an interface name as a feed or + export filter, expands it per commit, and refuses with + `ChangeScopeMembershipChanged` when membership changes. A namespace filter + (`type=crm::*`) would reuse that mechanism unchanged. It is left as an + unresolved question here. +- **Policy.** Visibility stays per concrete type for both. Neither + `implements` nor a namespace grants or removes it (see Wire and tools). + +### Lance + +Type names never reach Lance: +- Data paths are id-derived (`table_path_for_identity` in + `crates/omnigraph-catalog/src/lib.rs`). +- Lance field names are property names plus system columns. +- Index names default to `{column}_idx` (OmniGraph passes `name: None`). + +Namespaces are type-level, so no Lance schema, file or index changes, and +property names keep refusing `.`. + +Lance's own namespace specification is a hierarchical catalog of tables: +- identifiers are lists of names joined by `$`; +- its directory layout names table directories `_`; +- it offers per-table and batch table operations. + +Using it for type namespaces would derive storage locations from names, against +invariant 6. It would also put a second catalog beside `__manifest`, which the +deny-list rejects. Its plausible fit is graph discovery across a cluster, as a +derived read-only view, which belongs to the cross-graph reference work, not to +this RFC. OmniGraph keeps using `lance-namespace` only for its request models, +with a one-component id. + +## Invariants + +- **2 and 4, one publication.** A move is a set of `RenameType` steps that one + schema apply publishes in one `__manifest` CAS, together with its + `RenameTable` rows. +- **6, identity from stable ids.** A namespace is part of a name. Moves are + renames that must be declared; nothing infers identity from a name, a + namespace, a file or a path. +- **8, loud failures.** Ambiguous names, unknown namespaces, duplicate + declarations across files, reserved names and populated drops (#984) are + typed refusals. +- **9, typed semantics.** `use` and qualified names are AST and IR structures. + Resolution finishes in the compiler, and execution sees only type ids. +- **12, one source of truth.** No namespace registry exists, and the composed + source is the only stored schema text. File layout is not persisted; the + repository owns it. +- **13, evidence.** See below. + +## Compatibility and reversibility + +- **Multi-file schemas.** There is no storage stamp, IR, ledger or HTTP change. + A single-file schema keeps byte-identical source and digests, and + `cluster.yaml` gains an untagged alternative, a minor change under the + compatibility-surfaces RFC. +- **Namespaces.** + - PG and GQ grammars gain a block, a `use` line and qualified names, all minor + changes. + - The `namespaces` IR feature makes older binaries refuse a graph that uses it. + - Wire shapes do not change, but values may contain `::`. A client that + validates type names against `^[A-Z]\w*$` must widen its pattern. + `openapi.json` declares no pattern for these fields. +- **Reverting.** Moving types back to the root is a rename. Removing the feature + requires renaming every type back first, and history keeps the qualified keys + it recorded. + +## Alternatives + +| Alternative | Reason not selected | +|---|---| +| A prefix convention (`Crm_Contact`) | Works today with no engine change, and stays the baseline until a need appears. It gives no query scope, no policy scope, and edge names still collide across the graph. | +| `import "./x.pg"` between files | With declared namespaces, imports add cycle and order rules and nothing else. Protobuf shows file imports leak file structure to callers (`import public`, Buf `IMPORT_NO_PUBLIC`). Composition is declared once, in configuration. | +| File or directory = namespace (Go, TypeScript) | Ties persisted names to file layout: moving or renaming a file becomes a schema migration, and templates copied into different folders get different type names. | +| File-scoped `namespace crm;` header | Breaks joining files by concatenation, unless the loader rewrites the text, so the stored source would no longer be the files' bytes. Possible later as sugar the composer expands. | +| Namespace as a separate IR field | New uniqueness and sort keys, a namespace copy in every IR reference type and a new planner step, while every wire surface still needs one string. | +| Search path, or unqualified names that are unique anywhere | Shadowing and capture (CVE-2018-1058). Adding a type changes or breaks existing queries silently. | +| Lance namespaces for type namespaces | Derives storage from names and adds a second catalog beside `__manifest`. | +| Persist file boundaries in the contract | Needs a stamp bump, an upgrade route and a new history archive format, for display value only. Deferred until a user needs `schema show` to return files. | + +## Evidence and tests + +Facts this design rests on, checked at `fe69c039`: +- **The manifest key is a renameable alias.** `table_key` is a `Utf8` field of + the `__manifest` `table` row and of `__history` extents + (`crates/omnigraph-catalog/src/record.rs`, `history.rs`). A rename rewrites + it through `RenameTable` without moving the data path + (`crates/omnigraph-catalog/src/lib.rs`, `publisher.rs`). +- **Change-feed cursors.** They carry a digest of the caller's type-name filter + (`crates/omnigraph/src/changes/token.rs`). A cursor that filtered on + `Contact` does not validate for a filter on `crm::Contact`, which is already + true for any rename. +- **Graph-index stamps.** Its persisted stamps carry type and edge names + (`graph_index/persist.rs`). A mismatch rebuilds the derived index. +- **Declaration order.** It does not matter within a file: interfaces are + collected before resolution, and edge endpoints are checked after every node + is known. Joining files therefore cannot change meaning. + +Tests extend the existing owners in [testing](../dev/testing.md): +- **Compiler.** + - Schema parser tests: blocks, merged blocks, reserved and malformed paths, + and qualified references. + - Catalog tests: edge folding per namespace, and kind separation. + - `schema_ir` tests: the `namespaces` feature, byte-identical IR without it, + and a move into a namespace producing `RenameType`. + - Query parser and typecheck tests: `use`, ambiguity, unknown namespaces, and + the binding/edge choice by final segment. +- **Cluster.** Config tests that mirror the `QueriesDecl` tests: directory, + list, symlink and `..` refusal, caps, an empty directory, sorted order, and an + unchanged digest for the single-path form. +- **CLI.** + - `--schema ` for every command that takes a schema. + - Errors with `file:line`. + - The `embed --select` selector with a qualified type. +- **Engine.** + - `tests/schema_apply.rs`: a move keeps the table identity, data files, row + ids and indexes, and publishes once. + - Historical reads across a move. + - Export and load round-trip with qualified names. + - A change-feed cursor across a rename. +- **GQT.** Cases with a namespaced schema and queries, plus `expect shape` and + `expect plan` claims that spell qualified names. +- **Composition.** + - Composing the files gives the same schema shape as each file parsed alone and + merged, for every order of the files. + - Recompiling the stored composed text gives the shape the apply produced. + +## Rollout + +1. **Multi-file schemas.** + - The shared loader and source map, `SchemaDecl` in `cluster.yaml`, and + `--schema `. + - No format change. +2. **Fail-closed populated drops (#984).** A prerequisite for step 3. +3. **Namespaces.** + - `.pg` blocks and the `namespaces` IR feature. + - `.gq` qualified names and `use`. + - Qualified names on every wire surface. + - The selector and GQT parser fixes. +4. **Moving interfaces.** Follows interface `@rename_from` in Polymorphic + types. Abstract bindings, unions and polymorphic endpoints take qualified + names whenever both RFCs have shipped, in either order. +5. **Later, each on demand.** + - Namespace-scoped per-type policy rules. + - A layout lint. + - A helper that writes `@rename_from` hints for a bulk move. + - A stamp-gated `schema show --files`. + +## Unresolved questions + +- **Where `use` binds.** It applies per query text here. Gel binds per + statement (`with module`). Is a per-query clause better for stored queries + that share a file? +- **File banners.** They make `schema show` readable and let tools split the + stored text, but they also become part of the digest of every multi-file graph. + The alternative is no banners, with source maps on the client only, as Gel + does. +- **Recursive discovery.** It matches `queries:` (top level only) here. Should + both recurse together? +- **Bulk moves.** Are per-type `@rename_from` hints enough, or is a block-level + form wanted, such as Terraform's `moved` blocks? +- **Case of namespace segments.** Lowercase keeps the binding/edge choice + trivial. Is that too strict for organizations' naming? +- **Namespace filters.** Should change-feed and export filters accept + `crm::*`, reusing the per-commit expansion and membership refusal that + Polymorphic types defines for interface filters? + +## Decision log + +2026-10-15: drafted from a code investigation at `fe69c039` covering: +- the compiler grammar, parser, catalog, SchemaIR identity and schema plan; +- every surface where a type name crosses a boundary; +- the schema loading and contract path; +- `__manifest` and `__history` persistence, and Lance 11.0.0 (field names, + index names, the namespace specification and crate). + +Prior art read: Gel modules, Prisma multi-file schemas, PostgreSQL schemas and +`search_path`, Protocol Buffers and Buf, GraphQL, ISO GQL, Neo4j graph types, +Rust, TypeScript and Go modules, Terraform `moved` blocks, dbt and LinkML. + +2026-10-16: added the interaction with Polymorphic types (draft, PR #884): +one type-reference rule for unions, `implements`, type tests and typed +endpoints; interfaces spanning namespaces; moves keeping `StableTypeId` and +`@type_id`; registry pinning as the guard against re-resolution; namespaces +conferring no visibility. diff --git a/docs/rfcs/README.md b/docs/rfcs/README.md index c482e1f95..308d9cf76 100644 --- a/docs/rfcs/README.md +++ b/docs/rfcs/README.md @@ -248,4 +248,5 @@ then dated RFCs by date. | [2026-10-10](2026-10-10-merge-base-block-registry.md) | Block registry for merge base lookup | maintainer | draft | in-progress | | [2026-10-11](2026-10-11-unattached-graph-branches.md) | Unattached graph branches | maintainer | draft | not-started | | [2026-10-15](2026-10-15-bulk-load-as-a-query-plan.md) | Bulk load as a query plan | maintainer | draft | not-started | +| [2026-10-15](2026-10-15-schema-namespaces-and-multi-file-schemas.md) | Schema namespaces and multi-file schemas | maintainer | draft | not-started | | [2026-10-16](2026-10-16-engine-catalogue.md) | Engine catalogue | maintainer | draft | in-progress | From 9e446fd91a001a577487c6466416b7be83a6d573 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Fri, 16 Oct 2026 12:39:25 -0700 Subject: [PATCH 2/5] rfc: state that use and namespace add no reserved words --- ...schema-namespaces-and-multi-file-schemas.md | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md b/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md index 130e868db..dafa4b0f7 100644 --- a/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md +++ b/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md @@ -157,6 +157,10 @@ node Note { body: String } // the root namespace: today's schemas namespace, where every existing schema already sits. - **Reserved names.** `omnigraph`, `std` and segments that start with `_` are reserved for future built-ins. +- **`namespace` is a keyword only at the top level.** It starts a declaration + only where `node`, `edge` and `interface` can, so a property named + `namespace` stays valid. Type names start with an uppercase letter, so no + existing type name can collide with it. - **Kinds.** Node and edge names stay separate per kind, as today, inside each namespace. The interface rule of [Polymorphic types](https://github.com/ModernRelay/omnigraph/pull/884) (a new @@ -194,6 +198,14 @@ query open_deals() { - **`use` lines.** They come at the top of a query text and apply to every query in it. A stored query's payload is its whole file, so the lines are stored with it. +- **`use` is not a reserved word.** + - It is recognized only in the leading-statement position where `set …;` + lines already sit (`query_file` in `query.pest`), before the first query + declaration. + - It does not join the reserved words that edge, property and alias names + exclude (`and`, `or`, `not`, `is`, `null`, `in`). An edge named `Use`, a + property `use` and an alias `as use` therefore keep working, and no + existing query text changes meaning. - **Name resolution.** - A qualified name resolves exactly. - An unqualified name must match exactly one type among the root namespace and @@ -497,7 +509,8 @@ with a one-component id. compatibility-surfaces RFC. - **Namespaces.** - PG and GQ grammars gain a block, a `use` line and qualified names, all minor - changes. + changes. Neither adds a reserved word, so every accepted schema and query + text parses as before. - The `namespaces` IR feature makes older binaries refuse a graph that uses it. - Wire shapes do not change, but values may contain `::`. A client that validates type names against `^[A-Z]\w*$` must widen its pattern. @@ -623,4 +636,5 @@ Rust, TypeScript and Go modules, Terraform `moved` blocks, dbt and LinkML. one type-reference rule for unions, `implements`, type tests and typed endpoints; interfaces spanning namespaces; moves keeping `StableTypeId` and `@type_id`; registry pinning as the guard against re-resolution; namespaces -conferring no visibility. +conferring no visibility. Also stated that `use` and `namespace` are positional +keywords, not reserved words, so existing names keep parsing. From 2e672c0cadca9633c9228f05bb90f0b4d38e2379 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Sun, 18 Oct 2026 11:08:59 -0700 Subject: [PATCH 3/5] rfc: schema namespaces, align with polymorphic types and integer identity Registry pinning covers every resolved type and edge name, and a same-kind re-add in one apply is caught by pinning alone; ad-hoc text is not protected. A move respells integer-identity handles, which parse at their last colon. Adds the GQT load-plan and type/cast parsers to the fix list, view-catalog name resolution from Actor views, and the block form of a namespaced interface declaration. --- ...chema-namespaces-and-multi-file-schemas.md | 46 +++++++++++++++---- 1 file changed, 37 insertions(+), 9 deletions(-) diff --git a/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md b/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md index dafa4b0f7..2fdf94459 100644 --- a/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md +++ b/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md @@ -7,7 +7,7 @@ implementation: not-started authors: - ragnorc created: 2026-10-15 -updated: 2026-10-16 +updated: 2026-10-18 discussion: null supersedes: [] superseded_by: [] @@ -248,16 +248,24 @@ query open_deals() { - **Parsers to fix.** - The CLI `embed --select Type:field=value` selector, which splits on the first `:` (`crates/omnigraph-cli/src/embed.rs`). - - The GQT `expect plan` claim parser, which does the same - (`crates/omnigraph-gqt-core/src/plan.rs`). + - The GQT `expect plan` scan and expand selectors, which do the same, and its + `type` and `cast` claims, which split at the last `:` and so cut + `crm::Contact` in two (`crates/omnigraph-gqt-core/src/plan.rs`). - The GQT `expect shape` type-name check (`crates/omnigraph-gqt-core/src/shape.rs`). + - The GQT load-plan table keys, whose `type_name` check accepts only letters, + digits and `_` (`crates/omnigraph-gqt-core/src/load_plan.rs`). + - Under [Integer node identity](https://github.com/ModernRelay/omnigraph/pull/884), + the `Type:number` handle parser, which must split at the last `:`. - **Policy.** - A namespace confers no permission, as `implements` confers none under Polymorphic types. Otherwise a `schema_apply` holder could widen an actor's view by moving a type into a namespace. - Per-type visibility ([Actor views and subgraph sharing](https://github.com/ModernRelay/omnigraph/pull/951)) - stays decided per concrete type. A rule written over a namespace pattern + stays decided per concrete type. + - Name resolution for a restricted actor runs on its view catalog, as Actor + views specifies: an ambiguous name's candidate list and the + unknown-namespace error never name a hidden type or namespace. A rule written over a namespace pattern (`crm::*`) would be sugar, expanded to `StableTypeId`s when the policy is applied, and schema plan would report any move that changes the set. - Type names stay Cedar entity ids (strings) and never become Cedar entity @@ -408,7 +416,8 @@ Where they meet, this is how they compose. - **One rule for type references.** Every place that RFC accepts a type name takes a qualified one, resolved by the rules above: - edge endpoints and unions (`edge HeldBy: Holding -> crm::Contact | hr::Employee`); - - `implements` lists and interface inheritance (`interface crm::Party implements core::Entity`); + - `implements` lists and interface inheritance + (`namespace crm { interface Party implements core::Entity { … } }`); - abstract bindings and unions in queries (`$x: core::Entity`, `$x: crm::Contact | Note`); - type tests (`$x is crm::Contact`); - typed endpoints (`to: crm::Contact($id)`); @@ -428,15 +437,24 @@ Where they meet, this is how they compose. - `$x.@type`, `$e.@src_type` and `$e.@dst_type` return the new qualified name, while `$x.@type_id` stays the same. Clients that cache or route by type should key on `@type_id`, as that RFC advises. + - A move also respells integer-identity handles. The `@id` of every unkeyed + node and every edge of the moved type, and `@src` and `@dst` where they + render an unkeyed endpoint, change from `Note:10` to `crm::Note:10`, and + the old spelling is not an alias. A client that keeps handles re-reads + them after a move. - Ordering ties break on the packed reference, not on the name, so a move does not reorder results. - **Interface moves wait for interface renames.** Moving an existing interface into a namespace needs interface `@rename_from`, which that RFC adds. New interfaces may be declared in namespaces from the start. - **Registry pinning covers re-resolution.** That RFC records, for each stored - query, every binding name's kind and `StableTypeId`, and refuses when a name - later resolves to a different kind or id. Together with the - exactly-one-match rule: + query, the kind and `StableTypeId` of every type and edge name it resolves + (bindings, traversal edges, type tests, typed endpoints, and mutation and + load targets), and refuses when one later resolves to a different kind or + id. Its migration table refuses reusing a released name in one apply only + as a different kind, so a same-kind re-add, such as moving `OwnedBy` into + `crm` and adding a new root `OwnedBy`, is caught by pinning alone. Together + with the exactly-one-match rule: - A move keeps every stored query that still resolves the name, for example through `use crm;`, because the id is unchanged. - A query that named the old unqualified `Contact` without `use crm;` fails @@ -445,7 +463,9 @@ Where they meet, this is how they compose. ambiguous, and registry pinning refuses queries whose `Contact` would now bind a different id. - No path changes a stored query's meaning silently. + No path changes a stored query's meaning silently. Ad-hoc query text is + resolved again on every run, so the same-kind re-add above retargets it; + only stored queries are protected. - **Released names.** That RFC refuses reusing a name released by a rename or drop as a different kind in the same apply. A move into a namespace releases the root name, so the same rule applies to it. @@ -638,3 +658,11 @@ endpoints; interfaces spanning namespaces; moves keeping `StableTypeId` and `@type_id`; registry pinning as the guard against re-resolution; namespaces conferring no visibility. Also stated that `use` and `namespace` are positional keywords, not reserved words, so existing names keep parsing. + +2026-10-18: aligned with Polymorphic types and Integer node identity (PR +#884): registry pinning covers every resolved type and edge name, and a +same-kind re-add in one apply is caught by pinning alone; ad-hoc text is not +protected. A move respells integer-identity handles, which parse at their +last `:`. Added the GQT load-plan and `type`/`cast` parsers to the fix list, +view-catalog name resolution from Actor views, and the block form of a +namespaced interface declaration. From 7565f5ffb19d3150ae3c1857bf43b7a28592f492 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Sun, 18 Oct 2026 11:26:35 -0700 Subject: [PATCH 4/5] rfc: answer review of schema namespaces and multi-file schemas Links issue #838 and records its disposition, replaces the #984 prerequisite with a narrow refusal of a probable move without @rename_from, restates name-resolution safety on the mechanisms the query IR actually has, states the list-form skew rule, fixes one banner path rule and the composed size bound, and adds refusal and compatibility tests. --- ...chema-namespaces-and-multi-file-schemas.md | 268 ++++++++++++++---- 1 file changed, 214 insertions(+), 54 deletions(-) diff --git a/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md b/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md index 2fdf94459..4c6e719f0 100644 --- a/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md +++ b/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md @@ -8,7 +8,7 @@ authors: - ragnorc created: 2026-10-15 updated: 2026-10-18 -discussion: null +discussion: https://github.com/ModernRelay/omnigraph/issues/838 supersedes: [] superseded_by: [] blocked_on: [] @@ -27,8 +27,12 @@ never decides a type's name. Composition is declared once, in `cluster.yaml` or on the command line, and the loader joins the files in a fixed order into one canonical source text. Plan, apply, the stored contract, the contract digest and the cluster ledger all see that one text, as they see one file today. No storage -format, IR, ledger or wire shape changes, and a schema given as one file keeps -byte-identical source and digests. Errors name the file and line. +format, IR or ledger field changes. A schema given as one file keeps +byte-identical source and digests, and a single directory path keeps the shape +of every bundle a server receives. The list form is new `cluster.yaml` syntax +that servers without this change refuse (see Compatibility). Errors name the +file and line. This replaces the `import` design that issue #838 proposed (see +[Relation to issue #838](#relation-to-issue-838)). **Namespaces.** A declaration may sit in a namespace block, `namespace crm { node Contact { … } }`, which gives it the canonical name @@ -67,10 +71,13 @@ id. - `GraphConfig.schema` is one `PathBuf` (`crates/omnigraph-cluster/src/types.rs`). - Queries can already be a file, a directory or a list (`QueriesDecl` in `crates/omnigraph-cluster/src/config.rs`). -- **Names are flat.** `type_name = @{ ASCII_ALPHA_UPPER ~ (ASCII_ALPHANUMERIC | "_")* }` - in both grammars, so two templates that each declare `Event` or `OwnedBy` - cannot coexist. Edge names are also unique after case folding across the - whole graph (`normalize_edge_name` in `catalog/mod.rs`). +- **Names are flat.** The `.pg` grammar has + `type_name = @{ ASCII_ALPHA_UPPER ~ (ASCII_ALPHANUMERIC | "_")* }`. The `.gq` + rule is the same, plus the `@` prefix of the + [Engine catalogue](2026-10-16-engine-catalogue.md) types. Two templates that + each declare `Event` or `OwnedBy` therefore cannot coexist. Edge names are + also unique after case folding across the whole graph (`normalize_edge_name` + in `catalog/mod.rs`). - **Workarounds don't scale.** A prefix convention such as `Crm_Contact` works today and costs nothing. It gives no scope to a query, to a future per-type policy rule, or to an author reading a 400-type schema. @@ -79,6 +86,31 @@ An issue is not enough: both halves add a grammar surface, and namespaces add a spelling that is persisted in `__manifest` table keys and in immutable `__history` extents. +### Relation to issue #838 + +Issue #838 (accepted) asks for `import` statements in `.pg` files. This RFC +replaces that design: composition is declared once in configuration, and +namespaces, not imports, give names a scope. + +- **Kept from #838:** + - one schema split across files; + - a core file shared by several graphs of one cluster, as a list entry; + - errors that give a file and line; + - a server that never opens caller paths, since the bundle carries the + composed source. +- **Dropped:** + - named, selective imports; + - a `.pg` file that can be checked alone. An editor or a one-file `lint` + cannot resolve `core::Named` in `crm.pg` without the declaration that + composes the files. `omnigraph lint --schema ` checks a directory; + - sharing a core file across clusters, because `check_config_path` refuses + `..`. +- **Decided differently:** #838 includes a definition reached through several + paths once. Without imports there is no diamond: a file reaches the + composition only by being named. A file named twice, as in + `[schema/core.pg, schema/]`, is an error naming both entries, not a silent + deduplication. + ## User and operational behavior ### Multi-file schemas @@ -95,25 +127,51 @@ graphs: omnigraph schema plan --schema schema/ graph.omni ``` -Discovery copies the `queries:` rules exactly, so one rule covers both. +Discovery copies the `queries:` rules with one difference, so one rule covers +both. - **What a path may name.** A single path may be a file or a directory, and a list may mix both. - A directory contributes its top-level `*.pg` files, with no recursion. - Each directory's files are sorted by path. A list keeps its own order. + - A file named twice, directly or through a directory, is an error naming + both entries. - **Refusals and caps.** - Symlinks and `..` components are refused (`check_config_path`). - - Reads go through `SourceCapture`, with the same per-file and total byte caps - and the same discovery cap as queries. - - A directory with no `.pg` file is an error, not a warning, because a graph - has to have a schema. + - Reads go through `SourceCapture`, with the same per-file cap and the same + discovery cap as queries. + - The composed source, banners included, is one bundle source, and + `validate_bundle` refuses any source above `MAX_CONFIG_SOURCE_BYTES` + (1 MiB). A schema therefore stays within 1 MiB, as one file does today; + the 8 MiB total cap does not raise that limit. + - **The one difference:** a directory with no `.pg` file is an error, while + an empty query directory is the warning `query_dir_empty`. A graph has to + have a schema. - **Composition.** - - The composed source is each file's bytes in order. Each file is preceded by - a banner line `// file: ` and ends with exactly one newline. - - `` is relative to the declared directory, or to the `cluster.yaml` - directory for list entries. - - A schema declared as one file is composed as that file's bytes with no - banner, so every existing graph keeps its digest. + - The composed source is each file's bytes in order, each ending with exactly + one newline. + - With two or more files, each file is preceded by a banner line + `// file: `. With exactly one file there is no banner, whatever the + form of the declaration. Every existing graph therefore keeps its digest, + and so does a directory that holds one file. + - `` is the path discovery already records for a query file, + `declared.join(file_name)` (`config.rs`), relative to the `cluster.yaml` + directory and written with `/`. One file has one banner in every form: + `schema: schema/`, `schema: [schema/]` and `[schema/a.pg, schema/b.pg]` + compose the same bytes. + - On the command line (`--schema `), `` is the directory's final + component joined with the file name. The same files then compose the same + bytes from any working directory, and the same bytes as a cluster that + declares that directory beside its `cluster.yaml`. A graph is applied + either standalone or through its cluster, never both (`schema apply` + refuses a cluster graph), so the two never compete for one contract. + - Banners become part of `source_hash` for every multi-file graph, so their + format is a compatibility surface: changing it later gives each such graph + a new contract with no schema steps. +- **Validate and plan output.** `ResourceSummary.path` in `cluster validate` + and `plan --json` holds the declared path for a single path. For a list it + is absent, and a new optional `paths` field lists the composed files in + order. Both are additive to that output. - **What sees the composed text.** It is what `schema plan`, `schema apply`, the contract's `source`, `source_hash`, the `schema.` resource digest and `schema show` see. @@ -148,7 +206,9 @@ node Note { body: String } // the root namespace: today's schemas - **Paths.** A namespace path is one or more lowercase segments `[a-z][a-z0-9_]*` joined by `::`, as in `namespace acme::crm { … }`. The path - is flat: `acme::crm` is one namespace, and `acme` gains nothing from it. + is flat: `acme::crm` is one namespace, and `acme` gains nothing from it. A + segment may not be one of the `.gq` reserved words (`and`, `or`, `not`, `is`, + `null`, `in`). - **Blocks.** - Blocks do not nest. - Blocks with the same path merge, across files too. @@ -219,6 +279,12 @@ query open_deals() { names. - **Telling a binding from an edge.** The parser decides by the case of the final segment, as it decides by the case of the whole name today. +- **Edges named by reserved words.** Today such an edge is written as a string, + `$i "in" $b` (`query.pest`). + - The string may hold a qualified name: `$i "crm::in" $b`. + - The bare form `$i crm::in $b` is accepted too. After `::` the final segment + is never read as a reserved word, because the `::` already marks a name. + - Namespace segments themselves are never reserved words (see Paths). - **Engine catalogue types.** The built-in types of the [Engine catalogue](2026-10-16-engine-catalogue.md) (`@Operator`, `@Setting` and the rest) keep their `@` spelling and belong to no namespace. A qualified @@ -264,12 +330,15 @@ query open_deals() { ([Actor views and subgraph sharing](https://github.com/ModernRelay/omnigraph/pull/951)) stays decided per concrete type. - Name resolution for a restricted actor runs on its view catalog, as Actor - views specifies: an ambiguous name's candidate list and the - unknown-namespace error never name a hidden type or namespace. A rule written over a namespace pattern - (`crm::*`) would be sugar, expanded to `StableTypeId`s when the policy is - applied, and schema plan would report any move that changes the set. - - Type names stay Cedar entity ids (strings) and never become Cedar entity - types, because Cedar already uses `::` for those (`Omnigraph::…`). + views specifies. An ambiguous name's candidate list and the + unknown-namespace error never name a hidden type or namespace. + - A rule written over a namespace pattern (`crm::*`) would be sugar, expanded + to `StableTypeId`s when the policy is applied. Schema plan would report any + move that changes the set. + - No type name is a Cedar entity today: the policy engine builds only actor, + action, group, graph, server and cluster entities. If per-type policy lands, + type names become Cedar entity ids, never entity types, because Cedar + already uses `::` for its entity types (`Omnigraph::Actor`). ### Moving existing types @@ -287,10 +356,33 @@ namespace crm { - Old commits keep `node:Contact` in their immutable history extents. Historical reads rebind by identity (`bind_catalog_aliases`), so they keep working. -- **Prerequisite: refuse dropping populated types.** A move without +- **Hints are canonical names.** A `@rename_from` hint is always the canonical + name of an accepted declaration. The in-namespace reference rule never + applies to it: + - `@rename_from("Contact")` always means the root `Contact`; + - `@rename_from("crm::Contact")` moves a type out of `crm`, or to another + namespace. + + The resolver already looks a hint up as an accepted name. Its + `IgnoredExactMatchRenameHint` diagnostic still applies when the new + canonical name already exists. +- **A probable move without a hint fails closed.** A move without `@rename_from` plans today as `DropType` plus `AddType`, and a populated drop - is accepted silently (issue #984). With namespaces that mistake becomes easy, - so populated drops must fail closed, owned by #984, before namespaces ship. + is accepted silently (issue #984, currently undecided). Namespaces make that + mistake easy, so this RFC owns a narrow refusal: + - **What is refused.** A plan that drops a populated declaration and adds one + of the same kind with the same final segment in another namespace (root + `X` to `crm::X`, `crm::X` to `hr::X`, or back to the root), with no + `@rename_from`. + - **The refusal.** A new migration code, allocated at implementation, whose + message names the two declarations and suggests `@rename_from`. + - **Dropping and re-creating on purpose.** Split it into two applies: drop + in one, add in the next. + - **Why it is safe.** The refusal infers no identity and changes nothing it + accepts, so invariant 6 holds. Ordinary populated drops keep their current + behavior, which #984 decides separately. + - **No existing graph is affected.** The pattern needs a qualified name, so + no plan that runs today can trigger it. - **Interfaces wait.** Interfaces cannot be renamed today, so existing interfaces cannot move until [Polymorphic types](https://github.com/ModernRelay/omnigraph/pull/884) lands @@ -389,9 +481,31 @@ declined namespaces partly because "adding a field can break queries". Gel resolves within one active module and never searches others. The rule here goes one step stricter. A name either resolves to exactly one -declaration among the root and the `use`d namespaces, or it is an error. Every -name is bound at compile time to a `StableTypeId` in the typed IR, so a later -schema change can produce a loud plan-time error but never a different binding. +declaration among the root and the `use`d namespaces, or it is an error. + +The compiler resolves every name to its canonical name. The IR carries that +canonical name (`IROp::NodeScan { type_name }`, +`IROp::Expand { src_type, dst_type }`), and the planner and engine find tables +by it (`node:{type_name}`), not by `StableTypeId`. Four existing mechanisms keep +that safe, and the fifth arrives with Polymorphic types: +1. **One snapshot per query** (invariant 3), so the name lookup and the catalog + agree for the whole run. +2. **Stored queries are type-checked again** at `cluster plan` + (`validate_query_source`), at apply and at server load, so a schema change + that breaks one is caught before it serves. +3. **The resolver gives an exact `(kind, name)` match priority over a hint** + (`resolve` in `catalog/schema_ir.rs`). One apply therefore cannot release a + name and reuse it with the same kind: `node Beta @rename_from("Alpha")` plus + a new `node Alpha` is refused as "consumed by more than one desired + declaration". +4. **The exactly-one rule above.** +5. **Registry pinning** by kind and `StableTypeId`, which + [Polymorphic types](https://github.com/ModernRelay/omnigraph/pull/884) + adds for stored queries. + +A later schema change can produce a loud plan-time error, but not a silent +rebinding of a stored query. Carrying ids in the query IR would change the +planner and engine as well as the compiler; it is outside this RFC. **Files and namespaces stay independent.** Protobuf made packages independent of files and the ecosystem pushed them back together with lint (Buf @@ -424,8 +538,9 @@ Where they meet, this is how they compose. - edge alternations (`(crm::ownedBy | hr::ownedBy)`). In a union, each member resolves on its own. The grammar has one - `type_ref = (segment "::")* type_name` rule that both RFCs use, so they never - grow two spellings. + `qualified_type_name = (segment "::")* type_name` rule that both RFCs use, so + they never grow two spellings. The name avoids `type_ref`, which both grammars + already use for property and parameter types. - **Interfaces span namespaces.** An interface's member set is its concrete implementors in any namespace. `$x: core::Named` reads `crm::Contact` and `hr::Employee` alike. Namespaces never filter membership; a query narrows @@ -511,11 +626,15 @@ with a one-component id. - **6, identity from stable ids.** A namespace is part of a name. Moves are renames that must be declared; nothing infers identity from a name, a namespace, a file or a path. -- **8, loud failures.** Ambiguous names, unknown namespaces, duplicate - declarations across files, reserved names and populated drops (#984) are - typed refusals. -- **9, typed semantics.** `use` and qualified names are AST and IR structures. - Resolution finishes in the compiler, and execution sees only type ids. +- **8, loud failures.** These are typed refusals: + - ambiguous names and unknown namespaces; + - duplicate declarations across files, and a file named twice; + - reserved names; + - a probable move without `@rename_from`. +- **9, typed semantics.** `use` and qualified names are AST structures. + Resolution finishes in the compiler against the query's snapshot. The IR and + execution receive only resolved canonical names, never a relative or + unresolved one. - **12, one source of truth.** No namespace registry exists, and the composed source is the only stored schema text. File layout is not persisted; the repository owns it. @@ -523,10 +642,25 @@ with a one-component id. ## Compatibility and reversibility -- **Multi-file schemas.** There is no storage stamp, IR, ledger or HTTP change. - A single-file schema keeps byte-identical source and digests, and - `cluster.yaml` gains an untagged alternative, a minor change under the - compatibility-surfaces RFC. +- **Multi-file schemas.** + - **No format changes.** There is no storage stamp, IR or ledger field + change, and a single-file schema keeps byte-identical source and digests. + - **The `cluster.yaml` change.** It gains an untagged alternative, a minor + change under the compatibility-surfaces RFC. + - **The single-path form is safe across versions.** A file or one directory + stays a string in `config_semantics`, the JSON of the whole + `RawClusterConfig` that a deployment bundle carries to the server + (`PlanRequest`, `DeploymentRequest`) and stores as `deployment/{digest}.json`. + A server reads only the composed source from the bundle, so any version + accepts it. + - **The list form is a skew rule.** It is a JSON array where an older server + expects a path. `validate_bundle` re-parses `config_semantics` into + `RawClusterConfig`, so an older server refuses the bundle with + `deployment_input_invalid`, and an older binary refuses a stored bundle + that holds one. This fails closed. Use the list form only once every server + and binary that receives or reads the cluster's bundles has this change. + - **CLI JSON.** `cluster validate` and `plan --json` gain the optional + `ResourceSummary.paths` field, an additive change to that surface. - **Namespaces.** - PG and GQ grammars gain a block, a `use` line and qualified names, all minor changes. Neither adds a reserved word, so every accepted schema and query @@ -544,7 +678,7 @@ with a one-component id. | Alternative | Reason not selected | |---|---| | A prefix convention (`Crm_Contact`) | Works today with no engine change, and stays the baseline until a need appears. It gives no query scope, no policy scope, and edge names still collide across the graph. | -| `import "./x.pg"` between files | With declared namespaces, imports add cycle and order rules and nothing else. Protobuf shows file imports leak file structure to callers (`import public`, Buf `IMPORT_NO_PUBLIC`). Composition is declared once, in configuration. | +| `import "./x.pg"` between files (issue #838) | With declared namespaces, imports add cycle and order rules and nothing else. Protobuf shows file imports leak file structure to callers (`import public`, Buf `IMPORT_NO_PUBLIC`). Composition is declared once, in configuration. The cost of that choice: a `.pg` file does not describe itself, so an editor or a one-file `lint` needs the composing declaration to resolve a name from another file. | | File or directory = namespace (Go, TypeScript) | Ties persisted names to file layout: moving or renaming a file becomes a schema migration, and templates copied into different folders get different type names. | | File-scoped `namespace crm;` header | Breaks joining files by concatenation, unless the loader rewrites the text, so the stored source would no longer be the files' bytes. Possible later as sugar the composer expands. | | Namespace as a separate IR field | New uniqueness and sort keys, a namespace copy in every IR reference type and a new planner step, while every wire surface still needs one string. | @@ -579,9 +713,24 @@ Tests extend the existing owners in [testing](../dev/testing.md): and a move into a namespace producing `RenameType`. - Query parser and typecheck tests: `use`, ambiguity, unknown namespaces, and the binding/edge choice by final segment. -- **Cluster.** Config tests that mirror the `QueriesDecl` tests: directory, - list, symlink and `..` refusal, caps, an empty directory, sorted order, and an - unchanged digest for the single-path form. +- **Cluster.** + - Config tests that mirror the `QueriesDecl` tests: directory, list, symlink + and `..` refusal, caps, an empty directory, a file named twice, sorted + order, and an unchanged digest for the single-path form. + - `schema: schema/`, `schema: [schema/]` and the explicit file list compose + identical bytes, and the composed 1 MiB bound refuses one byte more. + - The list form sent through `validate_bundle`, pinning the skew rule. +- **Refusal and compatibility.** + - `crates/omnigraph/src/db/catalog_tests.rs` already proves that an unknown + feature name refuses open before any effect. Adding `namespaces` there pins + the older-binary refusal; the envelope gate in `schema_state.rs` runs before + the source is parsed. + - A probable move without `@rename_from` is refused with its code, while an + ordinary populated drop plans as it does today. + - `.gqt` cases under `crates/omnigraph-gqt/cases/` for the ambiguity, + unknown-namespace and reserved-segment errors. + - A regression row proving that a property named `namespace` or `use`, an + edge `Use` and the alias `as use` keep parsing. All four work today. - **CLI.** - `--schema ` for every command that takes a schema. - Errors with `file:line`. @@ -605,16 +754,17 @@ Tests extend the existing owners in [testing](../dev/testing.md): - The shared loader and source map, `SchemaDecl` in `cluster.yaml`, and `--schema `. - No format change. -2. **Fail-closed populated drops (#984).** A prerequisite for step 3. -3. **Namespaces.** +2. **Namespaces.** - `.pg` blocks and the `namespaces` IR feature. + - The refusal of a probable move without `@rename_from`, in the same change + as the blocks. - `.gq` qualified names and `use`. - Qualified names on every wire surface. - The selector and GQT parser fixes. -4. **Moving interfaces.** Follows interface `@rename_from` in Polymorphic +3. **Moving interfaces.** Follows interface `@rename_from` in Polymorphic types. Abstract bindings, unions and polymorphic endpoints take qualified names whenever both RFCs have shipped, in either order. -5. **Later, each on demand.** +4. **Later, each on demand.** - Namespace-scoped per-type policy rules. - A layout lint. - A helper that writes `@rename_from` hints for a bulk move. @@ -625,10 +775,6 @@ Tests extend the existing owners in [testing](../dev/testing.md): - **Where `use` binds.** It applies per query text here. Gel binds per statement (`with module`). Is a per-query clause better for stored queries that share a file? -- **File banners.** They make `schema show` readable and let tools split the - stored text, but they also become part of the digest of every multi-file graph. - The alternative is no banners, with source maps on the client only, as Gel - does. - **Recursive discovery.** It matches `queries:` (top level only) here. Should both recurse together? - **Bulk moves.** Are per-type `@rename_from` hints enough, or is a block-level @@ -666,3 +812,17 @@ protected. A move respells integer-identity handles, which parse at their last `:`. Added the GQT load-plan and `type`/`cast` parsers to the fix list, view-catalog name resolution from Actor views, and the block form of a namespaced interface declaration. + +2026-10-18: answered the first review: +- linked issue #838 as the discussion and recorded which of its needs this + design keeps, drops or decides differently; +- replaced the #984 prerequisite with a narrow refusal this RFC owns, a + probable move without `@rename_from`; +- restated the safety argument on the mechanisms that exist, since the query + IR carries canonical names rather than `StableTypeId`; +- stated the list-form skew rule for `config_semantics` and the new + `ResourceSummary.paths`; +- fixed one banner path rule, the composed 1 MiB bound, the error for a file + named twice, canonical rename hints, reserved-word segments, quoted + qualified edges and the `qualified_type_name` rule name; +- added refusal and compatibility tests. From 9c360b5a47723cd0dd8520dfa6b8145cdc9ac7f8 Mon Sep 17 00:00:00 2001 From: Ragnor Comerford Date: Sun, 18 Oct 2026 13:08:19 -0700 Subject: [PATCH 5/5] rfc: link the StableTypeId-in-IR issue from schema namespaces --- .../2026-10-15-schema-namespaces-and-multi-file-schemas.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md b/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md index 4c6e719f0..39ba7404e 100644 --- a/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md +++ b/docs/rfcs/2026-10-15-schema-namespaces-and-multi-file-schemas.md @@ -506,6 +506,10 @@ that safe, and the fifth arrives with Polymorphic types: A later schema change can produce a loud plan-time error, but not a silent rebinding of a stored query. Carrying ids in the query IR would change the planner and engine as well as the compiler; it is outside this RFC. +[Issue #1079](https://github.com/ModernRelay/omnigraph/issues/1079) tracks it, +as phase 0 of Polymorphic types. Namespaces do not depend on it, but it makes a +move cheaper: derived state keyed by id, such as the graph-index topology +cache, no longer rebuilds after a rename. **Files and namespaces stay independent.** Protobuf made packages independent of files and the ecosystem pushed them back together with lint (Buf