Repository navigation
Conversation
A direct CLI write or maintenance command on a cluster graph installed no policy, so actors the graph's applied policy refuses (and no actor at all) published on protected branches, while `omnigraph access` on the same target reported the policy's answer. Every writable embedded open now goes through admission::open_writable, which reads the bundle bound to the graph under the admission lock (ClusterAdmission::graph_policy, reading only that graph's bundles) and installs it on the handle. optimize and cleanup accept --as so maintenance can name the actor the policy judges. The engine's missing-actor refusal names the action instead of calling the case a bug. Closes ModernRelay#1414.
… mismatch exits 4 open_writable takes the storage future and awaits it after admission, so a write refused by a held cluster lock never builds the read-through adapter that writes a clone marker. A served optimize whose --as the credential does not match exits 4, as other served writes do. The S3 direct-root journey names an actor its graph policy admits for optimize and cleanup.
ragnorc
left a comment
There was a problem hiding this comment.
Review of #1423 at 4dbdfb6
Recommendation: merge after one small fix (B1). The direction is right and the change removes liability.
What and why
Before this change, a direct CLI write to a cluster graph (a root target with --graph, or the graph's own store) installed no policy. load --as act-outsider published on a protected main that the server on the same root refuses with 403, while access, the --as help, invariant 10 and the policy guide said the policy applies (#1414). Now every writable open in the CLI goes through one door, admission::open_writable. It admits the graph, reads the policy bound to it from the ledger under the admission lock, and installs it. The engine's existing _as gate then judges every write and maintenance command before any effect.
Impact on liability
- Removes: four call sites that each did "admit, then open" by hand become one opener. The direct path,
accessand the server now give one answer, and the docs, the skill and the--ashelp stop contradicting each other. A new writer in the CLI gets the policy from the opener; it does not need to remember it. - Adds: one more reader of "the policy bound to graph G" (
AppliedPolicies::load_graph, see O1), one more state read and one policy payload read for each direct write command, and a breaking change for actor-less direct writes on policied cluster graphs, which the changelog note states. - The decision you asked to be challenged (maintenance is covered too): I agree with covering it. One rule for one opener is simpler than two, and the actor is self-declared, so this keeps a storage owner inside the policy; it does not lock them out. If a policy grants
reclaimto nobody, the remedy is a policy change throughcluster apply, and that is the authority that should decide. The only cost is that the breaking note must also reach people who runcleanupon a stopped S3 root;docs/user/operations/s3-cluster-roots.md:14showsoptimize --target s3://… --graph knowledgewithout--as. One sentence there would help.
Blocking
B1. A policy-refused write under --base reports effects: none but writes the clone marker. After that, the graph refuses ordinary direct opens.
admission::open_writable (crates/omnigraph-cli/src/admission.rs:80-97) awaits the storage future after admission, and that future builds the read-through adapter. ReadThroughMount::open (crates/omnigraph-storage/src/read_through.rs:423-427) puts __clone/created into the target root while it is built. The engine's policy gate runs only later, in the _as call. 4dbdfb6 fixed this for a lock refusal, which Devin reported, but not for the new policy refusal.
Reproduced at 4dbdfb6 with a temporary test in cli_data (since removed). I used converged_loaded_cluster("knowledge", Some(WRITER_READER_POLICY_YAML)) and an independent base made with init_graph, then ran:
load --target <root>/graphs/knowledge.omni --base <base.omni> --data row.jsonl --mode append --as act-outsider --json
- stdout:
{"code": "forbidden", "disposition": {"effects": "none", …}}, exit 1, manifest version unchanged <root>/graphs/knowledge.omni/__clone/creatednow exists- a following
schema show --target <root>/graphs/knowledge.omniexits 1 withtarget_invalid: … is a graph shallow clone; open it with --base …
The cause is older than this PR: building the adapter is effectful, so every refusal that happens after the open (a type error too) leaves the marker. But this PR adds a refusal class that will be common (a missing or wrong --as) on one of the two addresses it covers, and the PR says each refusal has no effects. Fix it where the storage contract is owned, not in the CLI. Make ReadThroughMount::open refuse a layer that already holds objects but no __clone/created. Such a layer is not a clone, and reading another graph through it is not a supported shape. The refusal then comes before any write, for every caller and every later refusal. A real new clone (an empty layer) keeps its marker on first build, as today. If you want a smaller scope here, add the case to direct_writes_to_a_cluster_graph_obey_its_applied_policy_issue_1414 (assert no __clone after a forbidden --base load) and move the storage fix to a follow-up issue. Either way, the PR should not claim effects: none for that path while it is open.
Optional
O1. Three readers decide which policy binds a graph. AppliedPolicies::load_optional (crates/omnigraph-cluster/src/authorization.rs:218), the new load_graph (:307, 56 lines that mostly repeat load_optional), and the serving projection (crates/omnigraph-cluster/src/serve.rs:833-858), which access filters again (crates/omnigraph-cli/src/commands/targets/access.rs:193-215). They already differ at the edges. load_graph refuses a binding-less entry even when it belongs to another graph, but it ignores an unrecognized binding that load_optional refuses. The serving projection reports a binding-less entry as a diagnostic. Today access and the write agree on the cases your test covers, but only because the copies agree; nothing in the code makes them agree. Suggestion: one function that returns the bundle bound to graph G from an applied state, used by load_optional, the serving projection and the direct opener. Then "the write obeys what access reports" is true by construction, and the next binding change (#951 and #1210 both add authority shapes) edits one place, not three.
O2. A refused direct write still keeps the cluster admission lock. The new test has to call unlock_cluster_fixture after every refused command (cli_data.rs, the writes closure), and the refusal output shows cluster admission retained: … lock_id=… beside effects: none. This is the existing rule for every direct writer (s3-cluster-roots.md), so it is not a regression. But a forgotten --as now leaves a stopped server unable to boot until someone runs force-unlock. The policy is read before the writable open, and the engine gate runs before any staging, so a forbidden refusal is a settled, read-only outcome. When #1360 lands, its settlement check could release the lock on that path too. For now, consider saying in policy.md#actor-identity that a refused direct write still holds the lock.
O3. Nits.
crates/omnigraph-cli/src/client.rs:206still says "Direct-store access carries no Cedar policy (RFC-011 …)". This is now false, and RFC 0011 says the opposite.open_write_session_with_external_blob_policykeeps an unused_uriparameter (client.rs:481). Remove it rather than rename it, sinceopen_writabletakes the URI from the binding.
Checked and correct
- Every CLI writable open goes through
open_writable: load, mutate, branch create/delete/merge, commit publish/drop, Blob put/clear,load csv, index build and rebuild, optimize, cleanup. Each passes the resolved actor to an_asentry point (client.rs1316, 1404, 1555, 1662, 1749, 2262, 2309, 2837;maintenance/mod.rs).schema upgrade-system-columnsrefuses a cluster graph before it opens (schema_upgrade.rs:19-27), andschema applywas already refused there. - The policy is read under the admission lock, which a deployment needs to change it, so it cannot change during the command. Reads stay unjudged, as
policy.mdnow says. - The served
optimizemaps a 412actor_mismatchto exit 4 (served.rs), and an ordinary 412 still maps to before-acceptance. - No DST storage-cost row is needed: the golden covers engine operations, not CLI admission. The added cost is one state GET and one policy payload GET per direct write command, next to the admission it already pays for.
Validation
Clean detached worktree at 4dbdfb6 (merge base 4f7c207; main is now 8ab6571). Toolchain 1.97.1. Commands used the cargo dev-test feature set.
cargo dev-test --test integration -- cli_data::direct_writes_to_a_cluster_graph_obey_its_applied_policy_issue_1414 cli_cluster::v2_root_admission_blocks_cli_write_and_native_control_doors: both pass at 4dbdfb6.- Before the fix: with
crates/omnigraph-cli/srcandcrates/omnigraph-cluster/srcchecked out at 4f7c207 and the PR's tests kept, the regression fails atcli_data.rs:6673for the reported reason:load --as act-outsiderthrough the root target exits 0 and publishes a commit withactor_id: "act-outsider". The engine diff changes only a message and docs, so I left it at head. Restored afterwards. cargo dev-test -p omnigraph-cli --lib -- refusals_map_to_their_exits planes::: 5 passed.- B1 probe: the temporary test above, run at 4dbdfb6 and then removed.
- CI at 4dbdfb6: 27 passed, 4 skipped.
- GQT: the contract cannot be expressed there. A case has no actor, no Cedar policy input and no CLI direct-root target, as #1414 also says, so the CLI integration owner is the right tier.
- Not run: the full workspace graph, Clippy (CI covers both), S3 roots.
No Decision needed comments on this PR. Earlier review: Devin's at 2efb895; both of its points are fixed in 4dbdfb6 (B1 above is the same class on the policy path).
A read-through mount wrote __clone/created into any layer it had not marked, so a --base write refused after the open (by policy, say) left an ordinary graph marked as a clone, and it then refused every open without --base. The mount now refuses a layer that holds objects under its prefix but no marker, before any write. Also drops the unused uri argument of the writable openers and corrects the stale policy comment on GraphClient::Embedded.
|
Addressed in 915ef13:
Locally: the CLI integration suite (372), the storage lib tests, the engine shallow-clone and read-through tests, and workspace Clippy pass. Clippy ran on toolchain 1.98 with the two lints in untouched files allowed. |
Conflict in docs/user/operations/s3-cluster-roots.md: main added the paragraph saying a direct write or maintenance command is judged by the graph's applied policy (ModernRelay#1423), just before the admission-lock paragraph this branch rewrote. Keep main's new paragraph and this branch's lock text.
Closes #1414. A direct CLI write to a cluster graph (a root target with
--graph, or the graph's own store) installed no policy, soload --as outsider,mutate --as readerand an actor-lessloadall published on a protectedmainthat the same cluster's server refused with 403. Meanwhileomnigraph accesson that target evaluated the applied policy and reported nochange, and the--ashelp, the policy guide, invariant 10 and RFC 0011 ("'Direct' means 'no HTTP boundary,' not 'unpoliced'") all said the policy applies. Only the maintenance guide and a code comment said otherwise.Every writable open in the CLI now goes through one opener,
admission::open_writable. It admits a cluster graph as before, reads the policy bound to that graph through a newClusterAdmission::graph_policy, and installs it withwith_policy. Data writes (load, mutate, branch and commit commands, Blob put and clear) and maintenance (optimize, index build and rebuild, cleanup) are then judged by the engine's existing_asgate before any effect. The guard is where the policy comes from. It is read from the ledger under the admission lock the write already holds, and a deployment needs that lock to change it, so the policy cannot move during the command. Only the bundles bound to the graph are read, so a damaged payload the graph does not use does not block it, while a missing or invalid bundle of its own refuses. A standalone graph has no policy and is unchanged.optimizeandcleanupused to refuse--as, and the version-1 operator file refusesoperator.actor, so they now accept--as; a servedoptimizesends it as the usual assertion. The engine's missing-actor refusal used to call the case "almost certainly a bug", and now names the action instead.cli_data::direct_writes_to_a_cluster_graph_obey_its_applied_policy_issue_1414applies a policy in which onlyact-writermay changemain,act-readermay only read andact-maintaineralone may maintain and reclaim. Through both addresses it checks thataccessand the write agree foract-outsider,act-reader, no actor andact-writer. Each refusal isforbiddenwitheffects: noneand leaves the manifest version unchanged, and each permitted write publishes with the actor recorded.index build,optimizeandcleanupare refused foract-writerand run foract-maintainer. With the policy install disabled, the test fails on the firstact-outsiderload, which publishes as the issue reports. Test setups that seeded a policied cluster graph with an actor-less direct load now pass an actor the policy admits, or seed as an embedded host with no policy (seed_as_storage_owner) where the policy admits no writer tomain. The degraded-catalog optimize test now removes only the stored-query payloads, and asserts that removing the graph's own policy refuses. The CLI integration suite (372), the CLI and cluster unit tests and the engine policy chassis pass locally.For review: this is breaking for anyone who writes to a policied cluster graph directly without an admitted
--as, and the changelog note says so. The decision to challenge is that maintenance is covered too. A directoptimizeorcleanupon a policied cluster graph now needsmaintainorreclaimfor its--as, so it is no longer a policy-free break-glass path.cleanuphas no served route, so a graph whose policy grants nobodyreclaimcannot be cleaned up until the policy grants it. The alternative is to exempt maintenance, which keeps two rules for one opener. The direct actor is self-declared, so this is a guardrail, not authentication: the guides say storage permissions remain the trust boundary. Direct reads are still not judged, since the engine gates no reads, andaccessstill reportsreadfor a direct target; the policy guide now says so, and gating direct reads would be a separate change. Locally the session toolchain is 1.98 rather than the pinned 1.97.1. Its clippy flags two lints in untouched files (omnigraph-catalog/src/history.rs,omnigraph/src/db/graph_coordinator.rs); with those two allowed, clippy is clean.