diff --git a/docs/design/cross-ledger-model-enforcement.md b/docs/design/cross-ledger-model-enforcement.md index aaa5aadb41..314340dfcb 100644 --- a/docs/design/cross-ledger-model-enforcement.md +++ b/docs/design/cross-ledger-model-enforcement.md @@ -135,13 +135,16 @@ The helper performs, in order: error. Two different `atT` pins of the same `(ledger, graph)` are not a cycle; the same triple is. Push the tuple onto `active` before recursing into transitive imports. -6. **Translation and materialization.** The graph at `resolved_t` is - read and projected into term-neutral form (see next section). +6. **Translation and materialization.** The graph at M's head is + read and projected into term-neutral form (see next section). A + cached handle of M serves the read only when its head is that + commit; otherwise M is loaded from storage at `resolved_t`. Pop the tuple from `active` and insert the resolved artifact into `ctx.memo` and the global cache. 7. **Caching.** On cache hit the materialized artifact is returned directly. On miss the artifact is inserted under the key - `(ArtifactKind, canonical_model_ledger_id, graph_iri, resolved_t)`. + `(ArtifactKind, canonical_model_ledger_id, graph_iri, head)`, + where `head` is M's `resolved_t` together with its head commit id. A `ResolvedGraph` is term-neutral and t-fixed: @@ -307,11 +310,18 @@ stabilizes — keeping the two caches separate while artifact shapes are still evolving prevents premature coupling. The key is `(ArtifactKind, canonical_model_ledger_id, graph_iri, -resolved_t)`. `ArtifactKind` is part of the key so a memoized -`PolicyRules` entry never short-circuits a `Shapes` lookup of the -same `(M, graph, t)`. New commits to M produce new keys without -explicit invalidation; unreferenced entries age out under the -cache's eviction policy. There is no "watermark-on-write" channel. +head)`, where `head` is M's `resolved_t` and the commit id at that +head. `ArtifactKind` is part of the key so a memoized `PolicyRules` +entry never short-circuits a `Shapes` lookup of the same +`(M, graph, head)`. `t` alone is not M's identity: a ledger dropped +and created again under M's name restarts at `t` 1, so the head +commit is what keeps it from being served the dropped ledger's +artifacts. The artifact stored under a key is read at that commit too, +so an instance sharing storage with the one that dropped and recreated +M never fills the new key from its cached handle of the dropped +ledger. New commits to M produce new keys without explicit +invalidation; unreferenced entries age out under the cache's eviction +policy. There is no "watermark-on-write" channel. The cache value is the term-neutral `ResolvedGraph` (IRIs, not Sids). Per-data-ledger interning is not part of the cache key — the cache diff --git a/docs/security/cross-ledger-policy.md b/docs/security/cross-ledger-policy.md index fa7e710e76..923fc84d91 100644 --- a/docs/security/cross-ledger-policy.md +++ b/docs/security/cross-ledger-policy.md @@ -492,7 +492,7 @@ Specifics: its namespace; a constraint over vocabulary D has never seen anywhere is silently inert (no rows, never an error). - **Steady-state cost is one nameservice lookup.** The wire is - cached per `(model, graph, resolved_t)` and the *compiled* + cached per `(model, graph, head)` and the *compiled* shapes (including parsed `sh:sparql` queries) are reused across transactions while M's head and D's shape-affecting epochs are unchanged. A commit on M invalidates both on the @@ -658,11 +658,13 @@ clients can branch on the specific failure. ## Behavior on model ledger updates There is no explicit invalidation channel. The cache key includes -the model ledger's `resolved_t` (its commit head at the time of -capture), so new commits to M produce new cache keys -automatically. The next request after M advances captures the +the model ledger's head at the time of capture, its `t` and head +commit id, so new commits to M, and a new ledger created under M's +name after a drop, produce new cache keys automatically. The next request after M advances captures the new head; older entries age out under the cache's LRU/TinyLFU -policy. +policy. This holds across servers sharing the storage and +nameservice: one whose ledger cache still holds a dropped M enforces +the M created after it. Within a single request, every cross-ledger reference to the same M reuses one `resolved_t` value. Policy and any future diff --git a/fluree-db-api/src/cross_ledger/cache.rs b/fluree-db-api/src/cross_ledger/cache.rs index 4f780b72c9..bd75f10ed0 100644 --- a/fluree-db-api/src/cross_ledger/cache.rs +++ b/fluree-db-api/src/cross_ledger/cache.rs @@ -2,10 +2,10 @@ //! //! Lives on the `Fluree` handle so resolved artifacts are shareable //! across requests and across every data ledger that references the -//! same `(ArtifactKind, model_ledger_id, graph_iri, resolved_t)`. +//! same `(ArtifactKind, model_ledger_id, graph_iri, model head)`. //! That is the property that makes "model edit propagates atomically //! to every governed dataset" cheap — one cache entry is reused by -//! every D that points at the same M graph at the same M-t. +//! every D that points at the same M graph at the same M head. //! //! Phase 1a uses a Moka TinyLFU cache bounded by entry count. Unifying //! with `fluree-db-binary-index::LeafletCache`'s byte budget is a @@ -22,8 +22,9 @@ //! - Resolver: per-request memo miss → cache lookup → cache miss //! triggers materialization → write back into both per-request memo //! and instance cache → return. -//! - Cache invalidation is implicit: new commits to M produce new -//! `resolved_t` values and therefore new keys. Old entries age out +//! - Cache invalidation is implicit: every new head of M, including one +//! that repeats an earlier `t` (M dropped and recreated, or rebased), +//! has a new head commit and therefore a new key. Old entries age out //! under TinyLFU eviction. There is no watermark-on-write channel. use super::types::{ResolutionKey, ResolvedGraph}; diff --git a/fluree-db-api/src/cross_ledger/constraints_materializer.rs b/fluree-db-api/src/cross_ledger/constraints_materializer.rs index 459a80904a..a204a0c39d 100644 --- a/fluree-db-api/src/cross_ledger/constraints_materializer.rs +++ b/fluree-db-api/src/cross_ledger/constraints_materializer.rs @@ -11,7 +11,7 @@ //! a list of property IRIs, no per-rule body, no class targets, no //! values. -use super::types::{ConstraintsArtifactWire, WireOrigin}; +use super::types::{ConstraintsArtifactWire, ModelHead, WireOrigin}; use super::ResolveCtx; use super::{encode_system_iri, CrossLedgerError}; use fluree_db_core::{FlakeValue, IndexType, RangeMatch, RangeTest}; @@ -26,18 +26,19 @@ use fluree_vocab::config_iris; fields( model_ledger = canonical_model_ledger_id, graph_iri = graph_iri, - resolved_t = resolved_t, + resolved_t = head.t, ), )] pub(super) async fn materialize_constraints( canonical_model_ledger_id: &str, graph_iri: &str, - resolved_t: i64, + head: &ModelHead, ctx: &ResolveCtx<'_>, ) -> Result { - // 1. Open M at resolved_t. + let resolved_t = head.t; + // 1. Open M at its head. let m_db = ctx - .open_model_db(canonical_model_ledger_id, resolved_t) + .open_model_db(canonical_model_ledger_id, head) .await .map_err(|e| CrossLedgerError::TranslationFailed { ledger_id: canonical_model_ledger_id.to_string(), @@ -111,6 +112,7 @@ pub(super) async fn materialize_constraints( model_ledger_id: canonical_model_ledger_id.to_string(), graph_iri: graph_iri.to_string(), resolved_t, + commit_id: head.commit_id.clone(), }, property_iris, }) diff --git a/fluree-db-api/src/cross_ledger/mod.rs b/fluree-db-api/src/cross_ledger/mod.rs index 32aaec7480..09cb079864 100644 --- a/fluree-db-api/src/cross_ledger/mod.rs +++ b/fluree-db-api/src/cross_ledger/mod.rs @@ -22,8 +22,9 @@ pub use cache::GovernanceCache; pub use error::CrossLedgerError; pub use resolver::resolve_graph_ref; pub use types::{ - ArtifactKind, ConstraintsArtifactWire, GovernanceArtifact, ResolveCtx, ResolvedGraph, - RulesArtifactWire, SchemaArtifactWire, ShapesArtifactWire, WireObject, WireOrigin, WireTriple, + ArtifactKind, ConstraintsArtifactWire, GovernanceArtifact, ModelHead, ResolveCtx, + ResolvedGraph, RulesArtifactWire, SchemaArtifactWire, ShapesArtifactWire, WireObject, + WireOrigin, WireTriple, }; /// Resolve a `f:graphSelector` IRI against a model ledger snapshot. diff --git a/fluree-db-api/src/cross_ledger/policy_materializer.rs b/fluree-db-api/src/cross_ledger/policy_materializer.rs index 04587b3a99..5670e438ca 100644 --- a/fluree-db-api/src/cross_ledger/policy_materializer.rs +++ b/fluree-db-api/src/cross_ledger/policy_materializer.rs @@ -46,6 +46,7 @@ //! `TranslationFailed` (no silent drop; dropping a target would //! produce a structurally weaker policy than authored). +use super::types::ModelHead; use super::ResolveCtx; use super::{encode_system_iri, CrossLedgerError}; use fluree_db_core::{FlakeValue, IndexType, LedgerSnapshot, RangeMatch, RangeTest, Sid}; @@ -74,18 +75,19 @@ use std::collections::HashSet; fields( model_ledger = canonical_model_ledger_id, graph_iri = graph_iri, - resolved_t = resolved_t, + resolved_t = head.t, ), )] pub(super) async fn materialize_policy_rules( canonical_model_ledger_id: &str, graph_iri: &str, - resolved_t: i64, + head: &ModelHead, ctx: &ResolveCtx<'_>, ) -> Result { - // 1. Open M at resolved_t. + let resolved_t = head.t; + // 1. Open M at its head. let m_db = ctx - .open_model_db(canonical_model_ledger_id, resolved_t) + .open_model_db(canonical_model_ledger_id, head) .await .map_err(|e| CrossLedgerError::TranslationFailed { ledger_id: canonical_model_ledger_id.to_string(), @@ -234,6 +236,7 @@ pub(super) async fn materialize_policy_rules( model_ledger_id: canonical_model_ledger_id.to_string(), graph_iri: graph_iri.to_string(), resolved_t, + commit_id: head.commit_id.clone(), }, restrictions: wire_restrictions, }) diff --git a/fluree-db-api/src/cross_ledger/resolver.rs b/fluree-db-api/src/cross_ledger/resolver.rs index 3bed396987..5090a1a503 100644 --- a/fluree-db-api/src/cross_ledger/resolver.rs +++ b/fluree-db-api/src/cross_ledger/resolver.rs @@ -10,8 +10,8 @@ //! See `docs/design/cross-ledger-model-enforcement.md`. use super::types::{ - check_cycle, memo_hit, reject_if_reserved_graph, ArtifactKind, GovernanceArtifact, ResolveCtx, - ResolvedGraph, + check_cycle, memo_hit, reject_if_reserved_graph, ArtifactKind, GovernanceArtifact, ModelHead, + ResolutionKey, ResolveCtx, ResolvedGraph, }; use super::CrossLedgerError; use fluree_db_core::ledger_config::GraphSourceRef; @@ -31,15 +31,15 @@ use std::sync::Arc; /// ledger is absent or retracted on this instance. /// 3. Reject `#config` / `#txn-meta` selectors before any storage /// round-trip on M. -/// 4. Capture `resolved_t` lazily: read `ctx.resolved_ts[model_id]` -/// on hit, else read M's head `commit_t` once and store. Pinned -/// `f:atT` is rejected at step (1) until Phase 3. +/// 4. Capture M's head lazily: read `ctx.resolved_heads[model_id]` +/// on hit, else read M's head `commit_t` and commit id once and +/// store. Pinned `f:atT` is rejected at step (1) until Phase 3. /// 5. Form the resolution key -/// `(ArtifactKind, canonical_model_ledger_id, graph_iri, -/// resolved_t)` and check, in order: (a) `ctx.memo` (per-request +/// `(ArtifactKind, canonical_model_ledger_id, graph_iri, head)` +/// and check, in order: (a) `ctx.memo` (per-request /// de-dup); (b) `fluree.governance_cache()` (per-instance, shared /// across requests and across every data ledger that references -/// the same (M, graph, t)). On hit at either layer, return — +/// the same (M, graph, head)). On hit at either layer, return — /// cross-subsystem de-dup runs before cycle detection, and a /// governance-cache hit is also folded into `ctx.memo` so later /// calls in the same request short-circuit at (a). @@ -141,25 +141,26 @@ pub async fn resolve_graph_ref( // different alias spelling. reject_if_reserved_graph(&canonical_ledger_id, graph_iri)?; - // (4) resolved_t: lazy per-request capture. f:atT pins are + // (4) M's head: lazy per-request capture. f:atT pins are // rejected at step (1) above (Phase 3, not yet implemented), so // every cross-ledger reference resolves against M's current head // for the duration of the request. Per-model entries are written // once and reused; subsequent unpinned references to the same M // in the same request hit the cache. - let resolved_t = if let Some(t) = ctx.resolved_ts.get(&canonical_ledger_id) { - *t - } else { - let head_t = ns_record.commit_t; - ctx.resolved_ts.insert(canonical_ledger_id.clone(), head_t); - head_t - }; - - let key: (ArtifactKind, String, String, i64) = ( + let head = ctx + .resolved_heads + .entry(canonical_ledger_id.clone()) + .or_insert_with(|| ModelHead { + t: ns_record.commit_t, + commit_id: ns_record.commit_head_id.clone(), + }) + .clone(); + + let key: ResolutionKey = ( kind, canonical_ledger_id.clone().to_string(), graph_iri.to_string(), - resolved_t, + head.clone(), ); // (5a) Memo hit — short-circuit cross-subsystem de-dup before @@ -188,8 +189,7 @@ pub async fn resolve_graph_ref( // Materialize. Slice 3 provides the dispatch shape; the actual // projector lands in slice 4. - let materialize_result = - materialize(kind, &canonical_ledger_id, graph_iri, resolved_t, ctx).await; + let materialize_result = materialize(kind, &canonical_ledger_id, graph_iri, &head, ctx).await; // Always pop active, even on error, so a failure deeper in the // chain doesn't leave stale entries that trip cycle detection @@ -208,7 +208,7 @@ pub async fn resolve_graph_ref( // write its own `Arc`. The cache ends up holding whichever // write completed last; the loser's `Arc` is dropped when // its request finishes. Wire artifacts are pure functions of - // `(kind, model_ledger, graph, resolved_t)`, so the two + // `(kind, model_ledger, graph, head)`, so the two // materialized values are structurally identical — duplicate // work, not duplicate semantics. Single-flight is a possible // future optimization but not a correctness requirement. @@ -225,7 +225,7 @@ async fn materialize( kind: ArtifactKind, canonical_ledger_id: &str, graph_iri: &str, - resolved_t: i64, + head: &ModelHead, ctx: &mut ResolveCtx<'_>, ) -> Result { match kind { @@ -233,14 +233,15 @@ async fn materialize( let wire = super::policy_materializer::materialize_policy_rules( canonical_ledger_id, graph_iri, - resolved_t, + head, ctx, ) .await?; Ok(ResolvedGraph { model_ledger_id: canonical_ledger_id.to_string(), graph_iri: graph_iri.to_string(), - resolved_t, + resolved_t: head.t, + commit_id: head.commit_id.clone(), artifact: GovernanceArtifact::PolicyRules(wire), }) } @@ -248,14 +249,15 @@ async fn materialize( let wire = super::constraints_materializer::materialize_constraints( canonical_ledger_id, graph_iri, - resolved_t, + head, ctx, ) .await?; Ok(ResolvedGraph { model_ledger_id: canonical_ledger_id.to_string(), graph_iri: graph_iri.to_string(), - resolved_t, + resolved_t: head.t, + commit_id: head.commit_id.clone(), artifact: GovernanceArtifact::Constraints(wire), }) } @@ -263,14 +265,15 @@ async fn materialize( let wire = super::schema_materializer::materialize_schema( canonical_ledger_id, graph_iri, - resolved_t, + head, ctx, ) .await?; Ok(ResolvedGraph { model_ledger_id: canonical_ledger_id.to_string(), graph_iri: graph_iri.to_string(), - resolved_t, + resolved_t: head.t, + commit_id: head.commit_id.clone(), artifact: GovernanceArtifact::SchemaClosure(wire), }) } @@ -278,14 +281,15 @@ async fn materialize( let wire = super::shapes_materializer::materialize_shapes( canonical_ledger_id, graph_iri, - resolved_t, + head, ctx, ) .await?; Ok(ResolvedGraph { model_ledger_id: canonical_ledger_id.to_string(), graph_iri: graph_iri.to_string(), - resolved_t, + resolved_t: head.t, + commit_id: head.commit_id.clone(), artifact: GovernanceArtifact::Shapes(wire), }) } @@ -293,14 +297,15 @@ async fn materialize( let wire = super::rules_materializer::materialize_rules( canonical_ledger_id, graph_iri, - resolved_t, + head, ctx, ) .await?; Ok(ResolvedGraph { model_ledger_id: canonical_ledger_id.to_string(), graph_iri: graph_iri.to_string(), - resolved_t, + resolved_t: head.t, + commit_id: head.commit_id.clone(), artifact: GovernanceArtifact::Rules(wire), }) } diff --git a/fluree-db-api/src/cross_ledger/rules_materializer.rs b/fluree-db-api/src/cross_ledger/rules_materializer.rs index c157adc926..9f47cc3313 100644 --- a/fluree-db-api/src/cross_ledger/rules_materializer.rs +++ b/fluree-db-api/src/cross_ledger/rules_materializer.rs @@ -14,7 +14,7 @@ //! does the same. A future variant could surface these as a warn //! when authors mistype the rule literal. -use super::types::{RulesArtifactWire, WireOrigin}; +use super::types::{ModelHead, RulesArtifactWire, WireOrigin}; use super::CrossLedgerError; use super::ResolveCtx; use fluree_db_core::{FlakeValue, IndexType, RangeMatch, RangeOptions, RangeTest}; @@ -27,17 +27,18 @@ use fluree_vocab::fluree::RULE; fields( model_ledger = canonical_model_ledger_id, graph_iri = graph_iri, - resolved_t = resolved_t, + resolved_t = head.t, ), )] pub(super) async fn materialize_rules( canonical_model_ledger_id: &str, graph_iri: &str, - resolved_t: i64, + head: &ModelHead, ctx: &ResolveCtx<'_>, ) -> Result { + let resolved_t = head.t; let m_db = ctx - .open_model_db(canonical_model_ledger_id, resolved_t) + .open_model_db(canonical_model_ledger_id, head) .await .map_err(|e| CrossLedgerError::TranslationFailed { ledger_id: canonical_model_ledger_id.to_string(), @@ -62,6 +63,7 @@ pub(super) async fn materialize_rules( model_ledger_id: canonical_model_ledger_id.to_string(), graph_iri: graph_iri.to_string(), resolved_t, + commit_id: head.commit_id.clone(), }, rules: Vec::new(), }); @@ -99,6 +101,7 @@ pub(super) async fn materialize_rules( model_ledger_id: canonical_model_ledger_id.to_string(), graph_iri: graph_iri.to_string(), resolved_t, + commit_id: head.commit_id.clone(), }, rules, }) diff --git a/fluree-db-api/src/cross_ledger/schema_materializer.rs b/fluree-db-api/src/cross_ledger/schema_materializer.rs index 9a4781eda3..8bfa42f7c0 100644 --- a/fluree-db-api/src/cross_ledger/schema_materializer.rs +++ b/fluree-db-api/src/cross_ledger/schema_materializer.rs @@ -14,7 +14,7 @@ //! across ledgers. Transitive cross-ledger imports land in a //! follow-up. -use super::types::{SchemaArtifactWire, WireObject, WireOrigin, WireTriple}; +use super::types::{ModelHead, SchemaArtifactWire, WireObject, WireOrigin, WireTriple}; use super::CrossLedgerError; use super::ResolveCtx; use fluree_db_core::{ @@ -31,20 +31,21 @@ use fluree_db_core::{ fields( model_ledger = canonical_model_ledger_id, graph_iri = graph_iri, - resolved_t = resolved_t, + resolved_t = head.t, ), )] pub(super) async fn materialize_schema( canonical_model_ledger_id: &str, graph_iri: &str, - resolved_t: i64, + head: &ModelHead, ctx: &ResolveCtx<'_>, ) -> Result { + let resolved_t = head.t; use fluree_vocab::{owl, rdf, rdfs}; - // 1. Open M at resolved_t. + // 1. Open M at its head. let m_db = ctx - .open_model_db(canonical_model_ledger_id, resolved_t) + .open_model_db(canonical_model_ledger_id, head) .await .map_err(|e| CrossLedgerError::TranslationFailed { ledger_id: canonical_model_ledger_id.to_string(), @@ -198,6 +199,7 @@ pub(super) async fn materialize_schema( model_ledger_id: canonical_model_ledger_id.to_string(), graph_iri: graph_iri.to_string(), resolved_t, + commit_id: head.commit_id.clone(), }, triples, }) diff --git a/fluree-db-api/src/cross_ledger/shapes_materializer.rs b/fluree-db-api/src/cross_ledger/shapes_materializer.rs index 29d684517c..2dfac16629 100644 --- a/fluree-db-api/src/cross_ledger/shapes_materializer.rs +++ b/fluree-db-api/src/cross_ledger/shapes_materializer.rs @@ -13,7 +13,7 @@ //! must use the *staged* `NamespaceRegistry`, not D's pre-staging //! snapshot. See that method's docs for why. -use super::types::{ShapesArtifactWire, WireObject, WireOrigin, WireTriple}; +use super::types::{ModelHead, ShapesArtifactWire, WireObject, WireOrigin, WireTriple}; use super::CrossLedgerError; use super::ResolveCtx; use fluree_db_core::{ @@ -31,17 +31,18 @@ const OWL: &str = "http://www.w3.org/2002/07/owl#"; fields( model_ledger = canonical_model_ledger_id, graph_iri = graph_iri, - resolved_t = resolved_t, + resolved_t = head.t, ), )] pub(super) async fn materialize_shapes( canonical_model_ledger_id: &str, graph_iri: &str, - resolved_t: i64, + head: &ModelHead, ctx: &ResolveCtx<'_>, ) -> Result { + let resolved_t = head.t; let m_db = ctx - .open_model_db(canonical_model_ledger_id, resolved_t) + .open_model_db(canonical_model_ledger_id, head) .await .map_err(|e| CrossLedgerError::TranslationFailed { ledger_id: canonical_model_ledger_id.to_string(), @@ -192,6 +193,7 @@ pub(super) async fn materialize_shapes( model_ledger_id: canonical_model_ledger_id.to_string(), graph_iri: graph_iri.to_string(), resolved_t, + commit_id: head.commit_id.clone(), }, triples, }) diff --git a/fluree-db-api/src/cross_ledger/types.rs b/fluree-db-api/src/cross_ledger/types.rs index 0a9dab4546..1695caba72 100644 --- a/fluree-db-api/src/cross_ledger/types.rs +++ b/fluree-db-api/src/cross_ledger/types.rs @@ -9,7 +9,7 @@ use super::CrossLedgerError; use crate::view::GraphDb; use crate::Fluree; use fluree_db_core::graph_registry::{config_graph_iri, txn_meta_graph_iri}; -use fluree_db_core::LedgerId; +use fluree_db_core::{ContentId, LedgerId}; use fluree_db_ledger::LedgerState; use fluree_db_policy::PolicyArtifactWire; use std::collections::HashMap; @@ -58,15 +58,27 @@ impl std::fmt::Display for ArtifactKind { } } -/// Memo / cycle-detection key. Includes `ArtifactKind` so concurrent -/// (within one request) resolutions for different artifact kinds -/// against the same `(ledger, graph, t)` don't collide. -pub(crate) type ResolutionKey = (ArtifactKind, String, String, i64); +/// The model-ledger head a resolution reads. +/// +/// `t` alone does not identify it: a ledger dropped and created again +/// under its name restarts at `t` 1, and a rebase can return a branch to a +/// `t` it held before with other commits. The head commit's content id is +/// unique to the history; `t` is what the materializers open the model at. +#[derive(Debug, Clone, PartialEq, Eq, Hash)] +pub struct ModelHead { + pub t: i64, + pub commit_id: Option, +} + +/// Memo / cycle-detection / governance-cache key. Includes `ArtifactKind` +/// so concurrent (within one request) resolutions for different artifact +/// kinds against the same `(ledger, graph, head)` don't collide. +pub(crate) type ResolutionKey = (ArtifactKind, String, String, ModelHead); /// A successfully resolved, term-neutral governance artifact. /// /// Cached at the API layer by `(ArtifactKind, model_ledger_id, -/// graph_iri, resolved_t)` — see [`ResolutionKey`]. Per-data-ledger +/// graph_iri, model head)` — see [`ResolutionKey`]. Per-data-ledger /// interning is a separate step that happens at the wire→PolicySet /// boundary against D's snapshot. #[derive(Debug, Clone)] @@ -77,6 +89,8 @@ pub struct ResolvedGraph { pub graph_iri: String, /// Model ledger `t` at which the artifact was materialized. pub resolved_t: i64, + /// Model ledger head commit at `resolved_t`. + pub commit_id: Option, /// The artifact itself, tagged by subsystem. Pattern-match this /// against the expected `GovernanceArtifact` variant for the /// requesting `ArtifactKind`. @@ -538,6 +552,8 @@ pub struct WireOrigin { pub graph_iri: String, /// Model ledger `t` at which the artifact was materialized. pub resolved_t: i64, + /// Model ledger head commit at `resolved_t`. + pub commit_id: Option, } /// Per-request resolution context. @@ -545,7 +561,7 @@ pub struct WireOrigin { /// Holds the full lifetime / consistency model for cross-ledger /// resolution within a single request: /// -/// - `resolved_ts` captures the lazy per-request head-t per +/// - `resolved_heads` captures the lazy per-request head per /// canonical model ledger id (governance-context capture). /// Lookup on miss reads M's head once and stores it; subsequent /// references to the same M reuse the same value so policy and @@ -569,10 +585,11 @@ pub struct ResolveCtx<'a> { /// constraint) the referenced model ledger. pub fluree: &'a Fluree, /// Lazy governance-context capture: canonical model ledger id → - /// `resolved_t`. Phase 1a is the only producer (M's head at - /// first reference); pinned `f:atT` is rejected upstream until - /// Phase 3. - pub resolved_ts: HashMap, + /// its head. Phase 1a is the only producer (M's head at first + /// reference); pinned `f:atT` is rejected upstream until Phase 3. + /// The head's `t` and commit are pinned together: a later stage + /// opening M at the pinned `t` must key under the same commit. + pub resolved_heads: HashMap, /// Active resolution stack (cycle detection). Keyed on the full /// resolution tuple including `ArtifactKind` so a `PolicyRules` /// resolve doesn't see a `Shapes` resolution of the same @@ -595,7 +612,7 @@ impl<'a> ResolveCtx<'a> { Self { data_ledger_id, fluree, - resolved_ts: HashMap::new(), + resolved_heads: HashMap::new(), active: Vec::new(), memo: HashMap::new(), data_state: None, @@ -609,59 +626,64 @@ impl<'a> ResolveCtx<'a> { self } - /// Open the model ledger at `t` for a materializer. + /// Open the model ledger at `head` for a materializer. + /// + /// A state is used only at the head's commit, not just its `t`, so what + /// is materialized matches the key it is cached under. /// /// A model that is the data ledger itself never goes through the ledger /// cache: the request resolving it may hold that ledger's write lock /// (staging does), and the cache's read of the same lock would wait on /// it forever. It is served from the state the caller passed, at the - /// same `t`, or loaded without the cache otherwise. + /// same head, or loaded without the cache otherwise. pub async fn open_model_db( &self, canonical_model_ledger_id: &str, - t: i64, + head: &ModelHead, ) -> crate::error::Result { if canonical_model_ledger_id != self.data_ledger_id { return self .fluree - .load_graph_db_at_t(canonical_model_ledger_id, t) + .load_graph_db_at_commit(canonical_model_ledger_id, head.t, head.commit_id.as_ref()) .await; } - if let Some(state) = self.data_state.as_ref().filter(|s| s.t() == t) { + let at_head = + |state: &LedgerState| state.t() == head.t && state.head_commit_id == head.commit_id; + if let Some(state) = self.data_state.as_ref().filter(|s| at_head(s)) { return Ok(GraphDb::from_ledger_state(state)); } let state = self .fluree .load_ledger_uncached(canonical_model_ledger_id) .await?; - if state.t() == t { + if at_head(&state) { return Ok(GraphDb::from_ledger_state(&state)); } self.fluree - .load_graph_db_historical(canonical_model_ledger_id, t) + .load_graph_db_historical(canonical_model_ledger_id, head.t) .await } - /// Build a resolution context pre-seeded with `resolved_t` - /// captures from an earlier stage of the same logical request. + /// Build a resolution context pre-seeded with head captures from + /// an earlier stage of the same logical request. /// /// Used by query preparation when a prior `wrap_policy` call on - /// the same view already pinned a model ledger's `resolved_t`: + /// the same view already pinned a model ledger's head: /// the new context inherits that pin so a second cross-ledger /// reference to the same M can't re-capture a later head and /// disagree with policy on which M version is in effect. /// `memo` and `active` are *not* shared — those are /// per-resolution-call state (cycle detection, dedup within a /// single dispatch tree) and don't carry across calls. - pub fn with_resolved_ts( + pub fn with_resolved_heads( data_ledger_id: &'a LedgerId, fluree: &'a Fluree, - resolved_ts: HashMap, + resolved_heads: HashMap, ) -> Self { Self { data_ledger_id, fluree, - resolved_ts, + resolved_heads, active: Vec::new(), memo: HashMap::new(), data_state: None, @@ -713,8 +735,11 @@ pub(crate) fn check_cycle( key: &ResolutionKey, ) -> Result<(), CrossLedgerError> { if active.iter().any(|k| k == key) { - let mut chain = active.to_vec(); - chain.push(key.clone()); + let chain = active + .iter() + .chain(std::iter::once(key)) + .map(|(kind, ledger, graph, head)| (*kind, ledger.clone(), graph.clone(), head.t)) + .collect(); return Err(CrossLedgerError::CycleDetected { chain }); } Ok(()) @@ -752,7 +777,8 @@ mod tests { } fn key(ledger: &str, graph: &str, t: i64) -> ResolutionKey { - (ArtifactKind::PolicyRules, ledger.into(), graph.into(), t) + let head = ModelHead { t, commit_id: None }; + (ArtifactKind::PolicyRules, ledger.into(), graph.into(), head) } #[test] @@ -774,9 +800,15 @@ mod tests { CrossLedgerError::CycleDetected { chain } => { // The chain should contain the original active stack // (in order) plus the offending key appended. + let shown = ( + ArtifactKind::PolicyRules, + "a:main".to_string(), + "http://ex.org/p".to_string(), + 10, + ); assert_eq!(chain.len(), 3); - assert_eq!(chain[0], cycle_key); - assert_eq!(chain[2], cycle_key); + assert_eq!(chain[0], shown); + assert_eq!(chain[2], shown); } other => panic!("expected CycleDetected, got {other:?}"), } @@ -792,22 +824,12 @@ mod tests { // pattern; it will be expanded in Phase 1b when SchemaClosure // lands. The contract this guards: adding a new ArtifactKind // doesn't make existing resolutions look cyclic. - let active = vec![( - ArtifactKind::PolicyRules, - "a:main".to_string(), - "http://ex.org/p".to_string(), - 10, - )]; + let active = vec![key("a:main", "http://ex.org/p", 10)]; // Once Phase 1b adds e.g. ArtifactKind::SchemaClosure, this // test should use that variant to verify cross-kind isolation. // For now: same kind / different t (a clearly-non-cycle case) // exercises the same code path under the new tuple shape. - let later_pin = ( - ArtifactKind::PolicyRules, - "a:main".to_string(), - "http://ex.org/p".to_string(), - 20, - ); + let later_pin = key("a:main", "http://ex.org/p", 20); assert!(check_cycle(&active, &later_pin).is_ok()); } @@ -828,12 +850,14 @@ mod tests { let payload = Arc::new(ResolvedGraph { model_ledger_id: resolution_key.1.clone(), graph_iri: resolution_key.2.clone(), - resolved_t: resolution_key.3, + resolved_t: resolution_key.3.t, + commit_id: None, artifact: GovernanceArtifact::PolicyRules(PolicyArtifactWire { origin: fluree_db_policy::WireOrigin { model_ledger_id: resolution_key.1.clone(), graph_iri: resolution_key.2.clone(), - resolved_t: resolution_key.3, + resolved_t: resolution_key.3.t, + commit_id: None, }, restrictions: vec![], }), diff --git a/fluree-db-api/src/lib.rs b/fluree-db-api/src/lib.rs index f76f61e112..e87d4c9a96 100644 --- a/fluree-db-api/src/lib.rs +++ b/fluree-db-api/src/lib.rs @@ -3520,10 +3520,10 @@ pub struct Fluree { leaflet_cache: std::sync::Arc, /// Per-instance cache for resolved cross-ledger governance /// artifacts (`f:policySource` from a model ledger, etc.). Keyed - /// on `(ArtifactKind, model_ledger_id, graph_iri, resolved_t)` + /// on `(ArtifactKind, model_ledger_id, graph_iri, model head)` /// so a single entry is reusable across every data ledger on /// this instance that references the same model graph at the - /// same model `t`. + /// same model head commit. governance_cache: std::sync::Arc, /// Indexing mode (disabled or background with handle) pub indexing_mode: tx::IndexingMode, diff --git a/fluree-db-api/src/tx.rs b/fluree-db-api/src/tx.rs index 9da3f82859..ac760d2fa3 100644 --- a/fluree-db-api/src/tx.rs +++ b/fluree-db-api/src/tx.rs @@ -851,7 +851,7 @@ pub(crate) fn namespace_prefix_map(ns_registry: &NamespaceRegistry) -> HashMap, }, } @@ -1197,6 +1203,7 @@ pub(crate) async fn apply_shacl_policy_to_staged_view( model_ledger_id: wire.origin.model_ledger_id.clone(), graph_iri: wire.origin.graph_iri.clone(), resolved_t: wire.origin.resolved_t, + commit_id: wire.origin.commit_id.clone(), }; precompiled = probe_shacl_compile_cache(base, &source); cache_source = Some(source); diff --git a/fluree-db-api/src/view/fluree_ext.rs b/fluree-db-api/src/view/fluree_ext.rs index 593dfa2164..e3a7a77daa 100644 --- a/fluree-db-api/src/view/fluree_ext.rs +++ b/fluree-db-api/src/view/fluree_ext.rs @@ -6,12 +6,15 @@ use fluree_db_core::VerifiedIdentity; use std::sync::Arc; use crate::view::{GraphDb, ReasoningModePrecedence}; -use crate::{config_resolver, time_resolve, ApiError, Fluree, GovernanceOptions, Result, TimeSpec}; +use crate::{ + config_resolver, time_resolve, ApiError, Fluree, GovernanceOptions, LedgerView, Result, + TimeSpec, +}; use fluree_db_binary_index::BinaryIndexStore; use fluree_db_core::ids::GraphId; use fluree_db_core::{ - ContentStore, DictNovelty, IndexType, CONFIG_GRAPH_ID, DEFAULT_GRAPH_ID, DEFAULT_GRAPH_IRI, - TXN_META_GRAPH_ID, + ContentId, ContentStore, DictNovelty, IndexType, CONFIG_GRAPH_ID, DEFAULT_GRAPH_ID, + DEFAULT_GRAPH_IRI, TXN_META_GRAPH_ID, }; use fluree_db_query::ir::ReasoningModes; use fluree_db_query::BinaryRangeProvider; @@ -34,6 +37,17 @@ enum GraphRef { Named(String), } +/// A view of a cached handle's head, with its binary store attached. +fn graph_db_from_cached_head(snap: LedgerView) -> GraphDb { + let binary_store = snap.binary_store.clone(); + let ledger = snap.to_ledger_state(); + let view = GraphDb::from_ledger_state(&ledger); + match binary_store { + Some(store) => view.with_binary_store(store), + None => view, + } +} + impl Fluree { /// Split a graph reference like `ledger:main#txn-meta` into (ledger_id, graph_ref). /// @@ -345,18 +359,34 @@ impl Fluree { let handle = self.ledger_cached(ledger_id).await?; let snap = handle.snapshot().await; if target_t == snap.t { - let binary_store = snap.binary_store.clone(); - let ledger = snap.to_ledger_state(); - let view = GraphDb::from_ledger_state(&ledger); - return Ok(match binary_store { - Some(store) => view.with_binary_store(store), - None => view, - }); + return Ok(graph_db_from_cached_head(snap)); } self.load_graph_db_historical(ledger_id, target_t).await } + /// Load a ledger at `t`, where its head commit is `commit_id`. + /// + /// Unlike [`Self::load_graph_db_at_t`], the cached handle serves it only + /// when its head is that commit. `t` alone does not name the history: an + /// instance sharing this one's storage can drop the ledger and create it + /// again, and a handle cached here before the drop sits at the same `t` + /// until it is reloaded. The historical loader reads the nameservice. + pub(crate) async fn load_graph_db_at_commit( + &self, + ledger_id: &str, + t: i64, + commit_id: Option<&ContentId>, + ) -> Result { + let handle = self.ledger_cached(ledger_id).await?; + let snap = handle.snapshot().await; + if t == snap.t && commit_id == snap.head_commit_id.as_ref() { + return Ok(graph_db_from_cached_head(snap)); + } + + self.load_graph_db_historical(ledger_id, t).await + } + /// Open a ledger at a past `t` through the historical loader: no ledger /// cache, so no cache lock is taken. [`Self::load_graph_db_at_t`] uses /// this once it knows `t` is not the cached head. @@ -852,17 +882,17 @@ impl Fluree { let source = source.expect("checked above"); // Seed from any prior governance-context capture stored // on the view (e.g., an earlier `wrap_policy` in the - // same logical request). The merged resolved_ts is + // same logical request). The merged resolved_heads is // written back below so the subsequent `query` call's - // own ResolveCtx observes the same per-ledger head-t. + // own ResolveCtx observes the same per-ledger head. // // `ledger_id_owned` keeps the id alive past the // eventual `view` move at the end of this branch. let ledger_id_owned = view.snapshot.ledger_id.clone(); - let mut ctx = crate::cross_ledger::ResolveCtx::with_resolved_ts( + let mut ctx = crate::cross_ledger::ResolveCtx::with_resolved_heads( &ledger_id_owned, self, - (**view.cross_ledger_resolved_ts()).clone(), + (**view.cross_ledger_resolved_heads()).clone(), ); // The class-filter chain, identity contract (bind-only, never a // rule selector), and PolicyRules dispatch all live in the shared @@ -932,7 +962,7 @@ impl Fluree { // logical request sees the same M version policy did. let view = view .with_policy(Arc::new(policy_ctx)) - .with_cross_ledger_resolved_ts(Arc::new(ctx.resolved_ts)); + .with_cross_ledger_resolved_heads(Arc::new(ctx.resolved_heads)); return Ok(view); } @@ -957,10 +987,10 @@ impl Fluree { { Some(reasoning) => { let ledger_id_owned = view.snapshot.ledger_id.clone(); - let mut ctx = crate::cross_ledger::ResolveCtx::with_resolved_ts( + let mut ctx = crate::cross_ledger::ResolveCtx::with_resolved_heads( &ledger_id_owned, self, - (**view.cross_ledger_resolved_ts()).clone(), + (**view.cross_ledger_resolved_heads()).clone(), ); crate::cross_ledger::resolve_schema_closure_bundle( reasoning, diff --git a/fluree-db-api/src/view/query.rs b/fluree-db-api/src/view/query.rs index 0fd24960ab..919e2f342d 100644 --- a/fluree-db-api/src/view/query.rs +++ b/fluree-db-api/src/view/query.rs @@ -1608,15 +1608,15 @@ impl Fluree { // could disagree if M advances between awaits — that breaks // the resolver's per-request consistency contract. // - // Seeded from `db.cross_ledger_resolved_ts` so a preceding + // Seeded from `db.cross_ledger_resolved_heads` so a preceding // `wrap_policy` call's captures carry forward: policy and // reasoning/rules on the same M must agree on which // version of M they're enforcing, even though they enter // through separate Rust API calls. - let mut ctx = crate::cross_ledger::ResolveCtx::with_resolved_ts( + let mut ctx = crate::cross_ledger::ResolveCtx::with_resolved_heads( &db.as_graph_db_ref().snapshot.ledger_id, self, - (**db.cross_ledger_resolved_ts()).clone(), + (**db.cross_ledger_resolved_heads()).clone(), ); // Cross-ledger `f:rulesSource`: when M is referenced via diff --git a/fluree-db-api/src/view/types.rs b/fluree-db-api/src/view/types.rs index 3a1c8dc659..e9b3f66ed9 100644 --- a/fluree-db-api/src/view/types.rs +++ b/fluree-db-api/src/view/types.rs @@ -229,15 +229,15 @@ pub struct GraphDb { /// /// `wrap_policy` and `query` are separate Rust API calls but /// model a single HTTP request; if both touch the same model - /// ledger M, they must observe the same `resolved_t` for M. + /// ledger M, they must observe the same head of M. /// Carrying this map on the view lets `wrap_policy` capture - /// M's head-t and the subsequent `query` reuse that capture + /// M's head and the subsequent `query` reuse that capture /// when building its own `ResolveCtx`. /// /// Empty by default. Cloned via `Arc` so policy-wrap doesn't /// inflate the cached `GraphDb` cost. - pub(crate) cross_ledger_resolved_ts: - Arc>, + pub(crate) cross_ledger_resolved_heads: + Arc>, // ======================================================================== // Graph source context (optional — set when view is created from a graph source) @@ -316,7 +316,7 @@ impl GraphDb { query_time_rules_allowed: true, datalog_override_allowed: true, rules_source_g_id: None, - cross_ledger_resolved_ts: Arc::new(std::collections::HashMap::new()), + cross_ledger_resolved_heads: Arc::new(std::collections::HashMap::new()), graph_source_id: None, graph_source_time: None, } @@ -722,20 +722,23 @@ impl GraphDb { } /// Replace the carried governance-context capture (per-ledger - /// `resolved_t`s seen so far in this logical request). - pub fn with_cross_ledger_resolved_ts( + /// model heads seen so far in this logical request). + pub fn with_cross_ledger_resolved_heads( mut self, - ts: Arc>, + heads: Arc< + std::collections::HashMap, + >, ) -> Self { - self.cross_ledger_resolved_ts = ts; + self.cross_ledger_resolved_heads = heads; self } /// Read the carried governance-context capture. - pub fn cross_ledger_resolved_ts( + pub fn cross_ledger_resolved_heads( &self, - ) -> &Arc> { - &self.cross_ledger_resolved_ts + ) -> &Arc> + { + &self.cross_ledger_resolved_heads } } diff --git a/fluree-db-api/tests/it_cross_ledger_resolver.rs b/fluree-db-api/tests/it_cross_ledger_resolver.rs index 1fc6bc4f19..c01cbdf425 100644 --- a/fluree-db-api/tests/it_cross_ledger_resolver.rs +++ b/fluree-db-api/tests/it_cross_ledger_resolver.rs @@ -759,15 +759,16 @@ async fn single_resolution_t_is_stable_within_a_request() { resolved_b.resolved_t, t_at_first_resolution, ); - // Verify resolved_ts cached the head exactly once. A new + // Verify resolved_heads cached the head exactly once. A new // request would re-capture against the advanced M; the cache // is per-request, not per-instance. - assert_eq!(ctx.resolved_ts.len(), 1); + assert_eq!(ctx.resolved_heads.len(), 1); assert_eq!( - ctx.resolved_ts - .get(&fluree_db_api::LedgerId::parse(model_id).unwrap()), - Some(&t_at_first_resolution), - "resolved_ts must store the captured t once per canonical model id" + ctx.resolved_heads + .get(&fluree_db_api::LedgerId::parse(model_id).unwrap()) + .map(|head| head.t), + Some(t_at_first_resolution), + "resolved_heads must store the captured head once per canonical model id" ); // Sanity: a new ResolveCtx against the same Fluree DOES re-capture @@ -975,3 +976,87 @@ async fn empty_policy_graph_yields_empty_wire_artifact() { wire.restrictions.len() ); } + +/// A model ledger dropped and created again under its name restarts at the +/// same `t`. Its head commit differs, so the resolution is a fresh one, never +/// the artifact cached for the dropped ledger at that `t`. +#[tokio::test] +async fn recreated_model_ledger_is_not_served_the_dropped_ones_artifact() { + let fluree = FlureeBuilder::memory().build_memory(); + let model_id = "test/cross-ledger/recreate-model:main"; + let graph = "http://example.org/recreate-policy"; + let policy = |rule: &str, allow: bool| { + format!( + r" + @prefix f: . + @prefix rdf: . + @prefix ex: . + + GRAPH <{graph}> {{ + ex:{rule} rdf:type f:AccessPolicy ; f:action f:view ; f:allow {allow} . + }} + " + ) + }; + let seed = |rule: &'static str, allow: bool| { + let fluree = &fluree; + let policy = &policy; + async move { + fluree + .stage_owned(genesis_ledger(fluree, model_id)) + .upsert_turtle(&policy(rule, allow)) + .execute() + .await + .expect("seed M"); + } + }; + + let data_id = "test/cross-ledger/recreate-data:main"; + let _ = genesis_ledger(&fluree, data_id); + let data = fluree_db_api::LedgerId::parse(data_id).unwrap(); + let graph_ref = cross_ref(model_id, graph); + let rules = |r: &fluree_db_api::cross_ledger::ResolvedGraph| match &r.artifact { + GovernanceArtifact::PolicyRules(wire) => wire + .restrictions + .iter() + .map(|x| x.id.clone()) + .collect::>(), + other => panic!("expected PolicyRules, got {other:?}"), + }; + + seed("allowRule", true).await; + let before = resolve_graph_ref( + &graph_ref, + ArtifactKind::PolicyRules, + &mut ResolveCtx::new(&data, &fluree), + ) + .await + .expect("resolve the first M"); + assert_eq!(rules(&before), ["http://example.org/ns/allowRule"]); + + fluree + .drop_ledger( + "test/cross-ledger/recreate-model", + fluree_db_api::DropMode::Hard, + ) + .await + .expect("drop M"); + seed("denyRule", false).await; + + let after = resolve_graph_ref( + &graph_ref, + ArtifactKind::PolicyRules, + &mut ResolveCtx::new(&data, &fluree), + ) + .await + .expect("resolve the recreated M"); + assert_eq!( + after.resolved_t, before.resolved_t, + "the recreated M is at the dropped M's t" + ); + assert!( + !std::sync::Arc::ptr_eq(&before, &after), + "the recreated M must not be served the dropped M's cached artifact" + ); + assert_eq!(rules(&after), ["http://example.org/ns/denyRule"]); +} diff --git a/fluree-db-api/tests/it_policy_cross_ledger.rs b/fluree-db-api/tests/it_policy_cross_ledger.rs index 7f1c494a67..27ffa7bca3 100644 --- a/fluree-db-api/tests/it_policy_cross_ledger.rs +++ b/fluree-db-api/tests/it_policy_cross_ledger.rs @@ -8,7 +8,7 @@ #![cfg(feature = "native")] -use crate::support::genesis_ledger; +use crate::support::{genesis_ledger, instances_sharing_storage}; use fluree_db_api::{ CommitOpts, FlureeBuilder, GovernanceOptions, IndexConfig, ReindexOptions, TxnOpts, }; @@ -1280,3 +1280,152 @@ async fn multiple_targeted_query_rules_any_allow_grants_across_materializations( ); } } + +const USER_POLICY_GRAPH: &str = "http://example.org/m-policies"; + +/// Commit model ledger `model`'s one policy, in a named graph: users are +/// visible only if `allow`. +async fn seed_user_policy(fluree: &fluree_db_api::Fluree, model: &str, allow: bool) { + fluree + .stage_owned(genesis_ledger(fluree, &format!("{model}:main"))) + .upsert_turtle(&format!( + r" + @prefix f: . + @prefix rdf: . + @prefix ex: . + + GRAPH <{USER_POLICY_GRAPH}> {{ + ex:users + rdf:type f:AccessPolicy ; + f:action f:view ; + f:onClass ex:User ; + f:allow {allow} . + }} + " + )) + .execute() + .await + .expect("seed M"); +} + +/// Drop model ledger `model` and create it again with the policy `allow` +/// picks. Its one commit puts the new ledger at the dropped one's `t`. +async fn recreate_user_policy(fluree: &fluree_db_api::Fluree, model: &str, allow: bool) { + fluree + .drop_ledger(model, fluree_db_api::DropMode::Hard) + .await + .expect("drop M"); + seed_user_policy(fluree, model, allow).await; +} + +/// Data ledger `data_id` holds one user, and enforces `model`'s user policy +/// with nothing allowed by default. +async fn seed_data_governed_by(fluree: &fluree_db_api::Fluree, data_id: &str, model: &str) { + let data = fluree + .insert( + genesis_ledger(fluree, data_id), + &json!({ + "@context": {"ex": "http://example.org/ns/"}, + "@id": "ex:alice", "@type": "ex:User", "ex:name": "Alice" + }), + ) + .await + .expect("seed D") + .ledger; + let config_iri = config_graph_iri(data_id); + fluree + .stage_owned(data) + .upsert_turtle(&format!( + r" + @prefix f: . + @prefix rdf: . + + GRAPH <{config_iri}> {{ + rdf:type f:LedgerConfig . + f:policyDefaults . + f:defaultAllow false . + f:policyClass f:AccessPolicy . + f:policySource . + rdf:type f:GraphRef ; + f:graphSource . + f:ledger <{model}:main> ; + f:graphSelector <{USER_POLICY_GRAPH}> . + }} + " + )) + .execute() + .await + .expect("seed D config"); +} + +/// The users `fluree`'s policy view of `data_id` shows. +async fn visible_users(fluree: &fluree_db_api::Fluree, data_id: &str) -> serde_json::Value { + let wrapped = fluree + .db_with_policy(data_id, &GovernanceOptions::default()) + .await + .expect("db_with_policy"); + fluree + .query( + &wrapped, + &json!({"@context": {"ex": "http://example.org/ns/"}, "select": "?u", + "where": {"@id": "?u", "@type": "ex:User"}}), + ) + .await + .expect("query") + .to_jsonld(&wrapped.snapshot) + .expect("jsonld") +} + +/// A model ledger dropped and created again under its name restarts at the +/// same `t`. The policies D enforces are the new ledger's, never the dropped +/// one's cached at that `t`. +#[tokio::test] +async fn recreated_model_ledger_policies_replace_the_dropped_ones() { + let fluree = FlureeBuilder::memory().build_memory(); + let model = "test/cross-ledger-recreate/model"; + let data_id = "test/cross-ledger-recreate/data:main"; + seed_user_policy(&fluree, model, false).await; + seed_data_governed_by(&fluree, data_id, model).await; + assert_eq!( + visible_users(&fluree, data_id).await, + json!([]), + "M denies users" + ); + + recreate_user_policy(&fluree, model, true).await; + assert_eq!( + visible_users(&fluree, data_id).await, + json!(["ex:alice"]), + "the new M, at the dropped M's t, allows users" + ); +} + +/// The same, with M recreated by another instance sharing the storage and +/// nameservice. Nothing clears this instance's cached handle of the dropped +/// M, which sits at the `t` the new M reaches; the policies are still read +/// from the new M. +#[tokio::test] +async fn model_ledger_recreated_by_another_instance_replaces_the_dropped_policies() { + let (writer, reader) = instances_sharing_storage(); + let model = "test/cross-ledger-recreate-shared/model"; + let data_id = "test/cross-ledger-recreate-shared/data:main"; + seed_user_policy(&writer, model, true).await; + seed_data_governed_by(&writer, data_id, model).await; + assert_eq!( + visible_users(&reader, data_id).await, + json!(["ex:alice"]), + "M allows users, and the reader caches M" + ); + + recreate_user_policy(&writer, model, false).await; + assert_eq!( + visible_users(&writer, data_id).await, + json!([]), + "the writer enforces the new M" + ); + assert_eq!( + visible_users(&reader, data_id).await, + json!([]), + "the reader enforces the new M, not its cached handle of the dropped one" + ); +} diff --git a/fluree-db-api/tests/it_rules_cross_ledger.rs b/fluree-db-api/tests/it_rules_cross_ledger.rs index b31e1a43e1..c31e8d89a8 100644 --- a/fluree-db-api/tests/it_rules_cross_ledger.rs +++ b/fluree-db-api/tests/it_rules_cross_ledger.rs @@ -233,6 +233,7 @@ fn parsed_rules_fails_closed_on_malformed_json() { model_ledger_id: "test/m:main".into(), graph_iri: "http://example.org/rules".into(), resolved_t: 1, + commit_id: None, }, rules: vec!["{ valid: false".into()], }; diff --git a/fluree-db-api/tests/it_shapes_cross_ledger.rs b/fluree-db-api/tests/it_shapes_cross_ledger.rs index 960d6431b2..de89968669 100644 --- a/fluree-db-api/tests/it_shapes_cross_ledger.rs +++ b/fluree-db-api/tests/it_shapes_cross_ledger.rs @@ -12,7 +12,7 @@ #![cfg(all(feature = "native", feature = "shacl"))] -use crate::support::genesis_ledger; +use crate::support::{genesis_ledger, instances_sharing_storage}; use fluree_db_api::FlureeBuilder; use serde_json::json; @@ -426,11 +426,21 @@ async fn seed_cross_ledger_person_shape( .execute() .await .expect("seed M shapes"); + seed_shapes_source_config(fluree, data_id, model_id, extra_shacl_config).await +} - let data = genesis_ledger(fluree, data_id); +/// Data ledger D's `#config`: SHACL on, with `f:shapesSource` pointing at +/// model ledger M's shapes graph. Returns D's post-config `LedgerState`. +async fn seed_shapes_source_config( + fluree: &fluree_db_api::Fluree, + data_id: &str, + model_id: &str, + extra_shacl_config: &str, +) -> fluree_db_api::LedgerState { + let shapes_graph_iri = "http://example.org/governance/shapes"; let config_iri = config_graph_iri(data_id); let r = fluree - .stage_owned(data) + .stage_owned(genesis_ledger(fluree, data_id)) .upsert_turtle(&format!( r" @prefix f: . @@ -808,6 +818,231 @@ async fn model_head_advance_recompiles_shapes() { .expect("deactivated shape must no longer reject"); } +/// A model ledger dropped and created again under its name restarts at the +/// same `t`. D's compiled shapes are cached by the wire's origin, so a +/// recreated M with other shapes must still recompile. +#[tokio::test] +async fn recreated_model_ledger_shapes_replace_the_cached_compile() { + let fluree = FlureeBuilder::memory().build_memory(); + let model_id = "test/cross-ledger-shapes/recreate-model:main"; + let data_id = "test/cross-ledger-shapes/recreate-data:main"; + let data = seed_cross_ledger_person_shape(&fluree, model_id, data_id, "").await; + + // Conforming write first: an empty namespace delta afterwards makes the + // next transactions eligible for the compile cache. + fluree + .insert( + data, + &json!({ + "@context": {"ex": "http://example.org/ns/"}, + "@id": "ex:alice", + "@type": "ex:Person", + "ex:name": "Alice" + }), + ) + .await + .expect("conforming Person must pass"); + let nameless_bob = || async { + fluree + .graph(data_id) + .transact() + .insert(&json!({ + "@context": {"ex": "http://example.org/ns/"}, + "@id": "ex:bob", + "@type": "ex:Person" + })) + .commit() + .await + }; + let err = nameless_bob() + .await + .expect_err("M requires a name: compiled and cached"); + assert!( + err.to_string().contains("SHACL") || format!("{err:?}").contains("ShaclViolation"), + "expected ShaclViolation, got: {err:?}" + ); + + // The new M, at the dropped M's t, deactivates the shape. + fluree + .drop_ledger( + "test/cross-ledger-shapes/recreate-model", + fluree_db_api::DropMode::Hard, + ) + .await + .expect("drop M"); + seed_person_name_shape(&fluree, model_id, false).await; + + nameless_bob() + .await + .expect("the new M's deactivated shape must no longer reject"); +} + +/// Commit model ledger M: a Person shape requiring a name, active or not. +async fn seed_person_name_shape(fluree: &fluree_db_api::Fluree, model_id: &str, active: bool) { + let deactivated = !active; + fluree + .stage_owned(genesis_ledger(fluree, model_id)) + .upsert_turtle(&format!( + r" + @prefix sh: . + @prefix rdf: . + @prefix ex: . + GRAPH {{ + ex:PersonShape rdf:type sh:NodeShape ; + sh:targetClass ex:Person ; + sh:property ex:pshape_name ; + sh:deactivated {deactivated} . + ex:pshape_name sh:path ex:name ; sh:minCount 1 . + }} + " + )) + .execute() + .await + .expect("seed M"); +} + +/// The reverse: a recreated M that rejects what the dropped one let through. +/// The compiled shapes cached from the dropped M must not keep passing it. +#[tokio::test] +async fn recreated_model_ledger_shapes_reject_what_the_cached_compile_passed() { + let fluree = FlureeBuilder::memory().build_memory(); + let model_id = "test/cross-ledger-shapes/recreate-strict-model:main"; + let data_id = "test/cross-ledger-shapes/recreate-strict-data:main"; + seed_person_name_shape(&fluree, model_id, false).await; + let data = seed_shapes_source_config(&fluree, data_id, model_id, "").await; + + // Conforming write first: an empty namespace delta afterwards makes the + // next transactions eligible for the compile cache. + fluree + .insert( + data, + &json!({ + "@context": {"ex": "http://example.org/ns/"}, + "@id": "ex:alice", + "@type": "ex:Person", + "ex:name": "Alice" + }), + ) + .await + .expect("conforming Person must pass"); + let nameless = |person: &'static str| { + let fluree = &fluree; + async move { + fluree + .graph(data_id) + .transact() + .insert(&json!({ + "@context": {"ex": "http://example.org/ns/"}, + "@id": person, + "@type": "ex:Person" + })) + .commit() + .await + } + }; + nameless("ex:bob") + .await + .expect("M's shape is deactivated: compiled and cached"); + + fluree + .drop_ledger( + "test/cross-ledger-shapes/recreate-strict-model", + fluree_db_api::DropMode::Hard, + ) + .await + .expect("drop M"); + seed_person_name_shape(&fluree, model_id, true).await; + + let err = nameless("ex:carol") + .await + .expect_err("the new M, at the dropped M's t, requires a name"); + assert!( + err.to_string().contains("SHACL") || format!("{err:?}").contains("ShaclViolation"), + "expected ShaclViolation, got: {err:?}" + ); +} + +const US_STATE_SHAPES_GRAPH: &str = "http://example.org/governance/shapes"; + +/// Commit model ledger `model`: a shape whose `ex:homeState` must be an +/// `ex:USState`, and the vocabulary of `states`, in one graph. +async fn seed_us_state_model(fluree: &fluree_db_api::Fluree, model: &str, states: &[&str]) { + let members: String = states + .iter() + .map(|state| format!("ex:{state} rdf:type ex:USState .\n")) + .collect(); + fluree + .stage_owned(genesis_ledger(fluree, &format!("{model}:main"))) + .upsert_turtle(&format!( + r" + @prefix sh: . + @prefix rdf: . + @prefix ex: . + + GRAPH <{US_STATE_SHAPES_GRAPH}> {{ + ex:PersonShape rdf:type sh:NodeShape ; + sh:targetClass ex:Person ; + sh:property ex:pshape_state . + ex:pshape_state sh:path ex:homeState ; sh:class ex:USState . + {members} + }} + " + )) + .execute() + .await + .expect("seed M"); +} + +/// `sh:class` membership is read from M itself, not from the shapes wire. +/// With M recreated by another instance sharing the storage and nameservice, +/// nothing clears this instance's cached handle of the dropped M, which sits +/// at the `t` the new M reaches. Membership still comes from the new M. +#[tokio::test] +async fn sh_class_membership_follows_a_model_recreated_by_another_instance() { + let (writer, reader) = instances_sharing_storage(); + let model = "test/cross-ledger-shapes/recreate-vocab-model"; + let data_id = "test/cross-ledger-shapes/recreate-vocab-data:main"; + seed_us_state_model(&writer, model, &["illinois", "iowa"]).await; + seed_shapes_source_config(&writer, data_id, &format!("{model}:main"), "").await; + + let from_illinois = |person: &'static str| { + let reader = &reader; + async move { + reader + .graph(data_id) + .transact() + .insert(&json!({ + "@context": {"ex": "http://example.org/ns/"}, + "@id": person, + "@type": "ex:Person", + "ex:homeState": {"@id": "ex:illinois"} + })) + .commit() + .await + } + }; + from_illinois("ex:alice") + .await + .expect("M lists Illinois, and the reader caches M"); + + writer + .drop_ledger(model, fluree_db_api::DropMode::Hard) + .await + .expect("drop M"); + seed_us_state_model(&writer, model, &["iowa"]).await; + + let err = from_illinois("ex:bob") + .await + .expect_err("the new M does not list Illinois"); + assert!( + matches!( + err, + fluree_db_api::ApiError::Transact(fluree_db_transact::TransactError::ShaclViolation(_)) + ), + "expected ShaclViolation, got: {err:?}" + ); +} + /// A `f:shapesSource` whose `f:ledger` names the data ledger itself is a /// same-ledger reference in cross-ledger clothing. A write that stages under /// the ledger's write lock (a SPARQL update always does) then opened the diff --git a/fluree-db-api/tests/support/mod.rs b/fluree-db-api/tests/support/mod.rs index e176777a33..2d47af2779 100644 --- a/fluree-db-api/tests/support/mod.rs +++ b/fluree-db-api/tests/support/mod.rs @@ -264,6 +264,21 @@ pub fn genesis_ledger_for_fluree(_fluree: &fluree_db_api::Fluree, ledger_id: &st LedgerState::new(db, Novelty::new(0)) } +/// Two instances over one storage and one nameservice, each with its own +/// ledger cache: the shape of several servers sharing S3 and DynamoDB. A +/// change one makes reaches the other only through storage. +pub fn instances_sharing_storage() -> (MemoryFluree, MemoryFluree) { + let storage = fluree_db_core::MemoryStorage::new(); + let nameservice = Arc::new(fluree_db_nameservice::memory::MemoryNameService::new()); + let instance = || { + fluree_db_api::FlureeBuilder::memory().build_with( + storage.clone(), + fluree_db_api::NameServiceMode::ReadWrite(nameservice.clone()), + ) + }; + (instance(), instance()) +} + // ============================================================================= // Common seeding helpers // ============================================================================= diff --git a/fluree-db-policy/src/wire.rs b/fluree-db-policy/src/wire.rs index 89da564b45..73cfff8e02 100644 --- a/fluree-db-policy/src/wire.rs +++ b/fluree-db-policy/src/wire.rs @@ -25,7 +25,7 @@ use std::collections::HashSet; /// Term-neutral policy artifact materialized from a model ledger. /// /// Cached by the API layer keyed on `(origin.model_ledger_id, -/// origin.graph_iri, origin.resolved_t)`. +/// origin.graph_iri, origin.resolved_t, origin.commit_id)`. #[derive(Debug, Clone)] pub struct PolicyArtifactWire { /// Provenance for diagnostics and cache key derivation. @@ -45,6 +45,8 @@ pub struct WireOrigin { pub graph_iri: String, /// Model ledger `t` at which the artifact was materialized. pub resolved_t: i64, + /// Model ledger head commit at `resolved_t`. + pub commit_id: Option, } /// IRI-form mirror of [`PolicyRestriction`]. @@ -278,6 +280,7 @@ mod tests { model_ledger_id: "test/model:main".into(), graph_iri: "http://example.org/policy".into(), resolved_t: 42, + commit_id: None, } }