diff --git a/docs/concepts/datasets-and-named-graphs.md b/docs/concepts/datasets-and-named-graphs.md index 378757daf8..ee54152019 100644 --- a/docs/concepts/datasets-and-named-graphs.md +++ b/docs/concepts/datasets-and-named-graphs.md @@ -332,7 +332,7 @@ Two CLI commands cover the rest of the lifecycle: - **[`fluree graph list`](../cli/graph.md#fluree-graph-list)** — lists user graphs registered on a branch (with `--include-system` to also show the default and system graphs). Reads the `named-graphs` section of the standard `/info` response. - **[`fluree graph drop`](../cli/graph.md#fluree-graph-drop)** — transactionally retracts every triple currently asserted under a named graph. Produces one new commit at `t + 1` whose flakes are all retractions; history at older `t` values is preserved, and the graph IRI keeps its `g_id` so future inserts land in the same slot. Drops are per-branch. -The default graph, `urn:fluree:{ledger_id}#txn-meta`, and `urn:fluree:{ledger_id}#config` cannot be dropped. The Rust API entry point is `Fluree::drop_named_graph(ledger_id, graph_iri)`; over HTTP it is `POST /v1/fluree/drop-graph` (admin-protected). See the [server-integration contract](../cli/server-integration.md#drop-named-graph-contract) for the wire details. +The default graph, `urn:fluree:{ledger_id}#txn-meta`, and `urn:fluree:{ledger_id}#config` cannot be dropped. The Rust API entry point is `Fluree::drop_named_graph(ledger_id, graph_iri)`, whose report carries the drop commit's `commit_id` when it committed; over HTTP it is `POST /v1/fluree/drop-graph` (admin-protected). See the [server-integration contract](../cli/server-integration.md#drop-named-graph-contract) for the wire details. ### Graph Metadata diff --git a/docs/getting-started/rust-api.md b/docs/getting-started/rust-api.md index 57a4b475e3..df86745dba 100644 --- a/docs/getting-started/rust-api.md +++ b/docs/getting-started/rust-api.md @@ -393,6 +393,44 @@ async fn main() -> Result<()> { } ``` +#### Parameters + +`params` binds variables to values: each named variable (`?name` or `$name`, +the same variable) is replaced by its value wherever it appears — subqueries, +`OPTIONAL`, `FILTER`s and update templates included — before the query is +planned, so it runs exactly as if the value had been written inline. Values +take the JSON-LD forms: a JSON string, number or boolean; `{"@id": iri}`; +`{"@value": v, "@type": datatype}` (`"@type": "@id"` reads `v` as an IRI); +`{"@value": s, "@language": tag}`; and an embedding vector as its numbers, +`{"@value": [0.1, 0.2], "@type": "https://ns.flur.ee/db#embeddingVector"}`. +IRIs are full IRIs — neither the query's `PREFIX`es nor a `@context` apply. A +blank node is the `_:fdb-…` id a query returned for a stored node; any other +label is refused, since written inline it would be a variable and match every +node. + +```rust +let params = json!({ "name": "Alice", "min": 21 }); +let result = fluree.graph("mydb:main") + .query() + .sparql("PREFIX schema: + SELECT ?person WHERE { ?person schema:name $name ; schema:age ?age FILTER(?age > $min) }") + .params(params.as_object().unwrap().clone()) + .execute_formatted() + .await?; +``` + +A projected parameter stays a column holding its value. A parameter the query +never mentions is an error — its misspelt variable would otherwise be unbound +and match everything — as is one the query assigns itself (`BIND`, `VALUES`, +`AS`), one used inside a remote `SERVICE` (whose body is sent as written), and +any parameter on a JSON-LD query. Without parameters nothing changes: the +query is not rewritten at all. + +The same parameters go on `QueryExecutionOptions::with_params`, +`Fluree::explain_sparql_with_params`, a transact builder's +`sparql_update_with_params(sparql, ¶ms)`, and +`TxnOperation::SparqlUpdate(sparql, Some(params))`. + ### Streaming Query Results (NDJSON) The buffered `.query()` paths above collect the whole result set into a @@ -695,6 +733,7 @@ async fn main() -> Result<()> { DELETE { ex:alice ex:age ?a } INSERT { ex:alice ex:age ?b } \ WHERE { ex:alice ex:age ?a BIND(?a + 1 AS ?b) }" .into(), + None, // parameters )) .await?; diff --git a/docs/guides/cookbook-search.md b/docs/guides/cookbook-search.md index 889649b57d..3452b4f01a 100644 --- a/docs/guides/cookbook-search.md +++ b/docs/guides/cookbook-search.md @@ -83,6 +83,20 @@ fluree query '{ }' ``` +The same search in SPARQL: + +```bash +fluree query ' +PREFIX ex: +SELECT ?title ?score WHERE { + ?doc a ex:Article ; ex:body ?body ; ex:title ?title . + BIND(fulltext(?body, "graph database relationships") AS ?score) + FILTER(?score > 0) +} +ORDER BY DESC(?score) +LIMIT 10' +``` + The `fulltext()` function returns a BM25 relevance score. Higher scores mean better matches. Documents with none of the search terms score 0. ### 3. Combine search with graph filters diff --git a/docs/indexing-and-search/README.md b/docs/indexing-and-search/README.md index ce2af900c1..66af530bb9 100644 --- a/docs/indexing-and-search/README.md +++ b/docs/indexing-and-search/README.md @@ -126,6 +126,17 @@ For small-to-medium corpora (up to hundreds of thousands of documents per predic } ``` +**Or in SPARQL:** +```sparql +SELECT ?title ?score WHERE { + ?doc ex:content ?content ; ex:title ?title . + BIND(fulltext(?content, "Rust programming") AS ?score) + FILTER(?score > 0) +} +ORDER BY DESC(?score) +LIMIT 10 +``` + See [Inline Fulltext Search](fulltext.md) for details. ## Full-Text Search (BM25 Graph Source) diff --git a/docs/indexing-and-search/fulltext.md b/docs/indexing-and-search/fulltext.md index 0245aa343e..e394c27a84 100644 --- a/docs/indexing-and-search/fulltext.md +++ b/docs/indexing-and-search/fulltext.md @@ -416,7 +416,7 @@ Each language becomes its own arena; queries automatically look up the right one ## The `fulltext()` Scoring Function -The `fulltext()` function computes a BM25 relevance score for a bound text value against a query string. Use it in `bind` expressions within JSON-LD queries. +The `fulltext()` function computes a BM25 relevance score for a bound text value against a query string. Use it in `bind` expressions in JSON-LD queries, or in `BIND` in SPARQL (see [SPARQL Support](#sparql-support)). ### Basic usage @@ -698,29 +698,28 @@ Expected results (ordered by relevance): ### Inserting data -Fulltext annotation works in SPARQL UPDATE today using the `^^f:fullText` typed literal syntax (see the Turtle/SPARQL insertion examples above). +Annotate values in SPARQL UPDATE (and Turtle) with the `^^f:fullText` typed literal, as in the insertion examples above, or configure the property with [`f:fullTextDefaults`](#configured-full-text-properties-ffulltextdefaults) and insert plain strings. ### Querying -The `fulltext()` scoring function is currently available in **JSON-LD Query only**. SPARQL query support is planned for a future release, with anticipated syntax like: +`fulltext()` is a SPARQL function, used in `BIND` like the vector similarity functions: ```sparql PREFIX ex: -PREFIX f: SELECT ?title ?score WHERE { ?doc a ex:Article ; ex:content ?content ; ex:title ?title . - BIND(f:fulltext(?content, "Rust programming") AS ?score) + BIND(fulltext(?content, "Rust programming") AS ?score) FILTER(?score > 0) } ORDER BY DESC(?score) LIMIT 10 ``` -This mirrors the pattern established by inline vector similarity functions (`dotProduct`, `cosineSimilarity`, `euclideanDistance`), which also support JSON-LD Query today with SPARQL planned. +It scores exactly as the JSON-LD form does, over indexed and unindexed documents alike, for both `@fulltext` values and `f:fullTextDefaults` properties. The name is case-insensitive (`fulltext`, `FULLTEXT`, `full_text`), and the namespaced `f:fulltext(...)` (``) is the same function. ## Related Documentation diff --git a/docs/indexing-and-search/vector-search.md b/docs/indexing-and-search/vector-search.md index 8deab68faf..176575caf2 100644 --- a/docs/indexing-and-search/vector-search.md +++ b/docs/indexing-and-search/vector-search.md @@ -135,9 +135,9 @@ Or with the full IRI: ] ``` -## Inline Similarity Functions (JSON-LD Query) +## Inline Similarity Functions -Fluree provides three vector similarity functions that can be used in `bind` expressions within JSON-LD queries. These compute similarity scores directly during query execution without requiring a pre-built index. +Fluree provides three vector similarity functions for `bind` expressions in JSON-LD queries and `BIND` in SPARQL ([SPARQL support](#sparql-support)). These compute similarity scores directly during query execution without requiring a pre-built index. Function names are case-insensitive; `dotProduct`, `dotproduct`, and `dot_product` are all equivalent. diff --git a/docs/query/sparql.md b/docs/query/sparql.md index 796a2d469a..2663bf6760 100644 --- a/docs/query/sparql.md +++ b/docs/query/sparql.md @@ -376,6 +376,26 @@ WHERE { } ``` +### Parameters + +When Fluree is embedded (the Rust API and the Python package), a query or +update can take parameters: values bound to variables by name. A parameter +replaces its variable — written `?name` or `$name`, which SPARQL treats as the +same variable — everywhere in the request before it is planned, so +`SELECT ?person WHERE { ?person ex:name $name }` with `name = "Alice"` runs +exactly as `SELECT ?person WHERE { ?person ex:name "Alice" }`. Unlike a +`VALUES` block, the value is visible inside every `FILTER`, `OPTIONAL` and +subquery. + +- A projected parameter (`SELECT $name ?age`) is still a column, holding the + value; `SELECT *` does not list it. +- A parameter that the request never mentions is an error, since a misspelt + variable would otherwise be unbound and match everything. So is a parameter + the request assigns itself (`BIND`, `VALUES`, `AS`), and one used inside a + remote `SERVICE`, whose body is sent to the endpoint as written. +- `INSERT DATA` and `DELETE DATA` take no variables, so no parameters; use + `INSERT { ... } WHERE { ... }`. + ## Aggregation ### GROUP BY @@ -755,6 +775,33 @@ Per W3C SPARQL 1.1 §17.5, XSD constructor functions cast values between datatyp - `ISLITERAL(?x)` - Is literal - `ISNUMERIC(?x)` - Is numeric +### Search and Similarity Functions + +Fluree extensions that score values for ranking, used in `BIND`: + +- `fulltext(?text, "query")` - BM25 relevance of a full-text value to a query string; `0` when no query term occurs. Also written `f:fulltext(...)`. See [Inline Fulltext Search](../indexing-and-search/fulltext.md). +- `dotProduct(?v1, ?v2)` - Dot product of two vectors +- `cosineSimilarity(?v1, ?v2)` - Cosine similarity of two vectors +- `euclideanDistance(?v1, ?v2)` - Euclidean (L2) distance between two vectors + +Vectors are `f:embeddingVector` literals (`"[0.1, 0.2]"^^f:embeddingVector`); see [Vector Search](../indexing-and-search/vector-search.md). Names are case-insensitive and accept underscores (`cosine_similarity`). + +```sparql +PREFIX ex: +PREFIX f: + +SELECT ?doc ?text ?similar WHERE { + VALUES ?q { "[0.88, 0.12, 0.08]"^^f:embeddingVector } + ?doc ex:body ?body ; + ex:embedding ?vec . + BIND(fulltext(?body, "graph databases") AS ?text) + BIND(cosineSimilarity(?vec, ?q) AS ?similar) + FILTER(?text > 0 || ?similar > 0.8) +} +ORDER BY DESC(0.6 * ?text + 0.4 * ?similar) +LIMIT 10 +``` + ## Subqueries Nest queries: diff --git a/docs/transactions/sync.md b/docs/transactions/sync.md index c45b4e789f..2103835ed4 100644 --- a/docs/transactions/sync.md +++ b/docs/transactions/sync.md @@ -123,7 +123,9 @@ let report = fluree assert!(report.committed || (report.asserted == 0 && report.retracted == 0)); ``` -`SyncGraphOpts { dry_run, allow_empty }` mirror the query parameters; +`SyncGraphOpts { dry_run, allow_empty }` mirror the query parameters, and +`message` (Rust API only) is recorded as the commit's `f:message`. When the +sync committed, the report's `commit_id` names the new commit. `sync_named_graph_with` additionally takes explicit `TxnOpts` and a `PolicyContext`, and `sync_named_graph_rdf_with` is the same call for Turtle / N-Triples / TriG text. `sync_graph_with(ledger, &GraphSel, GraphPayload, …)` diff --git a/fluree-db-api/src/admin.rs b/fluree-db-api/src/admin.rs index b980ac16de..bc6c11553f 100644 --- a/fluree-db-api/src/admin.rs +++ b/fluree-db-api/src/admin.rs @@ -142,6 +142,8 @@ pub struct DropNamedGraphReport { /// Current commit `t` for the branch after the drop. Equal to the /// pre-drop `t` when `committed = false`. pub t: i64, + /// The drop commit's id; `None` when `committed = false`. + pub commit_id: Option, } /// Options for [`Fluree::sync_named_graph`]. @@ -153,6 +155,8 @@ pub struct SyncGraphOpts { /// graph. Off by default so a truncated or accidentally-empty export /// cannot silently wipe the graph. pub allow_empty: bool, + /// Recorded as the commit's `f:message`. + pub message: Option, } /// Report of a [`Fluree::sync_named_graph`] call. @@ -176,6 +180,8 @@ pub struct SyncGraphReport { /// Whether a new commit was created. `false` when the payload matched /// the graph exactly, and always `false` for a dry run. pub committed: bool, + /// The new commit, when `committed`. + pub commit_id: Option, /// Whether this was a dry run (staged and counted, nothing committed). pub dry_run: bool, /// Current commit `t` for the branch after the call. Equal to the @@ -1025,6 +1031,7 @@ impl crate::Fluree { retracted, committed, t: new_t, + commit_id: committed.then(|| result.receipt.commit_id.clone()), }) } @@ -1167,15 +1174,25 @@ impl crate::Fluree { asserted, retracted, committed: false, + commit_id: None, dry_run: true, t: staged_against_t, }); } + let mut commit_opts = fluree_db_transact::CommitOpts::default(); + if let Some(message) = opts.message { + commit_opts = commit_opts.with_txn_meta(vec![fluree_db_core::TxnMetaEntry::new( + fluree_vocab::namespaces::FLUREE_DB, + "message", + fluree_db_core::TxnMetaValue::string(message), + )]); + } let mut builder = self .stage(&handle) .sync_graph_payload(graph.clone(), payload, opts.allow_empty) - .txn_opts(txn_opts); + .txn_opts(txn_opts) + .commit_opts(commit_opts); if let Some(policy) = policy { builder = builder.policy(policy); } @@ -1209,6 +1226,7 @@ impl crate::Fluree { asserted: result.receipt.assert_count, retracted: result.receipt.retract_count, committed, + commit_id: committed.then_some(result.receipt.commit_id), dry_run: false, t, }) diff --git a/fluree-db-api/src/branch_validation.rs b/fluree-db-api/src/branch_validation.rs index 35c496b55f..9d1d9623e9 100644 --- a/fluree-db-api/src/branch_validation.rs +++ b/fluree-db-api/src/branch_validation.rs @@ -28,7 +28,7 @@ use std::collections::{BTreeSet, HashMap}; #[derive(Clone, Debug, Default)] pub(crate) struct BranchOpValidation { /// Formatted violation report, `None` when the view conforms. - pub(crate) report: Option, + pub(crate) report: Option, } impl BranchOpValidation { @@ -49,7 +49,10 @@ impl BranchOpValidation { None => Ok(()), #[cfg(feature = "shacl")] Some(report) => { - Err(fluree_db_transact::TransactError::ShaclViolation(describe(report)).into()) + let message = describe(report.message().to_string()); + let report = + fluree_db_transact::ShaclViolations::new(message, report.results().to_vec()); + Err(fluree_db_transact::TransactError::ShaclViolation(report).into()) } // Without the feature no validator runs, so no report is ever // produced; keep the match total without naming a variant that @@ -57,7 +60,7 @@ impl BranchOpValidation { #[cfg(not(feature = "shacl"))] Some(report) => Err(crate::error::ApiError::internal(format!( "SHACL violation reported without the shacl feature: {}", - describe(report) + describe(report.to_string()) ))), } } diff --git a/fluree-db-api/src/explain.rs b/fluree-db-api/src/explain.rs index 6fe5ea66ae..e51babd3b5 100644 --- a/fluree-db-api/src/explain.rs +++ b/fluree-db-api/src/explain.rs @@ -719,7 +719,7 @@ pub async fn explain_sparql( snapshot: &fluree_db_core::LedgerSnapshot, sparql: &str, ) -> Result { - explain_sparql_for_view(snapshot, sparql, None, false) + explain_sparql_for_view(snapshot, sparql, None, None, false) } /// Explain an openCypher query against a LedgerSnapshot. @@ -772,16 +772,17 @@ pub async fn explain_sparql_with_default_context( sparql: &str, default_context: Option<&JsonValue>, ) -> Result { - explain_sparql_for_view(snapshot, sparql, default_context, false) + explain_sparql_for_view(snapshot, sparql, default_context, None, false) } pub(crate) fn explain_sparql_for_view( snapshot: &fluree_db_core::LedgerSnapshot, sparql: &str, default_context: Option<&JsonValue>, + params: Option<&fluree_db_sparql::ParamMap>, allow_semantic_elision: bool, ) -> Result { - let (vars, parsed) = parse_sparql_to_ir(sparql, snapshot, default_context)?; + let (vars, parsed) = parse_sparql_to_ir(sparql, snapshot, default_context, params)?; explain_from_parsed( snapshot, diff --git a/fluree-db-api/src/graph_query_builder.rs b/fluree-db-api/src/graph_query_builder.rs index cc071c88d3..2dba0cec0d 100644 --- a/fluree-db-api/src/graph_query_builder.rs +++ b/fluree-db-api/src/graph_query_builder.rs @@ -96,6 +96,12 @@ impl<'a, 'g> GraphQueryBuilder<'a, 'g> { self } + /// Bind SPARQL variables to values; see [`QueryExecutionOptions::with_params`]. + pub fn params(mut self, params: fluree_db_sparql::ParamMap) -> Self { + self.core.set_params(params); + self + } + /// Set query execution controls. pub fn execution_options(mut self, options: QueryExecutionOptions) -> Self { self.core.set_execution_options(options); @@ -464,6 +470,12 @@ impl<'a: 'v, 'v> GraphSnapshotQueryBuilder<'a, 'v> { self } + /// Bind SPARQL variables to values; see [`QueryExecutionOptions::with_params`]. + pub fn params(mut self, params: fluree_db_sparql::ParamMap) -> Self { + self.core.set_params(params); + self + } + /// Set query execution controls. pub fn execution_options(mut self, options: QueryExecutionOptions) -> Self { self.core.set_execution_options(options); diff --git a/fluree-db-api/src/graph_transact_builder.rs b/fluree-db-api/src/graph_transact_builder.rs index 921eb63124..0bd5c00739 100644 --- a/fluree-db-api/src/graph_transact_builder.rs +++ b/fluree-db-api/src/graph_transact_builder.rs @@ -97,7 +97,18 @@ impl<'a, 'g> GraphTransactBuilder<'a, 'g> { /// registry. This avoids the namespace-conflict retry that pre-lowering /// against an unlocked snapshot would require. pub fn sparql_update(mut self, sparql: &'g str) -> Self { - self.core.set_sparql_update(sparql); + self.core.set_sparql_update(sparql, None); + self + } + + /// [`Self::sparql_update`] with variables bound to values; see + /// [`fluree_db_sparql::substitute_params`]. + pub fn sparql_update_with_params( + mut self, + sparql: &'g str, + params: &'g fluree_db_sparql::ParamMap, + ) -> Self { + self.core.set_sparql_update(sparql, Some(params)); self } diff --git a/fluree-db-api/src/lib.rs b/fluree-db-api/src/lib.rs index 1d8bb98201..23ca0ee2e5 100644 --- a/fluree-db-api/src/lib.rs +++ b/fluree-db-api/src/lib.rs @@ -365,7 +365,7 @@ pub use fluree_db_transact::{ // Re-export SPARQL types (product feature; always enabled) pub use fluree_db_sparql::{ lower_sparql, parse_sparql, validate as validate_sparql, Capabilities as SparqlCapabilities, - Diagnostic as SparqlDiagnostic, LowerError as SparqlLowerError, + Diagnostic as SparqlDiagnostic, LowerError as SparqlLowerError, ParamMap as SparqlParamMap, ParseOutput as SparqlParseOutput, Prologue as SparqlPrologue, QueryBody as SparqlQueryBody, Severity as SparqlSeverity, SourceSpan as SparqlSourceSpan, SparqlAst, UpdateOperation as SparqlUpdateOperation, diff --git a/fluree-db-api/src/merge_preview.rs b/fluree-db-api/src/merge_preview.rs index f1abb19c3f..9bd31dcd30 100644 --- a/fluree-db-api/src/merge_preview.rs +++ b/fluree-db-api/src/merge_preview.rs @@ -717,7 +717,7 @@ impl crate::Fluree { .await?; Some(ValidationSummary { conforms: outcome.conforms(), - report: outcome.report, + report: outcome.report.map(|report| report.to_string()), }) } else { None diff --git a/fluree-db-api/src/query/builder.rs b/fluree-db-api/src/query/builder.rs index d7aea3f72a..28a2f7ca5f 100644 --- a/fluree-db-api/src/query/builder.rs +++ b/fluree-db-api/src/query/builder.rs @@ -124,6 +124,10 @@ impl<'a> QueryCore<'a> { self.execution = options; } + pub(crate) fn set_params(&mut self, params: fluree_db_sparql::ParamMap) { + self.execution = std::mem::take(&mut self.execution).with_params(params); + } + #[cfg(feature = "iceberg")] pub(crate) fn set_r2rml(&mut self) { self.graph_sources = GraphSourceMode::R2rml; @@ -234,6 +238,12 @@ impl<'a> ViewQueryBuilder<'a> { self } + /// Bind SPARQL variables to values; see [`QueryExecutionOptions::with_params`]. + pub fn params(mut self, params: fluree_db_sparql::ParamMap) -> Self { + self.core.set_params(params); + self + } + /// Set query execution controls. pub fn execution_options(mut self, options: QueryExecutionOptions) -> Self { self.core.set_execution_options(options); @@ -509,6 +519,12 @@ impl<'a> DatasetQueryBuilder<'a> { self } + /// Bind SPARQL variables to values; see [`QueryExecutionOptions::with_params`]. + pub fn params(mut self, params: fluree_db_sparql::ParamMap) -> Self { + self.core.set_params(params); + self + } + /// Set query execution controls. pub fn execution_options(mut self, options: QueryExecutionOptions) -> Self { self.core.set_execution_options(options); @@ -871,6 +887,12 @@ impl<'a> FromQueryBuilder<'a> { self } + /// Bind SPARQL variables to values; see [`QueryExecutionOptions::with_params`]. + pub fn params(mut self, params: fluree_db_sparql::ParamMap) -> Self { + self.core.set_params(params); + self + } + /// Set query execution controls. pub fn execution_options(mut self, options: QueryExecutionOptions) -> Self { self.core.set_execution_options(options); @@ -1147,7 +1169,7 @@ impl<'a> FromQueryBuilder<'a> { execution.clone(), ) .await?; - let ast = crate::query::helpers::parse_and_validate_sparql(sparql)?; + let ast = crate::query::helpers::parse_and_validate_sparql(sparql, None)?; let spec = crate::query::helpers::extract_sparql_dataset_spec(&ast)?; return if let Some(alias) = spec .default_graphs @@ -1246,7 +1268,7 @@ impl<'a> FromQueryBuilder<'a> { } }, }; - let ast = crate::query::helpers::parse_and_validate_sparql(sparql)?; + let ast = crate::query::helpers::parse_and_validate_sparql(sparql, None)?; let spec = crate::query::helpers::extract_sparql_dataset_spec(&ast)?; if let Some(alias) = spec .default_graphs @@ -1317,7 +1339,7 @@ impl<'a> FromQueryBuilder<'a> { execution.clone(), ) .await?; - let ast = crate::query::helpers::parse_and_validate_sparql(sparql)?; + let ast = crate::query::helpers::parse_and_validate_sparql(sparql, None)?; let spec = crate::query::helpers::extract_sparql_dataset_spec(&ast)?; return if let Some(alias) = spec .default_graphs @@ -1422,7 +1444,7 @@ impl<'a> FromQueryBuilder<'a> { } }, }; - let ast = crate::query::helpers::parse_and_validate_sparql(sparql)?; + let ast = crate::query::helpers::parse_and_validate_sparql(sparql, None)?; let spec = crate::query::helpers::extract_sparql_dataset_spec(&ast)?; if let Some(alias) = spec .default_graphs diff --git a/fluree-db-api/src/query/connection.rs b/fluree-db-api/src/query/connection.rs index a326fceb8b..8538da6f50 100644 --- a/fluree-db-api/src/query/connection.rs +++ b/fluree-db-api/src/query/connection.rs @@ -269,7 +269,7 @@ impl Fluree { r2rml_table_provider: &dyn R2rmlTableProvider, options: QueryExecutionOptions, ) -> Result { - let ast = parse_and_validate_sparql(sparql)?; + let ast = parse_and_validate_sparql(sparql, None)?; let spec = extract_sparql_dataset_spec(&ast)?; if spec.is_empty() { @@ -703,7 +703,7 @@ impl Fluree { sparql: &str, opts: &GovernanceOptions, ) -> Result { - let ast = parse_and_validate_sparql(sparql)?; + let ast = parse_and_validate_sparql(sparql, None)?; let spec = extract_sparql_dataset_spec(&ast)?; if spec.is_empty() { @@ -736,7 +736,7 @@ impl Fluree { sparql: &str, options: QueryExecutionOptions, ) -> Result { - let ast = parse_and_validate_sparql(sparql)?; + let ast = parse_and_validate_sparql(sparql, None)?; let spec = extract_sparql_dataset_spec(&ast)?; if spec.is_empty() { @@ -756,7 +756,7 @@ impl Fluree { policy: &PolicyContext, options: QueryExecutionOptions, ) -> Result { - let ast = parse_and_validate_sparql(sparql)?; + let ast = parse_and_validate_sparql(sparql, None)?; let spec = extract_sparql_dataset_spec(&ast)?; if spec.is_empty() { @@ -789,7 +789,7 @@ impl Fluree { r2rml: Option>, options: QueryExecutionOptions, ) -> Result { - let ast = parse_and_validate_sparql(sparql)?; + let ast = parse_and_validate_sparql(sparql, None)?; let spec = extract_sparql_dataset_spec(&ast)?; if spec.is_empty() { @@ -825,7 +825,7 @@ impl Fluree { r2rml_table_provider: &dyn R2rmlTableProvider, options: QueryExecutionOptions, ) -> Result { - let ast = parse_and_validate_sparql(sparql)?; + let ast = parse_and_validate_sparql(sparql, None)?; let spec = extract_sparql_dataset_spec(&ast)?; if spec.is_empty() { @@ -880,7 +880,7 @@ impl Fluree { let floor = tracked_query_tracker(&input, &tracking_override, None); charge_query_floor(&floor) .map_err(|e| crate::query::TrackedErrorResponse::fuel_exceeded(&e, floor.tally()))?; - let ast = parse_and_validate_sparql(sparql).map_err(|e| { + let ast = parse_and_validate_sparql(sparql, None).map_err(|e| { crate::query::TrackedErrorResponse::new(400, e.to_string(), floor.tally()) })?; let spec = extract_sparql_dataset_spec(&ast).map_err(|e| { @@ -945,7 +945,7 @@ impl Fluree { let floor = tracked_query_tracker(&input, &tracking_override, None); charge_query_floor(&floor) .map_err(|e| crate::query::TrackedErrorResponse::fuel_exceeded(&e, floor.tally()))?; - let ast = parse_and_validate_sparql(sparql).map_err(|e| { + let ast = parse_and_validate_sparql(sparql, None).map_err(|e| { crate::query::TrackedErrorResponse::new(400, e.to_string(), floor.tally()) })?; let spec = extract_sparql_dataset_spec(&ast).map_err(|e| { @@ -993,7 +993,7 @@ impl Fluree { let floor = tracked_query_tracker(&input, &tracking_override, None); charge_query_floor(&floor) .map_err(|e| crate::query::TrackedErrorResponse::fuel_exceeded(&e, floor.tally()))?; - let ast = parse_and_validate_sparql(sparql).map_err(|e| { + let ast = parse_and_validate_sparql(sparql, None).map_err(|e| { crate::query::TrackedErrorResponse::new(400, e.to_string(), floor.tally()) })?; let spec = extract_sparql_dataset_spec(&ast).map_err(|e| { @@ -1043,7 +1043,7 @@ impl Fluree { let floor = tracked_query_tracker(&input, &tracking_override, None); charge_query_floor(&floor) .map_err(|e| crate::query::TrackedErrorResponse::fuel_exceeded(&e, floor.tally()))?; - let ast = parse_and_validate_sparql(sparql).map_err(|e| { + let ast = parse_and_validate_sparql(sparql, None).map_err(|e| { crate::query::TrackedErrorResponse::new(400, e.to_string(), floor.tally()) })?; let spec = extract_sparql_dataset_spec(&ast).map_err(|e| { @@ -1102,7 +1102,7 @@ impl Fluree { let floor = tracked_query_tracker(&input, &tracking_override, None); charge_query_floor(&floor) .map_err(|e| crate::query::TrackedErrorResponse::fuel_exceeded(&e, floor.tally()))?; - let ast = parse_and_validate_sparql(sparql).map_err(|e| { + let ast = parse_and_validate_sparql(sparql, None).map_err(|e| { crate::query::TrackedErrorResponse::new(400, e.to_string(), floor.tally()) })?; let spec = extract_sparql_dataset_spec(&ast).map_err(|e| { diff --git a/fluree-db-api/src/query/credential.rs b/fluree-db-api/src/query/credential.rs index 8e9dc08fca..748cf4cfe7 100644 --- a/fluree-db-api/src/query/credential.rs +++ b/fluree-db-api/src/query/credential.rs @@ -46,7 +46,7 @@ impl Fluree { let identity = jws_result.did; // Parse SPARQL to extract dataset spec - let ast = parse_and_validate_sparql(sparql)?; + let ast = parse_and_validate_sparql(sparql, None)?; // Extract dataset clause let dataset_clause = match &ast.body { diff --git a/fluree-db-api/src/query/graph_source.rs b/fluree-db-api/src/query/graph_source.rs index e7149d7360..4e1640e003 100644 --- a/fluree-db-api/src/query/graph_source.rs +++ b/fluree-db-api/src/query/graph_source.rs @@ -106,7 +106,7 @@ impl Fluree { sparql: &str, ) -> Result { // See `query_graph_source` above — no default-context injection here. - let (vars, parsed) = parse_sparql_to_ir(sparql, &ledger.snapshot, None)?; + let (vars, parsed) = parse_sparql_to_ir(sparql, &ledger.snapshot, None, None)?; let executable = ExecutableQuery::simple(parsed.clone()); let r2rml_provider = crate::r2rml_provider!(self); diff --git a/fluree-db-api/src/query/helpers.rs b/fluree-db-api/src/query/helpers.rs index e317ada170..68ae305638 100644 --- a/fluree-db-api/src/query/helpers.rs +++ b/fluree-db-api/src/query/helpers.rs @@ -70,8 +70,9 @@ pub(crate) fn parse_sparql_to_ir( sparql: &str, snapshot: &LedgerSnapshot, default_context: Option<&JsonValue>, + params: Option<&fluree_db_sparql::ParamMap>, ) -> Result<(VarRegistry, Query)> { - let ast = parse_and_validate_sparql(sparql)?; + let ast = parse_and_validate_sparql(sparql, params)?; lower_sparql_ast(ast, snapshot, default_context, sparql) } @@ -504,7 +505,21 @@ pub(crate) fn sparql_parse_count() -> usize { SPARQL_PARSE_COUNT.with(std::cell::Cell::get) } -pub(crate) fn parse_and_validate_sparql(sparql: &str) -> Result { +/// Parse and validate `sparql`, then substitute `params` into it. Without +/// parameters this is the parse alone. +pub(crate) fn parse_and_validate_sparql( + sparql: &str, + params: Option<&fluree_db_sparql::ParamMap>, +) -> Result { + let mut ast = parse_and_validate_unparameterized(sparql)?; + if let Some(params) = params { + fluree_db_sparql::substitute_params(&mut ast, params) + .map_err(|e| ApiError::invalid_query(e.to_string()))?; + } + Ok(ast) +} + +fn parse_and_validate_unparameterized(sparql: &str) -> Result { #[cfg(test)] SPARQL_PARSE_COUNT.with(|c| c.set(c.get() + 1)); @@ -809,7 +824,7 @@ mod tests { reset_sparql_parse_count(); let sparql = "SELECT * WHERE { ?s ?p ?o }"; - let ast = parse_and_validate_sparql(sparql).expect("parse"); + let ast = parse_and_validate_sparql(sparql, None).expect("parse"); assert_eq!(sparql_parse_count(), 1, "exactly one parse so far"); let snapshot = LedgerSnapshot::genesis("test:main"); @@ -828,7 +843,8 @@ mod tests { fn parse_sparql_to_ir_parses_once() { reset_sparql_parse_count(); let snapshot = LedgerSnapshot::genesis("test:main"); - let _ = parse_sparql_to_ir("SELECT * WHERE { ?s ?p ?o }", &snapshot, None).expect("lower"); + let _ = parse_sparql_to_ir("SELECT * WHERE { ?s ?p ?o }", &snapshot, None, None) + .expect("lower"); assert_eq!(sparql_parse_count(), 1); } diff --git a/fluree-db-api/src/query/mod.rs b/fluree-db-api/src/query/mod.rs index 132bf15780..afd2900519 100644 --- a/fluree-db-api/src/query/mod.rs +++ b/fluree-db-api/src/query/mod.rs @@ -51,6 +51,9 @@ pub struct QueryExecutionOptions { /// Same contract as `GovernanceOptions::server_identity`: set by an auth /// layer, never derived from the request body or headers. pub server_identity: Option, + /// SPARQL parameters: variable name → JSON-LD value, substituted into the + /// parsed query before lowering (see [`fluree_db_sparql::substitute_params`]). + params: Option>, lifecycle_guard: Option>, } @@ -60,6 +63,7 @@ impl fmt::Debug for QueryExecutionOptions { .field("cancellation", &self.cancellation) .field("trust_fk_refs", &self.trust_fk_refs) .field("server_identity", &self.server_identity) + .field("params", &self.params) .field("has_lifecycle_guard", &self.lifecycle_guard.is_some()) .finish() } @@ -93,6 +97,28 @@ impl QueryExecutionOptions { self } + /// Bind SPARQL variables to values for this query: each named variable + /// (`?name` or `$name`) is replaced by its value wherever it appears. A + /// name the query does not mention is an error, as are parameters on a + /// JSON-LD query. + pub fn with_params(mut self, params: fluree_db_sparql::ParamMap) -> Self { + self.params = (!params.is_empty()).then(|| Arc::new(params)); + self + } + + /// The SPARQL parameters for `input`; JSON-LD queries take none. + pub(crate) fn sparql_params( + &self, + input: &crate::view::QueryInput<'_>, + ) -> crate::Result> { + match (&self.params, input) { + (Some(_), crate::view::QueryInput::JsonLd(_)) => Err(crate::ApiError::invalid_query( + "parameters apply to SPARQL queries; a JSON-LD query takes its values in the query", + )), + (params, _) => Ok(params.as_deref()), + } + } + /// Attach an opaque guard that lives as long as these execution options. /// /// This is intended for adapters that need a runtime-specific cancellation diff --git a/fluree-db-api/src/revert_preview.rs b/fluree-db-api/src/revert_preview.rs index 70456113bf..1b0858c69c 100644 --- a/fluree-db-api/src/revert_preview.rs +++ b/fluree-db-api/src/revert_preview.rs @@ -298,7 +298,7 @@ impl crate::Fluree { Some(match staged { Some((_view, outcome)) => ValidationSummary { conforms: outcome.conforms(), - report: outcome.report, + report: outcome.report.map(|report| report.to_string()), }, // The strategy left nothing to apply, so nothing can reject // it. diff --git a/fluree-db-api/src/shacl_tests.rs b/fluree-db-api/src/shacl_tests.rs index ea177af83e..98ee59b75f 100644 --- a/fluree-db-api/src/shacl_tests.rs +++ b/fluree-db-api/src/shacl_tests.rs @@ -28,7 +28,7 @@ fn shacl_context() -> JsonValue { /// text is identical either way, and the text is what these tests are about. fn shacl_violation_message(err: ApiError) -> String { match err { - ApiError::Transact(TransactError::ShaclViolation(message)) => message, + ApiError::Transact(TransactError::ShaclViolation(message)) => message.to_string(), ApiError::Http { status: 400, message, @@ -5280,3 +5280,49 @@ async fn shacl_sparql_invalid_query_fails_closed() { "expected pre-binding restriction error naming MINUS, got: {message}" ); } + +/// A rejected transaction carries each violation resolved to full IRIs, as +/// `validate` reports them, alongside the readable message. +#[tokio::test] +async fn violation_carries_resolved_results() { + let fluree = FlureeBuilder::memory().build_memory(); + let context = shacl_context(); + let ledger = fluree + .create_ledger("shacl/violation-results:main") + .await + .unwrap(); + let shape = json!({ + "@context": context.clone(), + "@id": "ex:UserShape", + "@type": "sh:NodeShape", + "sh:targetClass": {"@id": "ex:User"}, + "sh:property": [{"sh:path": {"@id": "schema:name"}, "sh:minCount": 1}] + }); + let ledger = fluree.upsert(ledger, &shape).await.unwrap().ledger; + let err = fluree + .upsert( + ledger, + &json!({"@context": context, "@id": "ex:alex", "@type": "ex:User"}), + ) + .await + .unwrap_err(); + + let ApiError::Transact(TransactError::ShaclViolation(violations)) = err else { + panic!("expected SHACL violation, got {err:?}"); + }; + assert!(violations.contains("Focus node: ex:alex"), "{violations}"); + let [result] = violations.results() else { + panic!("expected one result, got {:?}", violations.results()); + }; + assert_eq!(result.focus_node, json!("http://example.org/ns/alex")); + assert_eq!( + result.result_path.as_deref(), + Some("http://schema.org/name") + ); + assert_eq!(result.source_shape, "http://example.org/ns/UserShape"); + assert_eq!( + result.constraint_component, + "http://www.w3.org/ns/shacl#MinCountConstraintComponent" + ); + assert_eq!(result.severity, "http://www.w3.org/ns/shacl#Violation"); +} diff --git a/fluree-db-api/src/transaction.rs b/fluree-db-api/src/transaction.rs index f17ffd2a20..a0850a8ce9 100644 --- a/fluree-db-api/src/transaction.rs +++ b/fluree-db-api/src/transaction.rs @@ -63,8 +63,9 @@ pub enum TxnOperation { InsertTurtle(String), /// Turtle or TriG upsert. UpsertTurtle(String), - /// SPARQL UPDATE, which may itself hold several `;`-separated operations. - SparqlUpdate(String), + /// SPARQL UPDATE, which may itself hold several `;`-separated operations, + /// with optional parameters (see [`fluree_db_sparql::substitute_params`]). + SparqlUpdate(String, Option), } impl TxnOperation { @@ -75,7 +76,7 @@ impl TxnOperation { Self::Update(json) => OpPlan::from_op(TransactOperation::UpdateJson(json))?, Self::InsertTurtle(turtle) => OpPlan::from_op(TransactOperation::InsertTurtle(turtle))?, Self::UpsertTurtle(turtle) => OpPlan::from_op(TransactOperation::UpsertTurtle(turtle))?, - Self::SparqlUpdate(sparql) => OpPlan::Sparql(sparql), + Self::SparqlUpdate(sparql, params) => OpPlan::Sparql(sparql, params.as_ref()), }) } @@ -87,7 +88,7 @@ impl TxnOperation { Self::Insert(_) | Self::Update(_) | Self::InsertTurtle(_) => { Some(cypher_write::fresh_skolem_txn_id()) } - Self::Upsert(_) | Self::UpsertTurtle(_) | Self::SparqlUpdate(_) => None, + Self::Upsert(_) | Self::UpsertTurtle(_) | Self::SparqlUpdate(..) => None, } } } diff --git a/fluree-db-api/src/tx.rs b/fluree-db-api/src/tx.rs index 4a1d268960..9160ac6a1c 100644 --- a/fluree-db-api/src/tx.rs +++ b/fluree-db-api/src/tx.rs @@ -1442,30 +1442,32 @@ pub(crate) async fn apply_shacl_policy_to_staged_view( ); } if !reject_violations.is_empty() { + // The structured results name every identifier in full, as + // `validate` reports them; the message compacts them for reading. + let full = crate::format::IriCompactor::from_namespaces(violation_namespaces(view, &ctx)); + let results = crate::validate::report_results(&reject_violations, &|sid| { + full.compact_id_sid(sid) + .unwrap_or_else(|_| fluree_db_shacl::unresolved_sid(sid)) + }); return Err(fluree_db_transact::TransactError::ShaclViolation( - format_violations(&reject_violations, &compactor), + fluree_db_transact::ShaclViolations::new( + format_violations(&reject_violations, &compactor), + results, + ), )); } Ok(true) } -/// Build the compactor that renders identifiers in violation messages. -/// -/// Namespaces come from the snapshot plus whatever the operation introduced -/// and has not committed yet, so a term it brought in itself still resolves. -/// An authoring context, where the request carried one, supplies the prefixes -/// that let the message name terms the way their author wrote them. Without -/// one — a Turtle insert, a replayed commit — the compactor reports full IRIs -/// rather than inventing prefixes the reader has no way to resolve. +/// The namespaces a violation's identifiers resolve against: the snapshot's +/// plus those the operation introduced and has not committed yet. #[cfg(feature = "shacl")] -fn violation_iri_compactor( +fn violation_namespaces( view: &StagedLedger, ctx: &StagedShaclContext<'_>, -) -> crate::format::IriCompactor { - use crate::format::IriCompactor; - +) -> std::sync::Arc> { let base = view.base().snapshot.shared_namespaces(); - let namespace_codes = match ctx.uncommitted_namespaces { + match ctx.uncommitted_namespaces { Some(uncommitted) if !uncommitted.is_empty() => { let mut merged = (*base).clone(); merged.extend( @@ -1476,8 +1478,25 @@ fn violation_iri_compactor( std::sync::Arc::new(merged) } _ => base, - }; + } +} +/// Build the compactor that renders identifiers in violation messages. +/// +/// Namespaces come from the snapshot plus whatever the operation introduced +/// and has not committed yet, so a term it brought in itself still resolves. +/// An authoring context, where the request carried one, supplies the prefixes +/// that let the message name terms the way their author wrote them. Without +/// one — a Turtle insert, a replayed commit — the compactor reports full IRIs +/// rather than inventing prefixes the reader has no way to resolve. +#[cfg(feature = "shacl")] +fn violation_iri_compactor( + view: &StagedLedger, + ctx: &StagedShaclContext<'_>, +) -> crate::format::IriCompactor { + use crate::format::IriCompactor; + + let namespace_codes = violation_namespaces(view, ctx); match ctx .txn_context .and_then(|raw| crate::ParsedContext::parse(None, raw).ok()) @@ -2044,7 +2063,13 @@ async fn enforce_unique_constraints( }) .unwrap_or_else(|| format!("g_id={g_id}")) }; - let value_str = format!("{o:?}"); + let value_str = match o { + fluree_db_core::FlakeValue::String(s) => s.clone(), + fluree_db_core::FlakeValue::Ref(sid) => { + snapshot.decode_sid(sid).unwrap_or_else(|| sid.to_string()) + } + other => other.to_string(), + }; // Pick two subjects for the error message let mut subj_iter = seen_subjects.iter(); diff --git a/fluree-db-api/src/tx_builder.rs b/fluree-db-api/src/tx_builder.rs index 1aefbbf559..071fd20d83 100644 --- a/fluree-db-api/src/tx_builder.rs +++ b/fluree-db-api/src/tx_builder.rs @@ -94,6 +94,7 @@ impl GraphPayload<'_> { /// last operation's delta carries the whole request's allocations. pub(crate) fn parse_and_lower_sparql_update( sparql: &str, + params: Option<&fluree_db_sparql::ParamMap>, snapshot: &LedgerSnapshot, txn_opts: TxnOpts, ) -> Result> { @@ -105,7 +106,7 @@ pub(crate) fn parse_and_lower_sparql_update( format!("SPARQL UPDATE parse error: {}", messages.join("; ")), )); } - let ast = parsed + let mut ast = parsed .ast .ok_or_else(|| ApiError::http(400, "Failed to parse SPARQL UPDATE".to_string()))?; @@ -131,6 +132,10 @@ pub(crate) fn parse_and_lower_sparql_update( .unwrap_or_else(|| "SPARQL UPDATE validation error".to_string()); return Err(ApiError::sparql(message, errors)); } + if let Some(params) = params { + fluree_db_sparql::substitute_params(&mut ast, params) + .map_err(|e| ApiError::http(400, e.to_string()))?; + } let mut ns = NamespaceRegistry::from_db(snapshot); lower_sparql_update_request(&ast, &mut ns, txn_opts) @@ -493,6 +498,8 @@ pub(crate) struct TransactCore<'a> { /// Raw SPARQL UPDATE text, lowered to a `Txn` under the write lock during /// `execute()` so its namespace allocation shares the staging registry. pub(crate) pending_sparql: Option<&'a str>, + /// Parameters substituted into `pending_sparql`. + pub(crate) sparql_params: Option<&'a fluree_db_sparql::ParamMap>, /// A multi-clause Cypher write staged clause-by-clause under the write /// lock by the sequential driver (see [`crate::cypher_seq`]). pub(crate) pre_built_cypher_seq: Option, @@ -525,6 +532,7 @@ impl<'a> TransactCore<'a> { pre_built_txn: None, pre_built_txn_followup: None, pending_sparql: None, + sparql_params: None, pre_built_cypher_seq: None, txn_opts: TxnOpts::default(), commit_opts: CommitOpts::default(), @@ -605,7 +613,11 @@ impl<'a> TransactCore<'a> { self.txn_opts = opts; } - pub(crate) fn set_sparql_update(&mut self, sparql: &'a str) { + pub(crate) fn set_sparql_update( + &mut self, + sparql: &'a str, + params: Option<&'a fluree_db_sparql::ParamMap>, + ) { if self.operation.is_some() || self.pre_built_txn.is_some() || self.pending_sparql.is_some() { self.errors.push(BuilderError::Conflict { @@ -614,6 +626,7 @@ impl<'a> TransactCore<'a> { }); } else { self.pending_sparql = Some(sparql); + self.sparql_params = params; } } @@ -1062,7 +1075,18 @@ impl<'a> RefTransactBuilder<'a> { /// namespace-conflict retry that pre-lowering against an unlocked /// snapshot would require. pub fn sparql_update(mut self, sparql: &'a str) -> Self { - self.core.set_sparql_update(sparql); + self.core.set_sparql_update(sparql, None); + self + } + + /// [`Self::sparql_update`] with variables bound to values; see + /// [`fluree_db_sparql::substitute_params`]. + pub fn sparql_update_with_params( + mut self, + sparql: &'a str, + params: &'a fluree_db_sparql::ParamMap, + ) -> Self { + self.core.set_sparql_update(sparql, Some(params)); self } @@ -1182,7 +1206,7 @@ pub(crate) enum OpPlan<'a> { /// A SPARQL UPDATE request, parsed and lowered against whichever state /// it is staged on — lowering allocates namespace codes relative to /// that state's table. - Sparql(&'a str), + Sparql(&'a str, Option<&'a fluree_db_sparql::ParamMap>), JsonLike { txn_type: TxnType, txn_json: JsonValue, @@ -1431,8 +1455,12 @@ impl Fluree { } if let Some(sparql) = core.pending_sparql { - let txns = - parse_and_lower_sparql_update(sparql, &ledger_state.snapshot, core.txn_opts)?; + let txns = parse_and_lower_sparql_update( + sparql, + core.sparql_params, + &ledger_state.snapshot, + core.txn_opts, + )?; // A single-op request reports its own type; a multi-op (or // empty no-op) request is reported as a generic update. let txn_type = match txns.as_slice() { @@ -1545,8 +1573,13 @@ impl Fluree { let ledger_id = ledger_state.ledger_id().to_string(); let store_raw_txn = txn_opts.store_raw_txn.unwrap_or(false); match op_plan { - OpPlan::Sparql(sparql) => { - let txns = parse_and_lower_sparql_update(sparql, &ledger_state.snapshot, txn_opts)?; + OpPlan::Sparql(sparql, params) => { + let txns = parse_and_lower_sparql_update( + sparql, + *params, + &ledger_state.snapshot, + txn_opts, + )?; // A single-op request reports its own type; a multi-op (or // empty no-op) request is reported as a generic update. let txn_type = match txns.as_slice() { @@ -2334,7 +2367,7 @@ impl Fluree { // commit itself. Later attempts take the lock first and stage under // it; they are for commit conflicts that `refresh` heals. let op_plan = match core.pending_sparql.take() { - Some(sparql) => OpPlan::Sparql(sparql), + Some(sparql) => OpPlan::Sparql(sparql, core.sparql_params), None => OpPlan::from_op(core.operation.take().unwrap())?, // safe: validate checks }; let txn_opts = core.txn_opts; diff --git a/fluree-db-api/src/validate.rs b/fluree-db-api/src/validate.rs index d22306f0c3..03edb33b4d 100644 --- a/fluree-db-api/src/validate.rs +++ b/fluree-db-api/src/validate.rs @@ -78,32 +78,7 @@ impl Default for ValidateOptions { } } -/// One validation result with all identifiers resolved to IRIs. -#[derive(Debug, Clone, serde::Serialize)] -pub struct ReportResult { - /// The node that failed validation (`sh:focusNode`): a JSON string for - /// IRIs / blank-node labels, or a JSON-LD value object (or native - /// scalar) for literal `sh:targetNode` targets. - pub focus_node: JsonValue, - /// The property path, when it is a single predicate (`sh:resultPath`). - /// Complex paths are omitted rather than misrepresented. - #[serde(skip_serializing_if = "Option::is_none")] - pub result_path: Option, - /// The node shape that produced this result. - pub source_shape: String, - /// The property shape that produced this result, when applicable. - #[serde(skip_serializing_if = "Option::is_none")] - pub source_constraint: Option, - /// The constraint component IRI (`sh:sourceConstraintComponent`). - pub constraint_component: String, - /// Severity IRI: `sh:Violation`, `sh:Warning`, or `sh:Info`. - pub severity: String, - /// Human-readable message (`sh:resultMessage`). - pub message: String, - /// The offending value, when applicable (`sh:value`). - #[serde(skip_serializing_if = "Option::is_none")] - pub value: Option, -} +pub use fluree_db_transact::ReportResult; /// A resolved validation report, ready for serialization. #[derive(Debug, Clone, serde::Serialize)] @@ -630,54 +605,7 @@ async fn validate_view_inner( .map_err(TransactError::from)?; let resolve = |sid: &Sid| resolve_sid(snapshot, inline_registry.as_ref(), sid); - let mut results: Vec = raw - .results - .iter() - .map(|r| ReportResult { - focus_node: match &r.focus_node { - fluree_db_shacl::FocusNode::Node(sid) => JsonValue::String(resolve(sid)), - fluree_db_shacl::FocusNode::Literal(lit) => { - match value_json( - &lit.value, - Some(&lit.datatype), - lit.lang.as_deref(), - &resolve, - ) { - // A bare string would be ambiguous with an IRI focus — - // wrap plain string literals as a value object. - JsonValue::String(s) => json!({"@value": s}), - other => other, - } - } - }, - result_path: r.result_path.as_ref().map(&resolve), - source_shape: resolve(&r.source_shape), - source_constraint: r.source_constraint.as_ref().map(&resolve), - constraint_component: r.constraint_component.to_string(), - severity: severity_iri(r.severity).to_string(), - message: r.message.clone(), - value: r.value.as_ref().map(|v| { - value_json( - v, - r.value_datatype.as_ref(), - r.value_lang.as_deref(), - &resolve, - ) - }), - }) - .collect(); - results.sort_by(|a, b| { - ( - a.focus_node.to_string(), - &a.constraint_component, - &a.message, - ) - .cmp(&( - b.focus_node.to_string(), - &b.constraint_component, - &b.message, - )) - }); + let results = report_results(&raw.results, &resolve); Ok(ValidateReport { conforms: raw.conforms, @@ -727,6 +655,62 @@ fn resolve_sid( .unwrap_or_else(|| sid.name.to_string()) } +/// Validation results with every identifier resolved by `resolve`, sorted by +/// (focus node, component, message). +pub(crate) fn report_results<'a>( + results: impl IntoIterator, + resolve: &impl Fn(&Sid) -> String, +) -> Vec { + let mut results: Vec = results + .into_iter() + .map(|r| ReportResult { + focus_node: match &r.focus_node { + fluree_db_shacl::FocusNode::Node(sid) => JsonValue::String(resolve(sid)), + fluree_db_shacl::FocusNode::Literal(lit) => { + match value_json( + &lit.value, + Some(&lit.datatype), + lit.lang.as_deref(), + resolve, + ) { + // A bare string would be ambiguous with an IRI focus — + // wrap plain string literals as a value object. + JsonValue::String(s) => json!({"@value": s}), + other => other, + } + } + }, + result_path: r.result_path.as_ref().map(resolve), + source_shape: resolve(&r.source_shape), + source_constraint: r.source_constraint.as_ref().map(resolve), + constraint_component: r.constraint_component.to_string(), + severity: severity_iri(r.severity).to_string(), + message: r.message.clone(), + value: r.value.as_ref().map(|v| { + value_json( + v, + r.value_datatype.as_ref(), + r.value_lang.as_deref(), + resolve, + ) + }), + }) + .collect(); + results.sort_by(|a, b| { + ( + a.focus_node.to_string(), + &a.constraint_component, + &a.message, + ) + .cmp(&( + b.focus_node.to_string(), + &b.constraint_component, + &b.message, + )) + }); + results +} + fn severity_iri(severity: Severity) -> &'static str { match severity { Severity::Violation => sh_vocab::VIOLATION, diff --git a/fluree-db-api/src/view/dataset_query.rs b/fluree-db-api/src/view/dataset_query.rs index 44c5f9dee0..000442990d 100644 --- a/fluree-db-api/src/view/dataset_query.rs +++ b/fluree-db-api/src/view/dataset_query.rs @@ -86,6 +86,8 @@ impl Fluree { options: QueryExecutionOptions, parsed_ast: Option, ) -> Result { + options.sparql_params(&input)?; + // Single-ledger fast path (only safe for JSON-LD or SPARQL without dataset clauses). if dataset.is_single_ledger() { if let Some(view) = dataset.primary() { @@ -100,7 +102,9 @@ impl Fluree { // clause, so this never delegates in that case. let has_dataset = match &parsed_ast { Some(ast) => sparql_ast_has_dataset(ast), - None => sparql_ast_has_dataset(&parse_and_validate_sparql(sparql)?), + None => { + sparql_ast_has_dataset(&parse_and_validate_sparql(sparql, None)?) + } }; if !has_dataset { return self.query_with_options(view, input, options).await; @@ -157,6 +161,7 @@ impl Fluree { sparql, &primary.snapshot, primary.default_context.as_ref(), + options.sparql_params(&input)?, )?, } } @@ -201,6 +206,7 @@ impl Fluree { options: QueryExecutionOptions, ) -> Result { let input = q.into(); + options.sparql_params(&input)?; // Single-ledger fast path (only safe for JSON-LD or SPARQL without dataset clauses). if dataset.is_single_ledger() { @@ -218,7 +224,7 @@ impl Fluree { .await; } QueryInput::Sparql(sparql) => { - let ast = parse_and_validate_sparql(sparql)?; + let ast = parse_and_validate_sparql(sparql, None)?; let has_dataset = match &ast.body { fluree_db_sparql::ast::QueryBody::Select(q) => q.dataset.is_some(), fluree_db_sparql::ast::QueryBody::Ask(q) => q.dataset.is_some(), @@ -261,9 +267,12 @@ impl Fluree { primary.default_context.as_ref(), None, )?, - QueryInput::Sparql(sparql) => { - parse_sparql_to_ir(sparql, &primary.snapshot, primary.default_context.as_ref())? - } + QueryInput::Sparql(sparql) => parse_sparql_to_ir( + sparql, + &primary.snapshot, + primary.default_context.as_ref(), + options.sparql_params(&input)?, + )?, }; // 1b. Auto-wrap for graph source context, then refuse whatever the wrap @@ -347,6 +356,9 @@ impl Fluree { { // Tracker: caller-provided options if given, else per-input defaults. let tracker = tracked_query_tracker(&input, &tracking_override, parsed_ast.as_ref()); + let params = options.sparql_params(&input).map_err(|e| { + crate::query::TrackedErrorResponse::new(400, e.to_string(), tracker.tally()) + })?; // Charge the one-time query floor before parsing (see `query_tracked`). charge_query_floor(&tracker) @@ -387,6 +399,7 @@ impl Fluree { sparql, &primary.snapshot, primary.default_context.as_ref(), + params, ), }; lowered.map_err(|e| { @@ -474,6 +487,9 @@ impl Fluree { let input = q.into(); let tracker = tracked_query_tracker(&input, &tracking_override, None); + let params = options.sparql_params(&input).map_err(|e| { + crate::query::TrackedErrorResponse::new(400, e.to_string(), tracker.tally()) + })?; // Charge the one-time query floor before parsing (see `query_tracked`). charge_query_floor(&tracker) @@ -499,12 +515,15 @@ impl Fluree { .map_err(|e| { crate::query::TrackedErrorResponse::new(400, e.to_string(), tracker.tally()) })?, - QueryInput::Sparql(sparql) => { - parse_sparql_to_ir(sparql, &primary.snapshot, primary.default_context.as_ref()) - .map_err(|e| { - crate::query::TrackedErrorResponse::new(400, e.to_string(), tracker.tally()) - })? - } + QueryInput::Sparql(sparql) => parse_sparql_to_ir( + sparql, + &primary.snapshot, + primary.default_context.as_ref(), + params, + ) + .map_err(|e| { + crate::query::TrackedErrorResponse::new(400, e.to_string(), tracker.tally()) + })?, }; // Auto-wrap for graph source context, then refuse whatever the wrap diff --git a/fluree-db-api/src/view/query.rs b/fluree-db-api/src/view/query.rs index 6f31666dd9..1df781310b 100644 --- a/fluree-db-api/src/view/query.rs +++ b/fluree-db-api/src/view/query.rs @@ -341,8 +341,9 @@ impl Fluree { // #1473: parse SPARQL once. The AST is reused for FROM-clause resolution // and IR lowering below, instead of re-lexing the string 2–3×. let parse_start = fluree_db_core::clock::Instant::now(); + let params = options.sparql_params(&input)?; let mut sparql_ast = match input { - QueryInput::Sparql(sparql) => Some(parse_and_validate_sparql(sparql)?), + QueryInput::Sparql(sparql) => Some(parse_and_validate_sparql(sparql, params)?), QueryInput::JsonLd(_) => None, }; @@ -721,9 +722,10 @@ impl Fluree { // connection/spawn contexts), so a dataset clause is rejected here rather // than silently ignored; plain within-ledger datasets are handled by // `query_with_options`. Otherwise the AST is reused for lowering. + let params = options.sparql_params(&input)?; let mut sparql_ast = match input { QueryInput::Sparql(sparql) => { - let ast = parse_and_validate_sparql(sparql)?; + let ast = parse_and_validate_sparql(sparql, params)?; if sparql_ast_has_dataset(&ast) { return Err(single_ledger_dataset_clause_error()); } @@ -817,11 +819,23 @@ impl Fluree { /// Explain a SPARQL query plan against a GraphDb. pub async fn explain_sparql(&self, db: &GraphDb, sparql: &str) -> Result { + self.explain_sparql_with_params(db, sparql, None).await + } + + /// [`Self::explain_sparql`] with parameters substituted, as + /// [`QueryExecutionOptions::with_params`] would run the query. + pub async fn explain_sparql_with_params( + &self, + db: &GraphDb, + sparql: &str, + params: Option<&fluree_db_sparql::ParamMap>, + ) -> Result { let db = self.prepare_explain_view(db).await?; crate::explain::explain_sparql_for_view( &db.snapshot, sparql, db.default_context.as_ref(), + params, db.is_root(), ) .map(|result| explain_policy_notice(result, &db)) @@ -870,9 +884,21 @@ impl Fluree { // already-parsed AST) before charging the floor (the dataset method // charges it) so fuel is accounted exactly once. `Err` = cross-ledger // clause IRI. + let params = match options.sparql_params(&input) { + Ok(params) => params, + Err(e) => { + let tracker = tracked_query_tracker(&input, &tracking_override, None); + let _ = charge_query_floor(&tracker); + return Err(crate::query::TrackedErrorResponse::new( + 400, + e.to_string(), + tracker.tally(), + )); + } + }; let mut sparql_ast = None; if let QueryInput::Sparql(sparql) = input { - let ast = match parse_and_validate_sparql(sparql) { + let ast = match parse_and_validate_sparql(sparql, params) { Ok(ast) => ast, Err(e) => { let tracker = tracked_query_tracker(&input, &tracking_override, None); @@ -1036,9 +1062,21 @@ impl Fluree { // FROM/FROM NAMED is unsupported (the R2RML dataset path is non-`Send`; // this runs from `Send` contexts), so a resolved clause is rejected // rather than silently ignored. Otherwise the AST is reused for lowering. + let params = match options.sparql_params(&input) { + Ok(params) => params, + Err(e) => { + let tracker = tracked_query_tracker(&input, &tracking_override, None); + let _ = charge_query_floor(&tracker); + return Err(crate::query::TrackedErrorResponse::new( + 400, + e.to_string(), + tracker.tally(), + )); + } + }; let mut sparql_ast = None; if let QueryInput::Sparql(sparql) = input { - let parsed = parse_and_validate_sparql(sparql); + let parsed = parse_and_validate_sparql(sparql, params); let tracker = tracked_query_tracker(&input, &tracking_override, parsed.as_ref().ok()); let ast = match parsed { Ok(ast) => ast, @@ -1367,7 +1405,7 @@ impl Fluree { /// [`build_within_ledger_dataset_from_ast`](Self::build_within_ledger_dataset_from_ast) /// and only reject cross-ledger clause IRIs. pub(crate) fn validate_sparql_for_view(&self, sparql: &str) -> Result<()> { - let ast = parse_and_validate_sparql(sparql)?; + let ast = parse_and_validate_sparql(sparql, None)?; if sparql_ast_has_dataset(&ast) { return Err(single_ledger_dataset_clause_error()); } diff --git a/fluree-db-api/src/view/stream_query.rs b/fluree-db-api/src/view/stream_query.rs index 3184fb31f6..3656137d22 100644 --- a/fluree-db-api/src/view/stream_query.rs +++ b/fluree-db-api/src/view/stream_query.rs @@ -98,6 +98,7 @@ impl Fluree { options: &QueryExecutionOptions, ) -> Result { let input = input.as_input(); + let params = options.sparql_params(&input)?; let (vars, mut parsed) = match &input { QueryInput::JsonLd(json) => crate::query::helpers::parse_jsonld_query( @@ -112,6 +113,7 @@ impl Fluree { sparql, &db.snapshot, db.default_context.as_ref(), + params, )? } }; @@ -339,7 +341,7 @@ impl Fluree { sparql: &str, qc_opts: &crate::GovernanceOptions, ) -> Result { - let ast = crate::query::helpers::parse_and_validate_sparql(sparql)?; + let ast = crate::query::helpers::parse_and_validate_sparql(sparql, None)?; let spec = crate::query::helpers::extract_sparql_dataset_spec(&ast)?; if spec.is_empty() { return Err(ApiError::query( @@ -378,6 +380,7 @@ impl Fluree { .primary() .ok_or_else(|| ApiError::query("Dataset has no graphs for query execution"))?; let input = input.as_input(); + let params = options.sparql_params(&input)?; let (vars, mut parsed) = match &input { QueryInput::JsonLd(json) => crate::query::helpers::parse_jsonld_query( @@ -390,6 +393,7 @@ impl Fluree { sparql, &primary.snapshot, primary.default_context.as_ref(), + params, )?, }; diff --git a/fluree-db-api/tests/grp_query_sparql.rs b/fluree-db-api/tests/grp_query_sparql.rs index 365e3b73f1..bf1c9b5133 100644 --- a/fluree-db-api/tests/grp_query_sparql.rs +++ b/fluree-db-api/tests/grp_query_sparql.rs @@ -70,5 +70,8 @@ mod it_query_subquery_streaming; #[path = "it_union_default_graph.rs"] mod it_union_default_graph; +#[path = "it_sparql_params.rs"] +mod it_sparql_params; + #[path = "it_query_sparql_vector_constant.rs"] mod it_query_sparql_vector_constant; diff --git a/fluree-db-api/tests/it_branch_shacl.rs b/fluree-db-api/tests/it_branch_shacl.rs index 45c81cd991..fd84c3c6d8 100644 --- a/fluree-db-api/tests/it_branch_shacl.rs +++ b/fluree-db-api/tests/it_branch_shacl.rs @@ -89,9 +89,12 @@ async fn head_t(fluree: &fluree_db_api::Fluree, ledger_id: &str) -> i64 { } fn assert_shacl_violation(err: ApiError, context: &str) { + let ApiError::Transact(TransactError::ShaclViolation(violations)) = err else { + panic!("{context}: expected ShaclViolation, got: {err:?}"); + }; assert!( - matches!(err, ApiError::Transact(TransactError::ShaclViolation(_))), - "{context}: expected ShaclViolation, got: {err:?}" + !violations.results().is_empty(), + "{context}: the rejection keeps its results: {violations:?}" ); } diff --git a/fluree-db-api/tests/it_config_graph.rs b/fluree-db-api/tests/it_config_graph.rs index bc08dd3aa1..c0a3fa93fc 100644 --- a/fluree-db-api/tests/it_config_graph.rs +++ b/fluree-db-api/tests/it_config_graph.rs @@ -2483,15 +2483,63 @@ async fn unique_basic_enforcement() { .unwrap_err(); assert!( matches!( - err, + &err, fluree_db_api::ApiError::Transact( - fluree_db_transact::TransactError::UniqueConstraintViolation { .. } - ) + fluree_db_transact::TransactError::UniqueConstraintViolation { value, .. } + ) if value == "alice@example.com" ), "duplicate email should trigger unique constraint violation: {err:?}" ); } +/// Test: a duplicate reference is reported as the IRI it points to. +#[tokio::test] +async fn unique_ref_value_reports_its_iri() { + let fluree = FlureeBuilder::memory().build_memory(); + let ledger_id = "it/unique-ref:main"; + let ledger = genesis_ledger(&fluree, ledger_id); + let result = fluree + .insert( + ledger, + &json!({ + "@context": { + "ex": "http://example.org/", + "f": "https://ns.flur.ee/db#" + }, + "@graph": [ + {"@id": "ex:badge", "f:enforceUnique": true}, + {"@id": "ex:alice", "ex:badge": {"@id": "ex:badge-7"}} + ] + }), + ) + .await + .unwrap(); + let ledger = write_unique_config(&fluree, result.ledger, ledger_id) + .await + .ledger; + + let err = fluree + .insert( + ledger, + &json!({ + "@context": {"ex": "http://example.org/"}, + "@id": "ex:bob", + "ex:badge": {"@id": "ex:badge-7"} + }), + ) + .await + .unwrap_err(); + assert!( + matches!( + &err, + fluree_db_api::ApiError::Transact( + fluree_db_transact::TransactError::UniqueConstraintViolation { value, .. } + ) if value == "http://example.org/badge-7" + ), + "{err:?}" + ); +} + /// Test: annotations exist but uniqueEnabled not set → duplicates allowed. #[tokio::test] async fn unique_not_enabled_no_enforcement() { diff --git a/fluree-db-api/tests/it_query_fulltext.rs b/fluree-db-api/tests/it_query_fulltext.rs index 286fc9e8fe..fa01232172 100644 --- a/fluree-db-api/tests/it_query_fulltext.rs +++ b/fluree-db-api/tests/it_query_fulltext.rs @@ -715,6 +715,18 @@ async fn fulltext_configured_property_indexed_after_reindex() { results.iter().all(|(_, score)| *score > 0.0), "all configured-property hits should have positive scores: {results:?}" ); + + let sparql = query_fulltext_sparql( + &fluree, + &loaded, + "SELECT ?id ?score WHERE { ?id ex:title ?title \ + BIND(fulltext(?title, \"Rust\") AS ?score) FILTER(?score > 0) } ORDER BY DESC(?score)", + ) + .await; + assert_eq!( + sparql, results, + "SPARQL fulltext() must score as JSON-LD does" + ); } /// When `f:fullTextDefaults` is NOT configured, plain-string values on @@ -1975,6 +1987,137 @@ async fn fulltext_perf_50k_labels() { } } +// ============================================================================= +// SPARQL `fulltext()` — twins of the JSON-LD queries above +// ============================================================================= + +/// Run a SPARQL query projecting `(label, score)` and return the pairs in +/// result order. +async fn query_fulltext_sparql( + fluree: &support::MemoryFluree, + ledger: &support::MemoryLedger, + select: &str, +) -> Vec<(String, f64)> { + let sparql = + format!("PREFIX ex: PREFIX f: {select}"); + let result = support::query_sparql(fluree, ledger, &sparql) + .await + .unwrap_or_else(|e| panic!("SPARQL fulltext query failed: {e}")); + let rows = result.to_jsonld(&ledger.snapshot).expect("jsonld"); + rows.as_array() + .expect("rows") + .iter() + .map(|row| { + let row = row.as_array().expect("row"); + ( + row[0].as_str().expect("label").to_string(), + row[1].as_f64().expect("score"), + ) + }) + .collect() +} + +fn sparql_content_query(call: &str) -> String { + format!( + "SELECT ?title ?score WHERE {{ ?doc ex:content ?content ; ex:title ?title \ + BIND({call} AS ?score) FILTER(?score > 0) }} ORDER BY DESC(?score)" + ) +} + +/// The same scores and order from SPARQL as from JSON-LD, over indexed +/// documents and over documents committed after the index was built. +#[tokio::test] +async fn fulltext_sparql_scores_as_jsonld_does() { + let fluree = FlureeBuilder::memory().build_memory(); + let alias = "it/fulltext-sparql:main"; + + let (local, handle) = start_background_indexer_local( + fluree.backend().clone(), + fluree + .nameservice_mode() + .publisher_arc() + .expect("test setup requires ReadWrite nameservice mode"), + fluree_db_indexer::IndexerConfig::small(), + ); + + local + .run_until(async move { + let db0 = LedgerSnapshot::genesis(alias); + let mut ledger = LedgerState::new(db0, Novelty::new(0)); + for (id, title, content) in [ + ( + "ex:doc1", + "Rust Guide", + "Rust is a systems programming language", + ), + ("ex:doc2", "Pasta", "Cooking pasta with tomatoes and basil"), + ( + "ex:doc3", + "Rust Macros", + "Rust macros generate Rust code at compile time", + ), + ] { + ledger = insert_doc(&fluree, ledger, id, title, content).await; + } + let indexed = index_and_load(&fluree, &handle, alias, ledger.snapshot.t).await; + + let sparql = sparql_content_query("fulltext(?content, \"Rust programming\")"); + let jsonld = query_fulltext(&fluree, &indexed, "Rust programming").await; + assert_eq!(jsonld.len(), 2, "{jsonld:?}"); + assert_eq!( + query_fulltext_sparql(&fluree, &indexed, &sparql).await, + jsonld + ); + + let with_novelty = insert_doc( + &fluree, + indexed, + "ex:doc4", + "Rust Book", + "Programming in Rust, the book", + ) + .await; + let jsonld = query_fulltext(&fluree, &with_novelty, "Rust programming").await; + assert_eq!(jsonld.len(), 3, "{jsonld:?}"); + assert_eq!( + query_fulltext_sparql(&fluree, &with_novelty, &sparql).await, + jsonld + ); + }) + .await; +} + +/// `fulltext`, its case and underscore variants, and the namespaced +/// `f:fulltext` all name the same function. +#[tokio::test] +async fn fulltext_sparql_spellings_agree() { + let fluree = FlureeBuilder::memory().build_memory(); + let mut ledger = support::genesis_ledger_for_fluree(&fluree, "it/fulltext-sparql-names:main"); + for (id, title, content) in [ + ( + "ex:doc1", + "Rust Guide", + "Rust is a systems programming language", + ), + ("ex:doc2", "Pasta", "Cooking pasta with tomatoes and basil"), + ] { + ledger = insert_doc(&fluree, ledger, id, title, content).await; + } + + let expected = query_fulltext(&fluree, &ledger, "Rust").await; + assert_eq!(expected.len(), 1, "{expected:?}"); + for call in [ + "fulltext(?content, \"Rust\")", + "FULLTEXT(?content, \"Rust\")", + "full_text(?content, \"Rust\")", + "f:fulltext(?content, \"Rust\")", + "(?content, \"Rust\")", + ] { + let rows = query_fulltext_sparql(&fluree, &ledger, &sparql_content_query(call)).await; + assert_eq!(rows, expected, "{call}"); + } +} + /// The JSON-LD form of `f:fullTextDefaults` from the docs: the config node /// names the config graph with a per-node `@graph`, and its nested /// `f:FullTextDefaults` and `f:FullTextProperty` nodes must land there too. diff --git a/fluree-db-api/tests/it_sparql_params.rs b/fluree-db-api/tests/it_sparql_params.rs new file mode 100644 index 0000000000..7ceb95a5bc --- /dev/null +++ b/fluree-db-api/tests/it_sparql_params.rs @@ -0,0 +1,365 @@ +//! SPARQL parameters: a variable bound to a value runs as if the value were +//! written in its place. + +use fluree_db_api::{ + Fluree, FlureeBuilder, FormatterConfig, GraphDb, QueryExecutionOptions, SparqlParamMap, + TxnOperation, +}; +use serde_json::{json, Value as JsonValue}; + +const LEDGER: &str = "it/sparql-params:main"; +const PREFIX: &str = "PREFIX ex: "; + +async fn seeded() -> Fluree { + let fluree = FlureeBuilder::memory().build_memory(); + fluree.create_ledger(LEDGER).await.expect("create"); + fluree + .graph(LEDGER) + .transact() + .insert(&json!({ + "@context": { "ex": "http://example.org/" }, + "@graph": [ + { "@id": "ex:alice", "ex:name": "Alice", "ex:age": 30 }, + { "@id": "ex:bob", "ex:name": "Bob", "ex:age": 17 }, + { "@id": "ex:carol", "ex:name": "Carol" }, + ], + })) + .commit() + .await + .expect("seed"); + fluree +} + +fn params(value: JsonValue) -> SparqlParamMap { + value.as_object().expect("an object").clone() +} + +async fn rows( + fluree: &Fluree, + db: &GraphDb, + sparql: &str, + params: Option, +) -> fluree_db_api::Result> { + let options = match params { + Some(params) => QueryExecutionOptions::new().with_params(params), + None => QueryExecutionOptions::new(), + }; + let result = fluree + .query_with_options(db, format!("{PREFIX}{sparql}").as_str(), options) + .await?; + let mut rows = match result + .to_jsonld_async(db.as_graph_db_ref()) + .await + .expect("format") + { + JsonValue::Array(rows) => rows, + other => vec![other], + }; + rows.sort_by_key(ToString::to_string); + Ok(rows) +} + +async fn head(fluree: &Fluree) -> GraphDb { + fluree.db(LEDGER).await.expect("db") +} + +#[tokio::test] +async fn parameters_run_as_the_inline_query() { + let fluree = seeded().await; + let db = head(&fluree).await; + let parameterized = + "SELECT ?s ?age WHERE { ?s ex:name $name ; ex:age ?age FILTER(?age > $min) }"; + let inline = "SELECT ?s ?age WHERE { ?s ex:name \"Alice\" ; ex:age ?age FILTER(?age > 21) }"; + let values = params(json!({ "name": "Alice", "min": 21 })); + + let with_params = rows(&fluree, &db, parameterized, Some(values.clone())) + .await + .unwrap(); + assert_eq!(with_params, vec![json!(["ex:alice", 30])]); + assert_eq!(with_params, rows(&fluree, &db, inline, None).await.unwrap()); + + let explained = fluree + .explain_sparql_with_params(&db, &format!("{PREFIX}{parameterized}"), Some(&values)) + .await + .unwrap(); + let explained_inline = fluree + .explain_sparql(&db, &format!("{PREFIX}{inline}")) + .await + .unwrap(); + assert_eq!(explained["plan"], explained_inline["plan"]); +} + +/// A trailing `VALUES` join would leave `$min` unbound inside the OPTIONAL, +/// so its filter would drop every age. +#[tokio::test] +async fn a_filter_inside_optional_sees_the_value() { + let fluree = seeded().await; + let db = head(&fluree).await; + let rows = rows( + &fluree, + &db, + "SELECT ?name ?age WHERE { ?s ex:name ?name OPTIONAL { ?s ex:age ?age FILTER(?age >= $min) } }", + Some(params(json!({ "min": 18 }))), + ) + .await + .unwrap(); + assert_eq!( + rows, + vec![ + json!(["Alice", 30]), + json!(["Bob", null]), + json!(["Carol", null]) + ] + ); +} + +#[tokio::test] +async fn a_projected_parameter_is_a_column() { + let fluree = seeded().await; + let db = head(&fluree).await; + let projected = rows( + &fluree, + &db, + "SELECT $name ?age WHERE { ?s ex:name $name ; ex:age ?age }", + Some(params(json!({ "name": "Bob" }))), + ) + .await + .unwrap(); + assert_eq!(projected, vec![json!(["Bob", 17])]); + + let grouped = rows( + &fluree, + &db, + "SELECT ?name (COUNT(?s) AS ?n) WHERE { ?s ex:name ?name } GROUP BY ?name", + Some(params(json!({ "name": "Bob" }))), + ) + .await + .unwrap(); + assert_eq!(grouped, vec![json!(["Bob", 1])]); +} + +#[tokio::test] +async fn a_misspelt_parameter_or_a_json_ld_query_is_refused() { + let fluree = seeded().await; + let db = head(&fluree).await; + let err = rows( + &fluree, + &db, + "SELECT ?s WHERE { ?s ex:name $name }", + Some(params(json!({ "nmae": "Alice" }))), + ) + .await + .unwrap_err(); + assert!(err.to_string().contains("not a variable"), "{err}"); + + let jsonld = + json!({ "select": ["?s"], "where": { "@id": "?s", "http://example.org/name": "?n" } }); + let Err(err) = fluree + .query_with_options( + &db, + &jsonld, + QueryExecutionOptions::new().with_params(params(json!({ "n": "Alice" }))), + ) + .await + else { + panic!("a JSON-LD query takes no parameters"); + }; + assert!(err.to_string().contains("SPARQL"), "{err}"); +} + +#[test] +fn empty_parameters_are_no_parameters() { + let options = QueryExecutionOptions::new().with_params(SparqlParamMap::new()); + assert!(format!("{options:?}").contains("params: None")); +} + +#[tokio::test] +async fn updates_take_parameters() { + let fluree = seeded().await; + let birthday = format!( + "{PREFIX}DELETE {{ ?s ex:age ?old }} INSERT {{ ?s ex:age $age }} \ + WHERE {{ ?s ex:name $name ; ex:age ?old }}" + ); + let bob = params(json!({ "name": "Bob", "age": 18 })); + fluree + .graph(LEDGER) + .transact() + .sparql_update_with_params(&birthday, &bob) + .commit() + .await + .unwrap(); + + let mut txn = fluree + .begin_transaction(LEDGER, fluree_db_api::TransactionOptions::default()) + .await + .unwrap(); + txn.stage(TxnOperation::SparqlUpdate( + birthday.clone(), + Some(params(json!({ "name": "Alice", "age": 31 }))), + )) + .await + .unwrap(); + txn.commit(Default::default()).await.unwrap(); + + let db = head(&fluree).await; + let ages = rows( + &fluree, + &db, + "SELECT ?name ?age WHERE { ?s ex:name ?name ; ex:age ?age }", + None, + ) + .await + .unwrap(); + assert_eq!(ages, vec![json!(["Alice", 31]), json!(["Bob", 18])]); + + let Err(err) = fluree + .graph(LEDGER) + .transact() + .sparql_update_with_params(&birthday, ¶ms(json!({ "nmae": "Bob", "age": 1 }))) + .commit() + .await + else { + panic!("a misspelt parameter is refused"); + }; + assert!(err.to_string().contains("not a variable"), "{err}"); +} + +#[tokio::test] +async fn a_connection_query_takes_parameters() { + let fluree = seeded().await; + let result = fluree + .query_from() + .sparql(&format!( + "{PREFIX}SELECT ?age FROM <{LEDGER}> WHERE {{ ?s ex:name $name ; ex:age ?age }}" + )) + .params(params(json!({ "name": "Alice" }))) + .format(FormatterConfig::sparql_json()) + .execute_formatted() + .await + .unwrap(); + let bindings = result["results"]["bindings"].as_array().unwrap(); + assert_eq!(bindings.len(), 1, "{result}"); + assert_eq!(bindings[0]["age"]["value"], "30"); +} + +#[tokio::test] +async fn a_blank_node_parameter_is_one_stored_node() { + let fluree = seeded().await; + fluree + .graph(LEDGER) + .transact() + .insert(&json!({ "http://example.org/name": "Dan" })) + .commit() + .await + .unwrap(); + let db = head(&fluree).await; + let dan = rows(&fluree, &db, "SELECT ?s WHERE { ?s ex:name \"Dan\" }", None) + .await + .unwrap(); + let dan = dan[0][0].as_str().expect("a blank node id").to_string(); + assert!(dan.starts_with("_:fdb-"), "{dan}"); + + // A label that isn't a stored id would lower to a variable and match every node. + for label in [ + json!({ "@id": "_:x" }), + json!({ "@value": "_:x", "@type": "@id" }), + ] { + let err = rows( + &fluree, + &db, + "SELECT ?n WHERE { $s ex:name ?n }", + Some(params(json!({ "s": label }))), + ) + .await + .unwrap_err(); + assert!(err.to_string().contains("blank node label"), "{err}"); + } + let Err(err) = fluree + .graph(LEDGER) + .transact() + .sparql_update_with_params( + &format!("{PREFIX}DELETE WHERE {{ $s ?p ?o }}"), + ¶ms(json!({ "s": { "@id": "_:x" } })), + ) + .commit() + .await + else { + panic!("a blank node label is refused"); + }; + assert!(err.to_string().contains("blank node label"), "{err}"); + + fluree + .graph(LEDGER) + .transact() + .sparql_update_with_params( + &format!("{PREFIX}DELETE WHERE {{ $s ?p ?o }}"), + ¶ms(json!({ "s": { "@id": dan } })), + ) + .commit() + .await + .unwrap(); + let db = head(&fluree).await; + let names = rows(&fluree, &db, "SELECT ?n WHERE { ?s ex:name ?n }", None) + .await + .unwrap(); + assert_eq!( + names, + vec![json!(["Alice"]), json!(["Bob"]), json!(["Carol"])] + ); +} + +#[tokio::test] +async fn an_id_typed_value_is_an_iri() { + let fluree = seeded().await; + let db = head(&fluree).await; + let sparql = "SELECT ?n WHERE { $s ex:name ?n }"; + let typed = params(json!({ "s": { "@value": "http://example.org/alice", "@type": "@id" } })); + let id = params(json!({ "s": { "@id": "http://example.org/alice" } })); + let names = rows(&fluree, &db, sparql, Some(typed)).await.unwrap(); + assert_eq!(names, vec![json!(["Alice"])]); + assert_eq!(names, rows(&fluree, &db, sparql, Some(id)).await.unwrap()); +} + +#[tokio::test] +async fn a_vector_parameter_scores_as_the_inline_vector() { + let fluree = seeded().await; + fluree + .graph(LEDGER) + .transact() + .insert(&json!({ + "@context": { "ex": "http://example.org/" }, + "@graph": [ + { "@id": "ex:alice", "ex:embedding": { "@value": [0.8, 0.14, 0.9], "@type": "@vector" } }, + { "@id": "ex:bob", "ex:embedding": { "@value": [0.1, 0.9, 0.3], "@type": "@vector" } }, + ], + })) + .commit() + .await + .unwrap(); + let db = head(&fluree).await; + let scored = |q: &str| { + format!( + "SELECT ?s ?score WHERE {{ ?s ex:embedding ?v BIND(dotProduct(?v, {q}) AS ?score) }}" + ) + }; + let inline = rows( + &fluree, + &db, + &scored("\"[0.5, 0.25, 1.0]\"^^"), + None, + ) + .await + .unwrap(); + let vector = + json!({ "@value": [0.5, 0.25, 1.0], "@type": "https://ns.flur.ee/db#embeddingVector" }); + let with_param = rows( + &fluree, + &db, + &scored("$q"), + Some(params(json!({ "q": vector }))), + ) + .await + .unwrap(); + assert_eq!(inline.len(), 2, "{inline:?}"); + assert_eq!(with_param, inline); +} diff --git a/fluree-db-api/tests/it_sync_graph.rs b/fluree-db-api/tests/it_sync_graph.rs index 8fab956d7b..f3c7cc700e 100644 --- a/fluree-db-api/tests/it_sync_graph.rs +++ b/fluree-db-api/tests/it_sync_graph.rs @@ -639,3 +639,32 @@ async fn graph_management_verbs_work_on_indexed_data() { }) .await; } + +#[tokio::test] +async fn a_sync_commit_carries_its_message_and_reports_its_id() { + let fluree = FlureeBuilder::memory().build_memory(); + let ledger_id = "it/sync-graph/message:main"; + seed(&fluree, ledger_id).await; + let report = fluree + .sync_named_graph( + ledger_id, + ONT_IRI, + &payload_v1(), + SyncGraphOpts { + message: Some("nightly ontology".to_string()), + ..Default::default() + }, + ) + .await + .expect("sync"); + let (log, _) = fluree.commit_log(ledger_id, Some(1)).await.expect("log"); + assert_eq!(report.commit_id.as_ref(), Some(&log[0].commit_id)); + assert_eq!(log[0].message.as_deref(), Some("nightly ontology")); + + let unchanged = fluree + .sync_named_graph(ledger_id, ONT_IRI, &payload_v1(), SyncGraphOpts::default()) + .await + .expect("identical sync"); + assert!(!unchanged.committed); + assert_eq!(unchanged.commit_id, None); +} diff --git a/fluree-db-api/tests/it_transaction.rs b/fluree-db-api/tests/it_transaction.rs index 5076131b2a..0a50c88320 100644 --- a/fluree-db-api/tests/it_transaction.rs +++ b/fluree-db-api/tests/it_transaction.rs @@ -80,7 +80,7 @@ async fn operations_see_earlier_ones_and_commit_once() { let mut txn = begin(&fluree).await; txn.stage(insert("alice", "Alice", 30)).await.unwrap(); // Reads alice's age from the insert above. - txn.stage(TxnOperation::SparqlUpdate(BIRTHDAY.into())) + txn.stage(TxnOperation::SparqlUpdate(BIRTHDAY.into(), None)) .await .unwrap(); txn.stage(TxnOperation::InsertTurtle( @@ -117,6 +117,7 @@ async fn a_failed_operation_leaves_the_transaction_as_it_was() { txn.stage(insert("alice", "Alice", 30)).await.unwrap(); txn.stage(TxnOperation::SparqlUpdate( "INSERT DATA { not sparql".into(), + None, )) .await .expect_err("parse error"); @@ -144,7 +145,7 @@ async fn rollback_to_a_savepoint_discards_the_operations_after_it() { txn.rollback_to(savepoint).await.unwrap(); assert_eq!(txn.len(), 1); // The staged state is rebuilt too, so later operations see only alice. - txn.stage(TxnOperation::SparqlUpdate(BIRTHDAY.into())) + txn.stage(TxnOperation::SparqlUpdate(BIRTHDAY.into(), None)) .await .unwrap(); assert_eq!( @@ -163,9 +164,10 @@ async fn net_zero_and_empty_transactions_commit_nothing() { let fluree = fluree().await; let mut txn = begin(&fluree).await; txn.stage(insert("alice", "Alice", 30)).await.unwrap(); - txn.stage(TxnOperation::SparqlUpdate(format!( - "{PREFIX}DELETE DATA {{ ex:alice ex:name \"Alice\" ; ex:age 30 }}" - ))) + txn.stage(TxnOperation::SparqlUpdate( + format!("{PREFIX}DELETE DATA {{ ex:alice ex:name \"Alice\" ; ex:age 30 }}"), + None, + )) .await .unwrap(); let result = txn.commit(CommitOpts::default()).await.unwrap(); @@ -213,7 +215,7 @@ async fn restages_over_a_concurrent_write_its_update_matches() { insert_alice(&fluree, 30).await; let mut txn = begin(&fluree).await; - txn.stage(TxnOperation::SparqlUpdate(BIRTHDAY.into())) + txn.stage(TxnOperation::SparqlUpdate(BIRTHDAY.into(), None)) .await .unwrap(); set_alice_age(&fluree, 50).await; @@ -262,7 +264,7 @@ async fn a_read_transaction_conflicts_when_the_ledger_moved() { // Unmoved, a read transaction commits as usual. let mut txn = begin(&fluree).await; txn.db().await.unwrap(); - txn.stage(TxnOperation::SparqlUpdate(BIRTHDAY.into())) + txn.stage(TxnOperation::SparqlUpdate(BIRTHDAY.into(), None)) .await .unwrap(); txn.commit(CommitOpts::default()).await.unwrap(); @@ -315,6 +317,7 @@ async fn cypher_and_sparql_mix_in_one_commit() { assert!(returned.is_none()); txn.stage(TxnOperation::SparqlUpdate( r#"INSERT { ?p 30 } WHERE { ?p "Ann" }"#.into(), + None, )) .await .unwrap(); @@ -407,9 +410,10 @@ async fn a_failed_operation_keeps_the_values_already_staged() { }))) .await .unwrap(); - txn.stage(TxnOperation::SparqlUpdate(format!( - "{PREFIX}INSERT {{ ex:order ex:token ?t }} WHERE {{ BIND(STRUUID() AS ?t) }}" - ))) + txn.stage(TxnOperation::SparqlUpdate( + format!("{PREFIX}INSERT {{ ex:order ex:token ?t }} WHERE {{ BIND(STRUUID() AS ?t) }}"), + None, + )) .await .unwrap(); let staged = select(&fluree, &txn.db().await.unwrap(), ORDER).await; @@ -417,6 +421,7 @@ async fn a_failed_operation_keeps_the_values_already_staged() { txn.stage(TxnOperation::SparqlUpdate( "INSERT DATA { not sparql".into(), + None, )) .await .expect_err("parse error"); @@ -676,9 +681,10 @@ async fn netting_sees_a_named_graph_value_held_only_in_the_index() { let fluree = FlureeBuilder::file(path).build().expect("cold reload"); let mut txn = begin(&fluree).await; for op in ["INSERT", "DELETE"] { - txn.stage(TxnOperation::SparqlUpdate(format!( - "{PREFIX}{op} DATA {{ {VALUE} }}" - ))) + txn.stage(TxnOperation::SparqlUpdate( + format!("{PREFIX}{op} DATA {{ {VALUE} }}"), + None, + )) .await .unwrap(); } diff --git a/fluree-db-cli/src/commands/graph_sync.rs b/fluree-db-cli/src/commands/graph_sync.rs index 1936cd3d96..303602fc04 100644 --- a/fluree-db-cli/src/commands/graph_sync.rs +++ b/fluree-db-cli/src/commands/graph_sync.rs @@ -180,6 +180,7 @@ pub async fn run(a: SyncArgs<'_>) -> CliResult<()> { SyncGraphOpts { dry_run: a.dry_run, allow_empty: a.allow_empty, + ..Default::default() }, TxnOpts::default(), policy_ctx, diff --git a/fluree-db-server/src/routes/transact.rs b/fluree-db-server/src/routes/transact.rs index 0987f0479b..994ee7fd28 100644 --- a/fluree-db-server/src/routes/transact.rs +++ b/fluree-db-server/src/routes/transact.rs @@ -1539,6 +1539,7 @@ async fn dry_run_sync( let opts = fluree_db_api::SyncGraphOpts { dry_run: true, allow_empty, + ..Default::default() }; let report = state .fluree diff --git a/fluree-db-sparql/src/ast/expr.rs b/fluree-db-sparql/src/ast/expr.rs index 3a3e794128..2b59fcf0cf 100644 --- a/fluree-db-sparql/src/ast/expr.rs +++ b/fluree-db-sparql/src/ast/expr.rs @@ -474,6 +474,9 @@ pub enum FunctionName { CosineSimilarity, EuclideanDistance, + /// BM25 relevance of a full-text value to a query string (Fluree extension) + Fulltext, + // SPARQL 1.2 triple-term functions (RDF-star). Accept-then-defer per // burn-down decision D-1: parsed and arity-validated here, but lowering // rejects them with `not_implemented` (no evaluable capability yet). @@ -550,6 +553,7 @@ impl FunctionName { "EUCLIDEANDISTANCE" | "EUCLIDEAN_DISTANCE" | "EUCLIDIANDISTANCE" => { Some(FunctionName::EuclideanDistance) } + "FULLTEXT" | "FULL_TEXT" => Some(FunctionName::Fulltext), // SPARQL 1.2 triple-term functions (case-insensitive, matching the // grammar's BuiltInCall keywords). `isTRIPLE` uppercases to // `ISTRIPLE`. diff --git a/fluree-db-sparql/src/lex/token.rs b/fluree-db-sparql/src/lex/token.rs index 5554e67965..5354218012 100644 --- a/fluree-db-sparql/src/lex/token.rs +++ b/fluree-db-sparql/src/lex/token.rs @@ -248,6 +248,9 @@ pub enum TokenKind { KwCosineSimilarity, KwEuclideanDistance, + // Full-text scoring (Fluree extension) + KwFulltext, + // Prologue KwBase, KwPrefix, @@ -497,6 +500,7 @@ impl TokenKind { | TokenKind::KwDotProduct | TokenKind::KwCosineSimilarity | TokenKind::KwEuclideanDistance + | TokenKind::KwFulltext ) } @@ -633,6 +637,7 @@ impl TokenKind { TokenKind::KwDotProduct => Some("dotProduct"), TokenKind::KwCosineSimilarity => Some("cosineSimilarity"), TokenKind::KwEuclideanDistance => Some("euclideanDistance"), + TokenKind::KwFulltext => Some("fulltext"), _ => None, } } @@ -796,6 +801,7 @@ pub fn keyword_from_str(s: &str) -> Option { "EUCLIDEANDISTANCE" | "EUCLIDEAN_DISTANCE" | "EUCLIDIANDISTANCE" => { Some(TokenKind::KwEuclideanDistance) } + "FULLTEXT" | "FULL_TEXT" => Some(TokenKind::KwFulltext), // SPARQL 1.2 triple-term builtins are recognized as contextual // function-call identifiers, not reserved keywords: emit a generic // `TripleTermFn` carrying the canonical name so the expression parser diff --git a/fluree-db-sparql/src/lib.rs b/fluree-db-sparql/src/lib.rs index de18bbaaea..730ffc2160 100644 --- a/fluree-db-sparql/src/lib.rs +++ b/fluree-db-sparql/src/lib.rs @@ -36,6 +36,7 @@ pub mod ast; pub mod diag; pub mod lex; +pub mod params; pub mod parse; pub mod protocol; pub mod span; @@ -47,6 +48,7 @@ pub mod lower; // Re-exports pub use ast::{MetaPragma, Pragmas, Prologue, QueryBody, SparqlAst, UpdateOperation}; pub use diag::{DiagCode, Diagnostic, ParseOutput, Severity}; +pub use params::{substitute_params, ParamError, ParamMap}; pub use parse::{parse_sparql, pragma_names, request_pragmas}; pub use span::SourceSpan; pub use validate::{validate, Capabilities}; diff --git a/fluree-db-sparql/src/lower/expression.rs b/fluree-db-sparql/src/lower/expression.rs index 02f162ad29..225ed5eb3b 100644 --- a/fluree-db-sparql/src/lower/expression.rs +++ b/fluree-db-sparql/src/lower/expression.rs @@ -13,6 +13,9 @@ use fluree_vocab::xsd; use super::{LowerError, LoweringContext, Result}; +/// `f:fulltext(?text, "query")`, the namespaced spelling of `fulltext`. +const FULLTEXT_FUNCTION: &str = "https://ns.flur.ee/db#fulltext"; + impl LoweringContext<'_, E> { pub(super) fn lower_expression(&mut self, expr: &AstExpression) -> Result { match expr { @@ -328,6 +331,7 @@ impl LoweringContext<'_, E> { FunctionName::DotProduct => Function::DotProduct, FunctionName::CosineSimilarity => Function::CosineSimilarity, FunctionName::EuclideanDistance => Function::EuclideanDistance, + FunctionName::Fulltext => Function::Fulltext, // Handled by the `not_implemented` early return above. FunctionName::Triple @@ -345,6 +349,7 @@ impl LoweringContext<'_, E> { "http://www.opengis.net/def/function/geosparql/distance" => { Function::GeofDistance } + FULLTEXT_FUNCTION => Function::Fulltext, // XSD datatype constructor (cast) functions — W3C SPARQL 1.1 §17.5 xsd::BOOLEAN => Function::XsdBoolean, xsd::INTEGER => Function::XsdInteger, diff --git a/fluree-db-sparql/src/params.rs b/fluree-db-sparql/src/params.rs new file mode 100644 index 0000000000..649fac6ff1 --- /dev/null +++ b/fluree-db-sparql/src/params.rs @@ -0,0 +1,1113 @@ +//! Query parameters. +//! +//! A parameter fixes a variable to one RDF term for the whole request: +//! `SELECT ?s WHERE { ?s ex:name $name }` with `name = "Alice"` runs as if +//! `"Alice"` were written in place of `$name` (`?name` and `$name` are the +//! same variable). Substitution rewrites the parsed AST before lowering, so +//! the planner only ever sees constants and a request without parameters +//! never reaches this module. +//! +//! The rules that keep the rewrite equivalent to writing the value inline: +//! +//! - every occurrence is replaced, subqueries and update templates included; +//! - a projected parameter stays a column, as `(value AS ?name)` (or as the +//! `GROUP BY` key it is grouped on); `SELECT *` no longer lists it; +//! - a parameter the query assigns itself (`BIND`, `VALUES`, `AS`) is an error, +//! as is one the query never mentions — a misspelt name would otherwise +//! leave its variable unbound and silently widen the match. +//! +//! Values use the JSON-LD forms: a JSON string, number or boolean; +//! `{"@id": iri}`; `{"@value": v, "@type": iri}` (`"@type": "@id"` reads `v` +//! as an IRI); `{"@value": s, "@language": tag}`. IRIs are full IRIs: no +//! prefix or `@context` applies. A blank node is a stored node's `_:fdb-…` +//! id; any other label would lower to a variable and match every node. An +//! `f:embeddingVector` takes its numbers as a JSON array. + +use std::collections::HashMap; +use std::sync::Arc; + +use serde_json::Value as JsonValue; + +use crate::ast::annotation::{Annotation, AnnotationVerb, ReifierId, TripleTerm}; +use crate::ast::pattern::{ServiceEndpoint, SubSelect}; +use crate::ast::query::DescribeTarget; +use crate::ast::{ + BlankNode, Expression, GraphName, GraphPattern, GroupCondition, Iri, IriValue, Literal, + LiteralValue, OrderExpr, PredicateTerm, QuadPatternElement, QueryBody, QuotedTriple, + SelectVariable, SelectVariables, SolutionModifiers, SparqlAst, SubjectTerm, Term, + TriplePattern, UpdateOperation, Var, VarOrIri, +}; +use crate::span::SourceSpan; +use crate::validate::STABLE_BLANK_NODE_LABEL_PREFIX; + +/// SERVICE endpoints under this prefix query a ledger of this instance from +/// the lowered patterns, not from the body text. +const LOCAL_LEDGER_SERVICE: &str = "fluree:ledger:"; + +/// Parameter name (without `?` or `$`) → JSON-LD value. +pub type ParamMap = serde_json::Map; + +/// A parameter that can't be substituted. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ParamError { + pub name: String, + pub reason: String, +} + +impl ParamError { + fn new(name: &str, reason: impl Into) -> Self { + Self { + name: name.to_string(), + reason: reason.into(), + } + } +} + +impl std::fmt::Display for ParamError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!(f, "parameter `${}` {}", self.name, self.reason) + } +} + +impl std::error::Error for ParamError {} + +type Result = std::result::Result; + +/// Replace every variable named in `params` with its value. +pub fn substitute_params(ast: &mut SparqlAst, params: &ParamMap) -> Result<()> { + if params.is_empty() { + return Ok(()); + } + let mut subst = Substitution::new(params)?; + match &mut ast.body { + QueryBody::Select(q) => subst.select( + &mut q.select.variables, + &mut q.where_clause.pattern, + &mut q.modifiers, + q.values.as_deref_mut(), + )?, + QueryBody::Construct(q) => { + if let Some(template) = &mut q.template { + for triple in &mut template.triples { + subst.triple(triple)?; + } + for graph in template.graphs.iter_mut().flatten() { + subst.graph_name(graph)?; + } + } + subst.pattern(&mut q.where_clause.pattern)?; + subst.modifiers(&mut q.modifiers, &[])?; + } + QueryBody::Ask(q) => { + subst.pattern(&mut q.where_clause.pattern)?; + subst.modifiers(&mut q.modifiers, &[])?; + } + QueryBody::Describe(q) => { + if let DescribeTarget::Resources(resources) = &mut q.target { + for resource in resources { + if let VarOrIri::Var(v) = resource { + if let Some(iri) = subst.iri(v, "is described")? { + *resource = VarOrIri::Iri(iri); + } + } + } + } + if let Some(where_clause) = &mut q.where_clause { + subst.pattern(&mut where_clause.pattern)?; + } + subst.modifiers(&mut q.modifiers, &[])?; + } + QueryBody::Update(request) => { + for op in &mut request.operations { + match &mut op.operation { + UpdateOperation::InsertData(d) => subst.quads(&mut d.data.quads)?, + UpdateOperation::DeleteData(d) => subst.quads(&mut d.data.quads)?, + UpdateOperation::DeleteWhere(d) => subst.quads(&mut d.pattern.patterns)?, + UpdateOperation::Modify(m) => { + if let Some(delete) = &mut m.delete_clause { + subst.quads(&mut delete.patterns)?; + } + if let Some(insert) = &mut m.insert_clause { + subst.quads(&mut insert.patterns)?; + } + subst.pattern(&mut m.where_clause)?; + } + UpdateOperation::Load(_) + | UpdateOperation::Clear(_) + | UpdateOperation::Drop(_) + | UpdateOperation::Create(_) + | UpdateOperation::Add(_) + | UpdateOperation::Copy(_) + | UpdateOperation::Move(_) => {} + } + } + } + } + subst.finish() +} + +#[derive(Clone)] +enum Value { + Iri(Arc), + Blank(Arc), + Literal(LiteralValue), +} + +impl Value { + fn parse(name: &str, json: &JsonValue) -> Result { + let literal = match json { + JsonValue::Null => { + return Err(ParamError::new(name, "is null, which is not an RDF term")); + } + JsonValue::Bool(b) => LiteralValue::Boolean(*b), + JsonValue::Number(n) => match n.as_i64() { + Some(i) => LiteralValue::Integer(i), + None if n.is_u64() => LiteralValue::BigInteger(Arc::from(n.to_string())), + None => LiteralValue::Double(n.as_f64().unwrap_or(f64::NAN)), + }, + JsonValue::String(s) => LiteralValue::Simple(Arc::from(s.as_str())), + JsonValue::Array(_) => { + return Err(ParamError::new( + name, + "is a list; a parameter is a single RDF term", + )); + } + JsonValue::Object(object) => return Self::parse_object(name, object), + }; + Ok(Value::Literal(literal)) + } + + fn parse_object(name: &str, object: &serde_json::Map) -> Result { + let unexpected = |allowed: &[&str]| { + object + .keys() + .find(|k| !allowed.contains(&k.as_str())) + .map(|k| ParamError::new(name, format!("has an unexpected key `{k}`"))) + }; + if let Some(id) = object.get("@id") { + if let Some(err) = unexpected(&["@id"]) { + return Err(err); + } + let Some(id) = id.as_str() else { + return Err(ParamError::new(name, "has an `@id` that is not a string")); + }; + return Self::node(name, id); + } + let Some(value) = object.get("@value") else { + return Err(ParamError::new( + name, + "is an object without `@id` or `@value`", + )); + }; + if let Some(err) = unexpected(&["@value", "@type", "@language"]) { + return Err(err); + } + match (object.get("@type"), object.get("@language")) { + (Some(_), Some(_)) => Err(ParamError::new(name, "has both `@type` and `@language`")), + (None, Some(lang)) => match (value.as_str(), lang.as_str()) { + (Some(value), Some(lang)) => Ok(Value::Literal(LiteralValue::LangTagged { + value: Arc::from(value), + lang: Arc::from(lang), + })), + _ => Err(ParamError::new( + name, + "has a language tag, so `@value` and `@language` must be strings", + )), + }, + (Some(datatype), None) => { + let Some(datatype) = datatype.as_str() else { + return Err(ParamError::new(name, "has a `@type` that is not a string")); + }; + if datatype == "@id" { + return match value.as_str() { + Some(id) => Self::node(name, id), + None => Err(ParamError::new( + name, + "has `\"@type\": \"@id\"`, so its `@value` must be a string", + )), + }; + } + if datatype.starts_with('@') { + return Err(ParamError::new( + name, + format!( + "has a `@type` of `{datatype}`; only `@id` or a datatype IRI is a type" + ), + )); + } + let lexical = match value { + JsonValue::String(s) => s.clone(), + JsonValue::Number(n) => n.to_string(), + JsonValue::Bool(b) => b.to_string(), + JsonValue::Array(items) + if datatype == fluree_vocab::fluree::EMBEDDING_VECTOR => + { + if !items.iter().all(JsonValue::is_number) { + return Err(ParamError::new( + name, + "is a vector whose items are not all numbers", + )); + } + value.to_string() + } + _ => { + return Err(ParamError::new( + name, + "has a `@value` that is not a string, number or boolean \ + (a list is only an `f:embeddingVector`)", + )); + } + }; + Ok(Value::Literal(LiteralValue::Typed { + value: Arc::from(lexical), + datatype: Box::new(Iri::full(datatype, SourceSpan::new(0, 0))), + })) + } + (None, None) if value.is_object() || value.is_array() => Err(ParamError::new( + name, + "has a `@value` that is not a string, number or boolean", + )), + (None, None) => Self::parse(name, value), + } + } + + /// An IRI, or a stored node's `_:fdb-…` id. Any other blank-node label + /// lowers to a non-distinguished variable, which fixes nothing. + fn node(name: &str, id: &str) -> Result { + match id.strip_prefix("_:") { + Some(label) if label.starts_with(STABLE_BLANK_NODE_LABEL_PREFIX) => { + Ok(Value::Blank(Arc::from(label))) + } + Some(_) => Err(ParamError::new( + name, + "is a blank node label, which would match any node; \ + only a stored node's `_:fdb-…` id can be a parameter", + )), + None => Ok(Value::Iri(Arc::from(id))), + } + } + + fn term(self, span: SourceSpan) -> Term { + match self { + Value::Iri(iri) => Term::Iri(Iri::full(iri, span)), + Value::Blank(label) => Term::BlankNode(BlankNode::labeled(label, span)), + Value::Literal(value) => Term::Literal(Literal { value, span }), + } + } +} + +struct Param { + value: Value, + used: bool, +} + +struct Substitution<'p> { + params: HashMap<&'p str, Param>, + /// Inside the body of a SERVICE that is not a local ledger: that body is + /// sent to the endpoint as written, so a substituted value would not reach it. + in_remote_service: bool, +} + +impl<'p> Substitution<'p> { + fn new(params: &'p ParamMap) -> Result { + let mut by_name = HashMap::with_capacity(params.len()); + for (key, json) in params { + let name = key.trim_start_matches(['?', '$']); + let value = Value::parse(name, json)?; + if by_name.insert(name, Param { value, used: false }).is_some() { + return Err(ParamError::new( + name, + format!("is given twice (`{name}`, `?{name}` and `${name}` are one variable)"), + )); + } + } + Ok(Self { + params: by_name, + in_remote_service: false, + }) + } + + fn finish(self) -> Result<()> { + let mut unused: Vec<&str> = self + .params + .iter() + .filter(|(_, p)| !p.used) + .map(|(name, _)| *name) + .collect(); + unused.sort_unstable(); + match unused.first() { + Some(name) => Err(ParamError::new(name, "is not a variable in the query")), + None => Ok(()), + } + } + + /// The value for `var`, when it is a parameter. + fn value(&mut self, var: &Var) -> Result> { + let Some(param) = self.params.get_mut(var.name.as_ref()) else { + return Ok(None); + }; + param.used = true; + if self.in_remote_service { + return Err(ParamError::new( + &var.name, + "is inside a remote SERVICE, whose body is sent to the endpoint as written", + )); + } + Ok(Some(param.value.clone())) + } + + /// Reject a parameter in a position that binds its variable. + fn assigned(&mut self, var: &Var) -> Result<()> { + match self.value(var)? { + Some(_) => Err(ParamError::new( + &var.name, + "is assigned in the query (by BIND, VALUES or AS), so it can't be a parameter", + )), + None => Ok(()), + } + } + + fn iri(&mut self, var: &Var, position: &str) -> Result> { + match self.value(var)? { + None => Ok(None), + Some(Value::Iri(iri)) => Ok(Some(Iri::full(iri, var.span))), + Some(_) => Err(ParamError::new( + &var.name, + format!("{position}, so it must be an IRI"), + )), + } + } + + fn subject(&mut self, subject: &mut SubjectTerm) -> Result<()> { + match subject { + SubjectTerm::Var(v) => match self.value(v)? { + None => {} + Some(Value::Iri(iri)) => *subject = SubjectTerm::Iri(Iri::full(iri, v.span)), + Some(Value::Blank(label)) => { + *subject = SubjectTerm::BlankNode(BlankNode::labeled(label, v.span)); + } + Some(Value::Literal(_)) => { + return Err(ParamError::new( + &v.name, + "is a subject, so it must be an IRI or a blank node", + )); + } + }, + SubjectTerm::QuotedTriple(q) => self.quoted(q)?, + SubjectTerm::TripleTerm(t) => self.triple_term(t)?, + SubjectTerm::Iri(_) | SubjectTerm::BlankNode(_) => {} + } + Ok(()) + } + + fn predicate(&mut self, predicate: &mut PredicateTerm) -> Result<()> { + if let PredicateTerm::Var(v) = predicate { + if let Some(iri) = self.iri(v, "is a predicate")? { + *predicate = PredicateTerm::Iri(iri); + } + } + Ok(()) + } + + fn object(&mut self, object: &mut Term) -> Result<()> { + match object { + Term::Var(v) => { + if let Some(value) = self.value(v)? { + *object = value.term(v.span); + } + } + Term::QuotedTriple(q) => self.quoted(q)?, + Term::TripleTerm(t) => self.triple_term(t)?, + Term::Iri(_) | Term::Literal(_) | Term::BlankNode(_) => {} + } + Ok(()) + } + + fn graph_name(&mut self, graph: &mut GraphName) -> Result<()> { + if let GraphName::Var(v) = graph { + if let Some(iri) = self.iri(v, "names a graph")? { + *graph = GraphName::Iri(iri); + } + } + Ok(()) + } + + fn reifier(&mut self, reifier: &mut ReifierId) -> Result<()> { + if let ReifierId::Var(v) = reifier { + match self.value(v)? { + None => {} + Some(Value::Iri(iri)) => *reifier = ReifierId::Iri(Iri::full(iri, v.span)), + Some(Value::Blank(label)) => { + *reifier = ReifierId::BlankNode(BlankNode::labeled(label, v.span)); + } + Some(Value::Literal(_)) => { + return Err(ParamError::new( + &v.name, + "is a reifier, so it must be an IRI or a blank node", + )); + } + } + } + Ok(()) + } + + fn quoted(&mut self, quoted: &mut QuotedTriple) -> Result<()> { + self.subject(&mut quoted.subject)?; + self.predicate(&mut quoted.predicate)?; + self.object(&mut quoted.object)?; + if let Some(id) = quoted.reifier.as_mut().and_then(|r| r.id.as_mut()) { + self.reifier(id)?; + } + Ok(()) + } + + fn triple_term(&mut self, triple: &mut TripleTerm) -> Result<()> { + self.subject(&mut triple.subject)?; + self.predicate(&mut triple.predicate)?; + self.object(&mut triple.object) + } + + fn triple(&mut self, triple: &mut TriplePattern) -> Result<()> { + self.subject(&mut triple.subject)?; + self.predicate(&mut triple.predicate)?; + self.object(&mut triple.object)?; + if let Some(annotation) = &mut triple.annotation { + self.annotation(annotation)?; + } + Ok(()) + } + + fn annotation(&mut self, annotation: &mut Annotation) -> Result<()> { + for unit in &mut annotation.units { + if let Some(reifier) = &mut unit.reifier { + self.reifier(reifier)?; + } + for entry in unit.block.iter_mut().flat_map(|b| &mut b.entries) { + if let AnnotationVerb::Simple(predicate) = &mut entry.verb { + self.predicate(predicate)?; + } + self.object(&mut entry.object)?; + } + } + Ok(()) + } + + fn quads(&mut self, quads: &mut [QuadPatternElement]) -> Result<()> { + for quad in quads { + match quad { + QuadPatternElement::Triple(triple) => self.triple(triple)?, + QuadPatternElement::Graph { name, triples, .. } => { + self.graph_name(name)?; + for triple in triples { + self.triple(triple)?; + } + } + } + } + Ok(()) + } + + fn pattern(&mut self, pattern: &mut GraphPattern) -> Result<()> { + match pattern { + GraphPattern::Bgp { patterns, .. } => { + for triple in patterns { + self.triple(triple)?; + } + } + GraphPattern::Group { patterns, .. } => { + for p in patterns { + self.pattern(p)?; + } + } + GraphPattern::Optional { pattern, .. } => self.pattern(pattern)?, + GraphPattern::Union { left, right, .. } | GraphPattern::Minus { left, right, .. } => { + self.pattern(left)?; + self.pattern(right)?; + } + GraphPattern::Filter { expr, .. } => self.expr(expr)?, + GraphPattern::Bind { expr, var, .. } => { + self.assigned(var)?; + self.expr(expr)?; + } + GraphPattern::Values { vars, .. } => { + for var in vars { + self.assigned(var)?; + } + } + GraphPattern::Graph { name, pattern, .. } => { + self.graph_name(name)?; + self.pattern(pattern)?; + } + GraphPattern::Service { + endpoint, pattern, .. + } => { + if let ServiceEndpoint::Var(v) = endpoint { + if let Some(iri) = self.iri(v, "names a SERVICE endpoint")? { + *endpoint = ServiceEndpoint::Iri(iri); + } + } + let local = matches!( + endpoint, + ServiceEndpoint::Iri(Iri { value: IriValue::Full(iri), .. }) + if iri.starts_with(LOCAL_LEDGER_SERVICE) + ); + let outer = std::mem::replace(&mut self.in_remote_service, !local); + let result = self.pattern(pattern); + self.in_remote_service = outer; + result?; + } + GraphPattern::SubSelect { query, .. } => { + let SubSelect { + variables, + pattern, + modifiers, + values, + .. + } = query.as_mut(); + self.select(variables, pattern, modifiers, values.as_deref_mut())?; + } + GraphPattern::Path { + subject, object, .. + } => { + self.subject(subject)?; + self.object(object)?; + } + GraphPattern::AnnotationTarget { + reifier, + predicate, + triple_term, + .. + } => { + self.subject(reifier)?; + self.predicate(predicate)?; + self.triple_term(triple_term)?; + } + } + Ok(()) + } + + fn select( + &mut self, + variables: &mut SelectVariables, + pattern: &mut GraphPattern, + modifiers: &mut SolutionModifiers, + values: Option<&mut GraphPattern>, + ) -> Result<()> { + // A parameter that is also a GROUP BY key is projected through the key. + let grouped: Vec> = modifiers + .group_by + .iter() + .flat_map(|g| &g.conditions) + .filter_map(|c| match c { + GroupCondition::Var(v) if self.params.contains_key(v.name.as_ref()) => { + Some(v.name.clone()) + } + _ => None, + }) + .collect(); + if let SelectVariables::Explicit(items) = variables { + for item in items { + match item { + SelectVariable::Var(v) if !grouped.contains(&v.name) => { + let v = v.clone(); + if let Some(value) = self.value(&v)? { + *item = SelectVariable::Expr { + expr: self.constant(&v, value)?, + span: v.span, + alias: v, + }; + } + } + SelectVariable::Var(_) => {} + SelectVariable::Expr { expr, alias, .. } => { + self.assigned(alias)?; + self.expr(expr)?; + } + } + } + } + self.pattern(pattern)?; + self.modifiers(modifiers, &grouped)?; + if let Some(values) = values { + self.pattern(values)?; + } + Ok(()) + } + + fn modifiers(&mut self, modifiers: &mut SolutionModifiers, grouped: &[Arc]) -> Result<()> { + if let Some(group_by) = &mut modifiers.group_by { + for condition in &mut group_by.conditions { + match condition { + GroupCondition::Var(v) if grouped.contains(&v.name) => { + let v = v.clone(); + let value = self.value(&v)?.expect("grouped names are parameters"); + *condition = GroupCondition::Expr { + expr: self.constant(&v, value)?, + span: v.span, + alias: Some(v), + }; + } + GroupCondition::Var(_) => {} + GroupCondition::Expr { expr, alias, .. } => { + if let Some(alias) = alias { + self.assigned(alias)?; + } + self.expr(expr)?; + } + } + } + } + if let Some(having) = &mut modifiers.having { + for condition in &mut having.conditions { + self.expr(condition)?; + } + } + if let Some(order_by) = &mut modifiers.order_by { + for condition in &mut order_by.conditions { + match &mut condition.expr { + OrderExpr::Var(v) => { + if let Some(value) = self.value(v)? { + let constant = self.constant(v, value)?; + condition.expr = OrderExpr::Expr(constant); + } + } + OrderExpr::Expr(expr) => self.expr(expr)?, + } + } + } + Ok(()) + } + + fn constant(&self, var: &Var, value: Value) -> Result { + match value.term(var.span) { + Term::Iri(iri) => Ok(Expression::Iri(iri)), + Term::Literal(literal) => Ok(Expression::Literal(literal)), + _ => Err(ParamError::new( + &var.name, + "is a blank node, which can't be used in an expression", + )), + } + } + + fn expr(&mut self, expr: &mut Expression) -> Result<()> { + match expr { + Expression::Var(v) => { + if let Some(value) = self.value(v)? { + *expr = self.constant(v, value)?; + } + } + Expression::Literal(_) | Expression::Iri(_) => {} + Expression::Binary { left, right, .. } => { + self.expr(left)?; + self.expr(right)?; + } + Expression::Unary { operand, .. } => self.expr(operand)?, + Expression::FunctionCall { args, .. } | Expression::Coalesce { args, .. } => { + for arg in args { + self.expr(arg)?; + } + } + Expression::If { + condition, + then_expr, + else_expr, + .. + } => { + self.expr(condition)?; + self.expr(then_expr)?; + self.expr(else_expr)?; + } + Expression::In { expr, list, .. } => { + self.expr(expr)?; + for item in list { + self.expr(item)?; + } + } + Expression::Exists { pattern, .. } | Expression::NotExists { pattern, .. } => { + self.pattern(pattern)?; + } + Expression::Aggregate { expr, .. } => { + if let Some(inner) = expr { + self.expr(inner)?; + } + } + Expression::Bracketed { inner, .. } => self.expr(inner)?, + } + Ok(()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::parse_sparql; + use serde_json::json; + + fn substituted(sparql: &str, params: JsonValue) -> Result { + let mut ast = parse_sparql(sparql).ast.expect("parses"); + let JsonValue::Object(params) = params else { + panic!("params must be an object") + }; + substitute_params(&mut ast, ¶ms)?; + Ok(ast) + } + + fn mentions_var(ast: &SparqlAst, name: &str) -> bool { + format!("{ast:?}").contains(&format!("Var {{ name: {name:?}")) + } + + fn select(ast: &SparqlAst) -> &crate::ast::SelectQuery { + match &ast.body { + QueryBody::Select(q) => q, + _ => panic!("not a SELECT"), + } + } + + fn first_triple(pattern: &GraphPattern) -> &TriplePattern { + match pattern { + GraphPattern::Bgp { patterns, .. } => &patterns[0], + GraphPattern::Group { patterns, .. } => first_triple(&patterns[0]), + other => panic!("no triple in {other:?}"), + } + } + + #[test] + fn dollar_and_question_mark_are_the_same_variable() { + for sparql in [ + "SELECT ?s WHERE { ?s $name }", + "SELECT ?s WHERE { ?s ?name }", + ] { + let ast = substituted(sparql, json!({"name": "Alice"})).unwrap(); + let triple = first_triple(&select(&ast).where_clause.pattern); + assert!(matches!( + &triple.object, + Term::Literal(Literal { value: LiteralValue::Simple(s), .. }) if s.as_ref() == "Alice" + )); + assert!(!mentions_var(&ast, "name")); + } + } + + #[test] + fn no_parameters_leave_the_query_untouched() { + let sparql = "SELECT ?s WHERE { ?s $name }"; + let ast = substituted(sparql, json!({})).unwrap(); + assert_eq!(ast, parse_sparql(sparql).ast.unwrap()); + } + + #[test] + fn values_follow_the_json_ld_forms() { + let ast = substituted( + "SELECT * WHERE { ?s

$a, $b, $c, $d, $e, $f, $g, $h }", + json!({ + "a": 7, + "b": 1.5, + "c": true, + "d": {"@id": "http://example.org/x"}, + "e": {"@id": "_:fdb-1"}, + "f": {"@value": "2024-01-02", "@type": "http://www.w3.org/2001/XMLSchema#date"}, + "g": {"@value": "chat", "@language": "fr"}, + "h": {"@value": 3}, + }), + ) + .unwrap(); + let GraphPattern::Bgp { patterns, .. } = &select(&ast).where_clause.pattern else { + panic!("expected a BGP") + }; + let objects: Vec<&Term> = patterns.iter().map(|t| &t.object).collect(); + assert!(matches!( + objects[0], + Term::Literal(Literal { + value: LiteralValue::Integer(7), + .. + }) + )); + assert!( + matches!(objects[1], Term::Literal(Literal { value: LiteralValue::Double(d), .. }) if *d == 1.5) + ); + assert!(matches!( + objects[2], + Term::Literal(Literal { + value: LiteralValue::Boolean(true), + .. + }) + )); + assert!( + matches!(objects[3], Term::Iri(Iri { value: crate::ast::IriValue::Full(i), .. }) if i.as_ref() == "http://example.org/x") + ); + assert!( + matches!(objects[4], Term::BlankNode(BlankNode { value: crate::ast::BlankNodeValue::Labeled(l), .. }) if l.as_ref() == "fdb-1") + ); + assert!( + matches!(objects[5], Term::Literal(Literal { value: LiteralValue::Typed { value, .. }, .. }) if value.as_ref() == "2024-01-02") + ); + assert!( + matches!(objects[6], Term::Literal(Literal { value: LiteralValue::LangTagged { lang, .. }, .. }) if lang.as_ref() == "fr") + ); + assert!(matches!( + objects[7], + Term::Literal(Literal { + value: LiteralValue::Integer(3), + .. + }) + )); + } + + #[test] + fn values_that_are_not_one_term_are_rejected() { + for (value, reason) in [ + (json!(null), "null"), + (json!([1, 2]), "list"), + (json!({"x": 1}), "without `@id` or `@value`"), + (json!({"@id": "_:"}), "blank node label"), + (json!({"@value": "_:x", "@type": "@id"}), "blank node label"), + (json!({"@value": 1, "@type": "@id"}), "must be a string"), + ( + json!({"@value": "x", "@type": "@vocab"}), + "only `@id` or a datatype IRI", + ), + ( + json!({"@value": [1, "a"], "@type": fluree_vocab::fluree::EMBEDDING_VECTOR}), + "not all numbers", + ), + ( + json!({"@value": [1, 2], "@type": "http://example.org/t"}), + "embeddingVector", + ), + (json!({"@id": "x", "@type": "y"}), "unexpected key"), + ( + json!({"@value": "a", "@type": "t", "@language": "en"}), + "both", + ), + (json!({"@value": 1, "@language": "en"}), "must be strings"), + ] { + let err = + substituted("SELECT ?s WHERE { ?s

$v }", json!({ "v": value })).unwrap_err(); + assert!(err.reason.contains(reason), "{err}"); + } + } + + #[test] + fn only_a_stored_blank_node_is_a_parameter() { + // Any other label lowers to a variable: under DELETE WHERE it would + // delete every triple instead of one node's. + for sparql in ["DELETE WHERE { $s ?p ?o }", "SELECT ?o WHERE { $s ?p ?o }"] { + for value in [ + json!({"@id": "_:x"}), + json!({"@value": "_:x", "@type": "@id"}), + ] { + let err = substituted(sparql, json!({ "s": value })).unwrap_err(); + assert!(err.reason.contains("blank node label"), "{sparql}: {err}"); + } + } + } + + #[test] + fn an_id_typed_value_is_a_node() { + let ast = substituted( + "SELECT * WHERE { ?s

$a, $b }", + json!({ + "a": {"@value": "http://example.org/x", "@type": "@id"}, + "b": {"@value": "_:fdb-1", "@type": "@id"}, + }), + ) + .unwrap(); + let GraphPattern::Bgp { patterns, .. } = &select(&ast).where_clause.pattern else { + panic!("expected a BGP") + }; + assert!( + matches!(&patterns[0].object, Term::Iri(Iri { value: IriValue::Full(i), .. }) if i.as_ref() == "http://example.org/x") + ); + assert!( + matches!(&patterns[1].object, Term::BlankNode(BlankNode { value: crate::ast::BlankNodeValue::Labeled(l), .. }) if l.as_ref() == "fdb-1") + ); + } + + #[test] + fn a_vector_takes_its_numbers_as_a_list() { + let ast = substituted( + "SELECT ?s WHERE { ?s

$v }", + json!({"v": {"@value": [0.5, 1, -2.25], "@type": fluree_vocab::fluree::EMBEDDING_VECTOR}}), + ) + .unwrap(); + let triple = first_triple(&select(&ast).where_clause.pattern); + assert!(matches!( + &triple.object, + Term::Literal(Literal { value: LiteralValue::Typed { value, datatype }, .. }) + if value.as_ref() == "[0.5,1,-2.25]" + && matches!(&datatype.value, IriValue::Full(d) if d.as_ref() == fluree_vocab::fluree::EMBEDDING_VECTOR) + )); + } + + #[test] + fn a_name_given_twice_is_an_error() { + let err = substituted( + "SELECT ?s WHERE { ?s $name }", + json!({"name": "Alice", "$name": "Bob"}), + ) + .unwrap_err(); + assert!(err.reason.contains("given twice"), "{err}"); + } + + #[test] + fn a_parameter_not_in_the_query_is_an_error() { + let err = substituted( + "SELECT ?s WHERE { ?s $name }", + json!({"nmae": "Alice"}), + ) + .unwrap_err(); + assert_eq!( + err.to_string(), + "parameter `$nmae` is not a variable in the query" + ); + } + + #[test] + fn positions_constrain_the_term() { + let ast = substituted( + "SELECT ?o WHERE { GRAPH $g { $s $p ?o } }", + json!({"s": {"@id": "_:fdb-b"}, "p": {"@id": "http://example.org/p"}, "g": {"@id": "http://example.org/g"}}), + ) + .unwrap(); + assert!(!mentions_var(&ast, "s") && !mentions_var(&ast, "p") && !mentions_var(&ast, "g")); + + for (sparql, reason) in [ + ("SELECT * WHERE { $v

?o }", "is a subject"), + ("SELECT * WHERE { ?s $v ?o }", "is a predicate"), + ("SELECT * WHERE { GRAPH $v { ?s ?p ?o } }", "names a graph"), + ("SELECT * WHERE { SERVICE $v { ?s ?p ?o } }", "SERVICE"), + ] { + let err = substituted(sparql, json!({"v": "literal"})).unwrap_err(); + assert!(err.reason.contains(reason), "{sparql}: {err}"); + } + let err = substituted( + "SELECT * WHERE { ?s

?o FILTER(?o = $v) }", + json!({"v": {"@id": "_:fdb-b"}}), + ) + .unwrap_err(); + assert!(err.reason.contains("blank node"), "{err}"); + } + + #[test] + fn only_a_local_service_body_takes_parameters() { + let ast = substituted( + "SELECT ?s WHERE { SERVICE { ?s

$v } }", + json!({"v": 1}), + ) + .unwrap(); + assert!(!mentions_var(&ast, "v")); + for sparql in [ + "SELECT ?s WHERE { SERVICE { ?s

$v } }", + "SELECT ?s WHERE { SERVICE ?e { ?s

$v } }", + ] { + let err = substituted(sparql, json!({"v": 1})).unwrap_err(); + assert!(err.reason.contains("remote SERVICE"), "{sparql}: {err}"); + } + let ast = substituted( + "SELECT ?s WHERE { SERVICE $e { ?s

?o } }", + json!({"e": {"@id": "http://example.org/sparql"}}), + ) + .unwrap(); + assert!(!mentions_var(&ast, "e")); + } + + #[test] + fn nested_scopes_see_the_value() { + let ast = substituted( + "SELECT ?s WHERE { + ?s

?o + OPTIONAL { ?s ?a FILTER(?a > $min) } + FILTER EXISTS { ?s $min } + { SELECT ?s WHERE { ?s $min } } + MINUS { ?s $min } + }", + json!({"min": 21}), + ) + .unwrap(); + assert!(!mentions_var(&ast, "min")); + } + + #[test] + fn projected_parameters_stay_columns() { + let ast = substituted( + "SELECT ?name ?s WHERE { ?s ?name } ORDER BY ?name", + json!({"name": "Alice"}), + ) + .unwrap(); + let q = select(&ast); + let SelectVariables::Explicit(items) = &q.select.variables else { + panic!("explicit projection") + }; + assert!(matches!( + &items[0], + SelectVariable::Expr { expr: Expression::Literal(_), alias, .. } if alias.name.as_ref() == "name" + )); + let order = &q.modifiers.order_by.as_ref().unwrap().conditions[0]; + assert!(matches!( + order.expr, + OrderExpr::Expr(Expression::Literal(_)) + )); + } + + #[test] + fn a_grouped_parameter_is_projected_through_its_key() { + let ast = substituted( + "SELECT ?name (COUNT(?s) AS ?n) WHERE { ?s ?name } GROUP BY ?name", + json!({"name": "Alice"}), + ) + .unwrap(); + let q = select(&ast); + let SelectVariables::Explicit(items) = &q.select.variables else { + panic!("explicit projection") + }; + assert!(matches!(&items[0], SelectVariable::Var(v) if v.name.as_ref() == "name")); + assert!(matches!( + &q.modifiers.group_by.as_ref().unwrap().conditions[0], + GroupCondition::Expr { expr: Expression::Literal(_), alias: Some(v), .. } if v.name.as_ref() == "name" + )); + } + + #[test] + fn a_parameter_the_query_assigns_is_an_error() { + for sparql in [ + "SELECT ?x WHERE { ?s

?o BIND(?o AS ?x) }", + "SELECT ?x WHERE { VALUES ?x { 1 2 } ?s

?x }", + "SELECT (?o AS ?x) WHERE { ?s

?o }", + "SELECT ?x WHERE { ?s

?o } GROUP BY (?o AS ?x)", + "SELECT ?s WHERE { ?s

?o } VALUES ?x { 1 }", + ] { + let err = substituted(sparql, json!({"x": 1})).unwrap_err(); + assert!(err.reason.contains("assigned"), "{sparql}: {err}"); + } + } + + #[test] + fn update_templates_and_where_are_substituted() { + let ast = substituted( + "DELETE { ?s ?old } INSERT { ?s $new } WHERE { ?s $id ; ?old }", + json!({"id": 7, "new": "done"}), + ) + .unwrap(); + assert!(!mentions_var(&ast, "id") && !mentions_var(&ast, "new")); + assert!(mentions_var(&ast, "old")); + + let ast = substituted( + "DELETE WHERE { $s ?p ?o }", + json!({"s": {"@id": "http://example.org/x"}}), + ) + .unwrap(); + assert!(!mentions_var(&ast, "s")); + } + + #[test] + fn construct_and_describe_are_substituted() { + let ast = substituted( + "CONSTRUCT { $s true } WHERE { $s ?p ?o }", + json!({"s": {"@id": "http://example.org/x"}}), + ) + .unwrap(); + assert!(!mentions_var(&ast, "s")); + let ast = + substituted("DESCRIBE $s", json!({"s": {"@id": "http://example.org/x"}})).unwrap(); + assert!(!mentions_var(&ast, "s")); + } +} diff --git a/fluree-db-sparql/src/parse/expr.rs b/fluree-db-sparql/src/parse/expr.rs index 00fc58bbae..08c839ee4b 100644 --- a/fluree-db-sparql/src/parse/expr.rs +++ b/fluree-db-sparql/src/parse/expr.rs @@ -750,6 +750,9 @@ fn check_builtin_function_keyword(tokens: &TokenStream) -> Option if tokens.check_keyword(TokenKind::KwEuclideanDistance) { return Some(FunctionName::EuclideanDistance); } + if tokens.check_keyword(TokenKind::KwFulltext) { + return Some(FunctionName::Fulltext); + } None } @@ -1288,4 +1291,24 @@ mod tests { _ => panic!("Expected function call"), } } + + #[test] + fn fulltext_is_a_function_but_not_a_reserved_word() { + for input in ["fulltext(?text, \"rust\")", "FULL_TEXT(?text, \"rust\")"] { + match parse_expr_str(input).unwrap() { + Expression::FunctionCall { name, args, .. } => { + assert_eq!(name, FunctionName::Fulltext); + assert_eq!(args.len(), 2); + } + other => panic!("expected a function call, got {other:?}"), + } + } + assert!(matches!( + parse_expr_str("?fulltext").unwrap(), + Expression::Var(v) if v.name.as_ref() == "fulltext" + )); + let query = "PREFIX fulltext: \ + SELECT ?s WHERE { ?s fulltext:title ?fulltext }"; + assert!(!crate::parse_sparql(query).has_errors()); + } } diff --git a/fluree-db-sparql/src/validate/mod.rs b/fluree-db-sparql/src/validate/mod.rs index fd332a3652..80b6c23325 100644 --- a/fluree-db-sparql/src/validate/mod.rs +++ b/fluree-db-sparql/src/validate/mod.rs @@ -52,7 +52,7 @@ use crate::span::SourceSpan; /// `fluree_db_core::ns_encoding::STABLE_BLANK_NODE_LABEL_PREFIX` (that crate /// is a feature-gated dependency, so the value is duplicated here; a /// lowering-feature test asserts equality). -const STABLE_BLANK_NODE_LABEL_PREFIX: &str = "fdb-"; +pub(crate) const STABLE_BLANK_NODE_LABEL_PREFIX: &str = "fdb-"; /// Fluree capability configuration. /// diff --git a/fluree-db-transact/src/error.rs b/fluree-db-transact/src/error.rs index 12d32e3371..73df07dd1c 100644 --- a/fluree-db-transact/src/error.rs +++ b/fluree-db-transact/src/error.rs @@ -1,5 +1,6 @@ //! Transaction error types +use serde_json::Value as JsonValue; use thiserror::Error; /// Transaction errors @@ -156,7 +157,7 @@ pub enum TransactError { /// SHACL validation violation (only available with `shacl` feature) #[cfg(feature = "shacl")] #[error("{0}")] - ShaclViolation(String), + ShaclViolation(ShaclViolations), /// Transaction exceeded the configured max-fuel limit #[error("{0}")] @@ -269,5 +270,84 @@ pub enum TransactError { }, } +/// One validation result with all identifiers resolved to IRIs. +#[derive(Debug, Clone, serde::Serialize)] +pub struct ReportResult { + /// The node that failed validation (`sh:focusNode`): a JSON string for + /// IRIs / blank-node labels, or a JSON-LD value object (or native + /// scalar) for literal `sh:targetNode` targets. + pub focus_node: JsonValue, + /// The property path, when it is a single predicate (`sh:resultPath`). + /// Complex paths are omitted rather than misrepresented. + #[serde(skip_serializing_if = "Option::is_none")] + pub result_path: Option, + /// The node shape that produced this result. + pub source_shape: String, + /// The property shape that produced this result, when applicable. + #[serde(skip_serializing_if = "Option::is_none")] + pub source_constraint: Option, + /// The constraint component IRI (`sh:sourceConstraintComponent`). + pub constraint_component: String, + /// Severity IRI: `sh:Violation`, `sh:Warning`, or `sh:Info`. + pub severity: String, + /// Human-readable message (`sh:resultMessage`). + pub message: String, + /// The offending value, when applicable (`sh:value`). + #[serde(skip_serializing_if = "Option::is_none")] + pub value: Option, +} + +/// The violations that rejected a write: the readable report the error +/// displays, and each violation resolved to IRIs, as +/// `fluree_db_api::validate` reports them. Reads as the report text, so it +/// can be matched and searched as a string. +#[derive(Debug, Clone)] +pub struct ShaclViolations { + message: String, + results: Vec, +} + +impl ShaclViolations { + pub fn new(message: String, results: Vec) -> Self { + Self { message, results } + } + + pub fn message(&self) -> &str { + &self.message + } + + /// The violations, resolved; empty where only the text was kept. + pub fn results(&self) -> &[ReportResult] { + &self.results + } +} + +/// Violations known only by their text. +impl From for ShaclViolations { + fn from(message: String) -> Self { + Self::new(message, Vec::new()) + } +} + +impl From<&str> for ShaclViolations { + fn from(message: &str) -> Self { + Self::from(message.to_string()) + } +} + +impl std::fmt::Display for ShaclViolations { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.message) + } +} + +impl std::ops::Deref for ShaclViolations { + type Target = str; + + fn deref(&self) -> &str { + &self.message + } +} + /// Result type for transaction operations pub type Result = std::result::Result; diff --git a/fluree-db-transact/src/lib.rs b/fluree-db-transact/src/lib.rs index 9dfc3a492e..d982673129 100644 --- a/fluree-db-transact/src/lib.rs +++ b/fluree-db-transact/src/lib.rs @@ -62,7 +62,7 @@ pub use commit::{ build_commit, commit, CommitOpts, CommitOptsRequest, CommitReceipt, StagedCommit, }; pub use commit_flakes::generate_commit_flakes; -pub use error::{Result, TransactError}; +pub use error::{ReportResult, Result, ShaclViolations, TransactError}; pub use flake_sink::FlakeSink; pub use generate::{apply_cancellation, FlakeGenerator}; pub use ir::{