Align the SQL on FHIR data operations with the upstream specification - #2710
Merged
Conversation
Introduce the polymorphic subject resolver that both of the collapsed SQL on FHIR operations act through, admitting a ViewDefinition, a SQLQuery Library or a SQLView Library named by canonical URL, literal reference or inline resource. Add a snapshot data source that pins each resource table's Delta version at one instant, so an export computes every subject against a single consistent view of the data regardless of concurrent writes.
Extend dependency resolution so that supplied artefacts are matched at every level of the graph, admit SQLView Libraries as well as ViewDefinitions, honour version pins, and record which entries were consulted so an entry matching nothing can be reported. Allow a caller to share one memoisation map across several queries, which an export job needs to resolve a shared dependency once. Add the per-subject-kind output format rules for the synchronous operation, where parquet and fhir come from the SQL engine and so are offered only for SQL subjects.
Introduces the system-level $sql-run operation, which executes a single subject - a ViewDefinition, a SQLQuery or a SQLView - and streams the result in the negotiated format. The subject's kind decides which parameters apply, which formats are available and which evaluation engine runs it.
Introduces the system-level $sql-export operation, which exports any mixture of subjects in one job. Every problem in a kick-off body is reported in a single outcome, so a job that starts is never rejected later at its status URL, and every subject reads through one snapshot of the data captured when execution begins.
Collapses the UI's four operation clients, four hooks and two job types onto the two new operations. A run and an export are now one client and one hook each, because the operations themselves no longer distinguish a view from a query. Fixes the CSV header flag, which was sent as _header and so was silently ignored.
Adds a basket on the SQL on FHIR page that captures views and queries - inline or stored, with their bindings - and exports them as one job with one named output each. An entry is frozen when it is added, so later edits to the form cannot change what a queued subject exports, and a name collision is caught before a job is started rather than after.
Deletes the old operation surface outright: the providers, the five configuration flags, the four authorities, the two job types and the documentation pages. Discovery, configuration, authorisation and the examples now describe exactly two operations.
The export set's "Add to export set" action made the substring match for the result card's Export button ambiguous.
…own error HAPI refuses to bind a non-primitive from a query string before the provider is reached, so subjectResource, parameters and context over GET were answered with its 405 rather than the operation's 400 naming the parameter. They are now read from the raw body, as the export operation already does.
A repeated name inside a Parameters part list was rendered as a single value in the plain-JSON manifest, so an output spanning several files named only the last of them and the rest were unreachable. The same repeated-name rule now applies at every level. A binding failure on $sql-run reached the client as a bare message; it is now labelled against the parameters part, as $sql-export already did.
A unit test over mocked request details passed while the framework's own binding answered the real request first, so the rejection is now asserted at the layer where it has to happen.
The run path had wire-level coverage for a subject whose table source the server does not hold; the export path did not.
johngrimes
added
breaking change
|
SonarCloud could not connect the supplied-count guard to the nullability of the individual naming forms (java:S2637).
# Conflicts: # site/docs/server/authorization.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #2663 (an inline query could not use a view definition supplied in the same request), as part of aligning the SQL on FHIR data operations with the upstream specification.
$viewdefinition-run,ViewDefinition/[id]/$run,$viewdefinition-export,$sqlquery-runand$sqlquery-exportare replaced by two system-level operations,$sql-runand$sql-export. Each acts on a polymorphic subject that may be a ViewDefinition, a SQLQuery Library or a SQLView Library, named by canonical URL, by relative reference, or supplied inline. Which kind the subject resolves to decides which parameters apply and which output formats are available; the endpoint, the filters and the error contract are the same for all three. Errors carry anOperationOutcomewhoseissue.codeandexpressionname the parameter at fault.Both operations accept a repeating
contextparameter carrying inline supporting artefacts, matched to a dependency by canonical URL. That is what resolves #2663: a query can now name a table source the server does not hold and supply it with the request. An entry matching no dependency is rejected rather than ignored, since that usually means a mistyped URL.$sql-exporttakes many subjects in one job and produces one named output each. Every subject is computed against a single snapshot of the data, pinned when the job begins, so a write landing mid-job cannot leave two outputs disagreeing with one another. Every subject is validated at kick-off and every problem in a request is reported in one outcome, so a job that starts is never rejected later at its status URL.This is a breaking change for anyone on server v2.0.1.
$viewdefinition-run,ViewDefinition/[id]/$run,$viewdefinition-exportand$sqlquery-runall shipped there, along with thepathling:view-run,pathling:view-exportandpathling:sqlquery-runauthorities and the configuration flags that gated them. They are replaced by$sql-runand$sql-export, thepathling:sql-runandpathling:sql-exportauthorities, and thesqlRunEnabledandsqlExportEnabledflags.$sqlquery-exportis also removed, but never appeared in a release.Two defects were found and fixed on the way. The plain-JSON manifest rendered a repeated name as an array only at the top level, so an export output spanning several files named only the last of them and the rest were unreachable; the FHIR Parameters form was unaffected. Separately, the admin UI sent the CSV header flag as
_header, so it was silently ignored.The admin UI moves onto the new operations and gains an export set: a basket that captures views and queries, inline or stored and with their bindings, and exports them as one job with one named output each.
Also raised while validating this, but not addressed here: #2709, a pre-existing limitation where a filtered read cannot see a resource type whose table was created after the server started.