Skip to content

Expand versioned and cross-provider DML capabilities - #244

Merged
tse-wei-chen merged 27 commits into
mainfrom
feature/dml-capability-expansion
Sep 2, 2026
Merged

tse-wei-chen merged 27 commits into
mainfrom
feature/dml-capability-expansion

Conversation

@tse-wei-chen

@tse-wei-chen tse-wei-chen commented Sep 1, 2026 •

Copy link
Copy Markdown
Owner

Summary

#244 expands the closed F# DML compiler across all six database families while keeping provider/version, row-image, and cross-provider semantics explicit.

The PR now covers six capability-expansion areas:

  1. versioned joined UPDATE
  2. joined DELETE source/target translation
  3. rich result-producing DML
  4. SQL Server OUTPUT with metadata-backed trigger assurance
  5. DML target aliases
  6. existing conflict/upsert contracts and their fail-closed boundaries

The common INSERT / UPDATE / DELETE floor is unchanged; every added capability is represented structurally and admitted only after source and target proofs.

Capability expansion

SQLite 3.33+ UPDATE ... FROM

  • source and target profiles must explicitly prove SQLite ServerVersion 3.33+
  • rendered natively from the structured UPDATE/FROM AST
  • cross-provider joined UPDATE remains fail-closed when SQLite participates because duplicate-match target-row selection is provider-bound

Oracle 26+ direct-join UPDATE ... FROM

  • source and target profiles must explicitly prove Oracle ServerVersion 26.0+
  • rendered natively for Oracle 26+
  • same-provider native-only
  • cross-provider lowering remains fail-closed because Oracle raises ORA-30926 when a target row is matched more than once

Joined DELETE

The closed Delete.Using source-list model represents the proven joined-delete intersection:

  • PostgreSQL source: DELETE ... USING
  • SQL Server source: native DELETE ... FROM
  • Oracle 26+ source: DELETE ... FROM or DELETE ... USING
  • PostgreSQL target: native USING
  • SQL Server target: joined DELETE by restating the target in the Transact-SQL FROM list
  • Oracle 26+ target: native direct-join FROM

Oracle documents duplicate source matches as deleting each target row once, so Oracle 26 joined DELETE participates in the PostgreSQL / SQL Server target-row intersection.

Joined DML approval semantics

The production DML approval path now preserves the same joined-mutation structure instead of compiling a richer mutation than it previews.

  • physical target resolution preserves UPDATE/DELETE target aliases
  • full FromSources / UsingSources survive the compatibility-AST target replacement
  • approval match queries reproduce auxiliary DML sources as the same comma/cross-join row scope
  • target primary-key projections are DISTINCT so duplicate auxiliary matches do not inflate the affected target-row set
  • joined UPDATE/DELETE with count-only identity remains fail-closed because duplicate source matches cannot be collapsed safely without target row identity

Rich RETURNING

  • PostgreSQL: proven binder-resolved local-row scalar/predicate subset
  • SQLite 3.35+: structured scalar RETURNING over the modified target row; UPDATE-FROM auxiliary sources remain outside RETURNING scope
  • Firebird 5.0+: proven structured scalar/predicate RETURNING subset
  • PostgreSQL and Firebird 5 rich RETURNING can cross-lower within the proven expression intersection
  • Firebird OLD/NEW row contexts remain outside the portable Core AST

SQL Server OUTPUT

SQL Server INSERT/UPDATE/DELETE OUTPUT is represented by the same closed result-row AST for the proven target-only column/wildcard subset.

  • INSERT / UPDATE use INSERTED row images
  • DELETE uses DELETED row images
  • INSERTED.* / DELETED.* are grammar-covered and render natively
  • opposite row-image semantics fail closed
  • rich OUTPUT expressions remain outside the current portable subset
  • direct OUTPUT is admitted only for same-provider SQL Server compilation
  • the SQL Server provider exposes an optional DML result-row metadata contract that checks the exact resolved schema/table and DML event through sys.triggers / sys.trigger_events
  • production DmlPlanFactory requires complete trigger metadata visibility, resolves the physical target first, and injects DmlResultRowAssurance.NoEnabledTriggers only after proving no enabled trigger exists for that exact operation
  • SQL Server trigger assurance requires VIEW DEFINITION metadata visibility; inability to prove metadata completeness remains fail-closed
  • enabled triggers, missing metadata support, or mismatched low-level assurance fail closed
  • cross-provider RETURNING/OUTPUT lowering remains fail-closed because SQL Server OUTPUT exposes the pre-trigger row image and the trigger-timing contract is not proven equivalent

Firebird DML target aliases

  • UPDATE and DELETE target aliases are structurally represented and rendered natively
  • Firebird shares the proven alias-hides-original-target contract with PostgreSQL
  • PostgreSQL and Firebird target aliases can cross-lower inside that intersection
  • aliases for providers with different declaration/scope rules remain target-gated

Existing conflict/upsert contracts preserved

  • PostgreSQL explicit-target ON CONFLICT
  • versioned SQLite ON CONFLICT
  • metadata-assured MySQL 8.0.19+ proposed-row alias lowering
  • metadata-assured Firebird UPDATE OR INSERT ... MATCHING
  • omitted INSERT target columns remain same-provider native-only
  • any-unique-key MySQL behavior is not treated as equivalent to a specific conflict target without assurance

Explicit fail-closed boundaries

  • general MERGE remains unsupported
  • SQL Server OUTPUT INTO remains outside the current result-row contract
  • SQL Server rich OUTPUT expressions and opposite-row images remain unsupported
  • Oracle RETURNING INTO host/bind-variable semantics remain outside the Core result-row contract
  • MySQL gains no generic RETURNING equivalent
  • joined UPDATE is not cross-lowered when duplicate-match behavior is unproven
  • DML target aliases do not cross into providers whose declaration/name-resolution rules differ
  • rich RETURNING cannot see auxiliary row sources unless the provider contract explicitly proves that scope
  • provider/version-dependent grammar requires explicit capability profiles; no optimistic default versions are assumed

TDD / grammar strategy

Every capability expansion starts with source/target/profile/assurance boundaries and production code makes only the declared proof green.

Current declared grammar floor:

  • positive query: 5,189
  • positive DML: 434
  • negative query: 196
  • negative DML: 83
  • aggregate declared floor: 5,902

The native DML capability matrix now contains 32 explicit cases across PostgreSQL, SQLite, Firebird, SQL Server, MySQL, and Oracle.

SQL Server OUTPUT adds:

  • 3 native column cases
  • 3 native wildcard cases
  • 15 wrong-source negative grammar cases
  • explicit missing/mismatched trigger-assurance negatives
  • opposite-row-image negatives
  • cross-provider native-only negatives

Negative DML cases preserve stage ownership: syntax errors fail at Parse, version/profile problems fail at SourceValidation, and unsupported or unproven target semantics fail at TargetCapability.

Architecture

source SQL + source provider/version profile
    ↓
dialect grammar + source capability proof
    ↓
closed F# DML AST
    ↓
binder / row-scope validation
    ↓
semantic + target capability + statement assurance proof
    ↓
provider-aware rendering
    ↓
execution / approval contract

No capability is declared equivalent merely because two providers expose similarly named syntax.

Capability matrix version: 2026-09-02.76.

Validation

Current head: cfd4a5f03416da0f0f2f30cf6804b115aa5facfd

Tests workflow run #1411:

  • sqlagent-test: success — 2,104 / 2,104
  • sqlagent-syntax-matrix: success — 6,144 / 6,144
  • sql-syntax-parity: success
  • server-test: success
  • sqlagent-package-smoke: success
  • sqlagent-test-rest: success
  • frontend/auth/admin/infrastructure jobs: success

CodeQL Advanced run #1514:

  • actions: success
  • javascript-typescript: success
  • csharp: success

Runtime convergence coverage additionally proves:

  • SQL Server OUTPUT production compilation obtains no-trigger assurance from provider metadata after physical target resolution
  • enabled SQL Server operation triggers fail closed before mutation compilation
  • joined UPDATE preserves target alias plus derived FROM sources into both mutation and approval match plans
  • joined DELETE with count-only identity fails closed rather than guessing duplicate-match target-row cardinality

@tse-wei-chen tse-wei-chen changed the title Expand versioned native DML capabilities Expand versioned and cross-provider DML capabilities Sep 2, 2026
@tse-wei-chen
tse-wei-chen marked this pull request as ready for review September 2, 2026 05:56
@tse-wei-chen
tse-wei-chen merged commit cde0848 into main Sep 2, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant