Build the vector backend decision harness

[TheGreenCedar]

Context

Issue #1202 needs a predeclared, reproducible comparison between sqlite-vec and USearch before CodeStory can replace its current embedded vector scan. This draft adds the comparison and evidence machinery without selecting or adopting either backend.

Current remote head: e718ac01832199a91e9a9b910b37a78dee28a1d5, based on dev/codestory-next at 6de1c2717b68e6f97996ae21f38256a2645d7a5a.

On 2026-07-16 the owner changed the blocking decision profile from Windows x64 to macOS arm64 on the local Apple Silicon host. The live issue and Project reflect that decision. A replacement branch head aligning the criteria, harness, workflow, and documentation is in progress; the current remote head still contains the prior Windows-specific contract.

Closes #1202
Refs #1179

What changed on the current remote head

• Pin sqlite-vec 0.1.9 and USearch 2.26.0 for the benchmark-only lane.
• Add machine-readable adoption criteria and operating documentation.
• Add an evidence-bound comparison harness using identical production-attested inputs, representative query identities, cosine semantics, and an exact oracle.
• Model both candidates as immutable generations behind one atomic current/rollback pointer pair.
• Validate exact index and directory bytes on every reader open; cover corrupt candidates, rollback, pinned readers, referenced-generation tampering, and failed publication.
• Require complete source attestation, canonical database validation, and one pinned SQLite read transaction before sampling.
• Add an opt-in fixture freezer that uses retrieval-owned attestation, ordered records, and the exact compatible production query engine.
• Publish frozen fixtures through an opaque native directory-identity token captured before open and used with create-new publication.

The replacement head will change only the blocking profile and evidence infrastructure. It will not adopt a backend.

How to review

1. Start with benchmarks/vector-backend-spike/criteria.json; it is the decision contract.
2. Review retrieval-owned source attestation and pinned sampling in crates/codestory-retrieval/src/embedded_vector.rs.
3. Review native identity capture, handle comparison, and no-replace publication in crates/codestory-workspace/src/owned_publication.rs.
4. Review the isolated candidate runner, aggregate gate evaluation, and fixture-path checks in crates/codestory-bench/tests/vector_backend_spike.rs and its driver.
5. Confirm synthetic smoke output remains diagnostic and decision mode fails closed without complete production fixtures and every required measurement.
6. Review the protected macOS arm64 manual workflow and its retained raw/aggregate artifacts.

Verification

Focused checks on exact remote head e718ac01832199a91e9a9b910b37a78dee28a1d5:

• cargo test --locked -p codestory-workspace owned_publication — 4 passed.
• cargo test --locked -p codestory-retrieval vector_fixture_sampling_is_ordered_and_fails_closed_on_wal_drift — 1 passed.
• cargo test --locked -p codestory-bench --no-default-features --features fixture-generator --test vector_backend_spike — 9 passed, 2 intentionally ignored.
• Native and x86_64-pc-windows-msvc codestory-workspace Clippy with warnings denied — passed.
• cargo check --locked -p codestory-workspace --target x86_64-pc-windows-msvc — passed.
• cargo fmt --all -- --check — passed.
• git diff --check — passed.
• node .github/scripts/check-doc-links.mjs — 73 files, 243 relative links.

Those checks prove the current source contract only. Replacement-head checks and independent exact-head review are required after the Mac evidence lane lands.

Risk and remaining evidence

This PR cannot support a backend recommendation yet and must remain draft. criteria.json remains blocked_pending_required_evidence.

Missing decision evidence includes:

• an independently reviewed 30-query production catalog;
• one real production publication with more than 100,100 anchors for the declared 1k, 10k, 25k, and 100k workloads plus the real incremental tail;
• isolated macOS arm64 artifacts for every declared workload;
• cold/warm timing, peak and warm RSS, cancellation, deep validation, and current embedded-scan comparison;
• macOS offline-build, native-dependency, storage, archive-size, and license review;
• a demonstrated reversible fallback to the existing embedded scan;
• retained raw and aggregate artifacts, explicit gate results, limitations, a clear recommendation, and independent reproduction.

Existing local caches do not contain a complete decision corpus. Synthetic or undersized data remains non-decision evidence. Linux and Windows packaging proof is non-blocking for this non-adopting spike; if a candidate wins, those platforms move to the later adoption PR. Keep the existing embedded scan unless one candidate clears every predeclared macOS arm64 gate.

[TheGreenCedar]

Review context

PR #1207 is the measured decision lane for #1202. It compares sqlite-vec and USearch without adopting either backend; dev/codestory-next remains on the incumbent exact scan until the declared production evidence supports a decision.

Review scope

Independently reviewed exact head 50bbbed74920453ead2df5d9fb7df6bad97df122 against #1202 and the vector-scale contract in #1179.

Checks and findings

• Exact-head linux-contracts is red: vector_fixture_sampling_is_ordered_and_fails_closed_on_wal_drift passes dimension 2 into the canonical 768-dimensional producer contract.
• Fixture publication still has a parent-directory replacement window between validation and the path-based hard_link; publication must be handle-relative or equivalently contained.
• criteria.json remains blocked_pending_required_evidence; no exact-head workflow artifact or Windows x64 production comparison exists.
• Cold/warm timing, RSS, cancellation, deep validation, current-scan baseline, offline/native dependency, archive-size, license, fallback, clean-host reproduction, and the final recommendation remain unproven.
• Replace the ungated full vector scan using measured index evidence #1196 already owns conditional adoption and remains blocked by [Spike] Compare sqlite-vec and USearch under CodeStory’s production retrieval contract #1202. No duplicate implementation issue is needed.

Disposition

• Disposition: keep draft and route the two code fixes on this branch.
• Reason: the harness is useful, but neither candidate has cleared the predeclared decision gate and the exact head is not green.

This review does not select a backend, close #1202, unblock #1196, or prove Windows x64 production behavior. After the code fixes, the next gate is exact-head CI and independent review followed by the full retained Windows evidence set.

[TheGreenCedar]

Independent review checkpoint

Exact head 94cb2cd remains draft. The 768-dimensional WAL fixture fix is correct and draft CI is green, but code acceptance still has two publication-boundary gaps now being repaired:

• the validated output-parent identity is not yet bound to the directory handle opened afterward, leaving a pre-open rename/replacement window;
• the newest Windows hostile test depends on symlink privilege instead of a privilege-free rename/replacement path.

Separately, #1202 remains a non-decision: the reviewed 30-query catalog, production 1k/10k/25k/100k Windows artifacts, isolated timing/RSS/cancellation/deep-validation/package/license evidence, clean-host reproduction, fallback proof, and recommendation are still missing. No backend adoption is authorized.

[TheGreenCedar]

Corrected exact-head checkpoint

Exact head e718ac01832199a91e9a9b910b37a78dee28a1d5 remains draft.

The publication-boundary re-review found no source defect: native parent identity is captured before open, verified against the retained handle, reused for later path checks, and covered by privilege-free pre-open replacement tests on Windows and Unix. Focused workspace and fixture tests, Windows-target checking, git diff --check, and all draft CI checks pass.

#1202 is still a non-decision. The reviewed 30-query catalog, production 1k/10k/25k/100k Windows artifacts, isolated timing/RSS/cancellation/deep-validation/package/license evidence, clean-host reproduction, fallback proof, and recommendation remain missing. This head fixes the harness; it does not authorize backend adoption or close the issue.

[TheGreenCedar]

Closing this draft without merge. Exact head
e718ac01832199a91e9a9b910b37a78dee28a1d5 preserves useful immutable-generation,
attestation, corruption, rollback, and replay ideas, but it encodes the
superseded Windows-first evidence contract. The unpublished macOS replacement
also failed fresh source review, so this head cannot support a backend decision
or be repaired honestly through another incremental patch.

#1202 remains open and parked. A future attempt starts from the then-current
dev/codestory-next only after the experiment contract, corpus and sampling
provenance, vector-free truth catalog, isolated paired runner, and
clean-reproduction bar are accepted. No backend was measured, selected, or
adopted.