Skip to content

fix(ci): preserve browser evidence and pin source freshness - #95

Open
BunsDev wants to merge 4 commits into
mainfrom
fix/e2e-source-freshness-89
Open

BunsDev wants to merge 4 commits into
mainfrom
fix/e2e-source-freshness-89

Conversation

@BunsDev

@BunsDev BunsDev commented Sep 14, 2026 •

Copy link
Copy Markdown
Member

Current scope and disposition — 2026-09-26

Refs #89; closes no issue. Browser/build verification passes; the canonical release check remains blocked by unfinished source-to-page review. Do not merge by bypassing it.

Head: 571e3e7ac32387379c93dd3fef36a0c1dad5769e.
Base main: 49c7a7df8890f80dcb3d2ee82d619a1c60d70770.
Hosted merge tree: 976ce7f397024a5f26ff13113837cc81bfc6fe4f.

Implemented in this branch

  • Independent source-freshness and complete browser/build jobs. The canonical Verify documentation release rollup requires two literal success results. Missing, skipped, cancelled and failed results are not acceptance.
  • Pinned Git-tree freshness comparison, with exact ancestry and file/mode/type identity rather than timestamp-filtered moving-ref history. Malformed, truncated and unresolved-both snapshots fail closed. No comparison-file-list truncation or ancestor obstruction can establish missing leaf identity.
  • Custom-adapter documentation for the reviewed interpreter/prompt-prefix refusal contract and safe migration, with page-specific checks. Source: Coven #1089; the earlier retained evidence identifies 8a0c7f445cc3c8ad499d8970a06ffd204ea9f021 and harness blob 63eace3993959ba6b8350d391af29163498cf923.
  • Current dependencies, install documentation and pnpm action v6 preserved from main.
  • Latest three-file slice: correct the API example from reserved project_root_violation to emitted invalid_request; document negotiated policy-refusal handling without unrestricted fallback, availability versus authorization, and Rust peer-bound renegotiation without automatic mutation replay. Ten page regressions are wired through check:automation and therefore full pnpm verify.

Latest API facts were reviewed against immutable Coven 801e9f219b1d50336ad4111ca92ce2a3a1eafa3c, docs/API-CONTRACT.md blob ccdc69acb5662959ffb88d02c721a795de1fa660 (reusable client, structured errors and stable error codes). The source contract, not the new documentation tests, establishes runtime behavior.

Verification actually performed

Local latest-slice proof

The old API page was reconstructed with exact blob 243f1a355b547de0f33d5b5c4972e44750e3379c. Its three positive page checks failed before the repair. The full initial ten-case run had three passes and seven failures; four failures were mutation setup against the uncorrected page, not seven separate runtime bugs. After correction, all ten cases passed, including seven negative mutations.

node --test scripts/check-source-drift.test.mjs scripts/docs-release-gate.test.mjs scripts/api-reference-contract.test.mjs: 98 passed, zero failed/skipped/cancelled on Node 22.16.0. Syntax and whitespace checks passed. All three GitHub blob identities match the locally tested bytes. The local workspace is a sparse inspection copy, not a full dependency-installed checkout; no local full-site or native-runtime acceptance is claimed.

Exact-head hosted proof

Run 36249152337 completed:

  • Browser/build job 108423806443: success, including frozen dependency installation, Chrome installation, complete pnpm verify, clean-generated-tree check and evidence upload.
  • Freshness job 108423806536: failure after successful regression and source-lock validation steps; six real watched paths differ.
  • Canonical release job 108424131698: failure, correctly refusing release despite browser success.

Downloaded and independently verified both ZIP SHA-256 values:

Artifact ID SHA-256
docs-certification-36249152337 10907973447 e8501ef4e513d5d3889966091f9565e46dc926931aad58fcbd8358ddb279ce8d
docs-source-drift-36249152337 10908293813 c095adb53f2965e7d99a1747209e718184a1bd0b7aa6e6b3fc47617d9b14e621

The browser report binds merge tree 976ce7f... and returns ok:true: 12 HTTP-200 routes, including the changed /docs/reference/api page with its canonical URL and one H1/main, four journey categories, three 390px no-overflow checks and five retained screenshots. This is CI-localhost site evidence, not a production deployment.

Remaining #89 work

The authenticated freshness report binds reviewed upstream 9c7615a75c0a84dc9ef6f323bc542f30633189f0 to target 801e9f219b1d50336ad4111ca92ce2a3a1eafa3c. Differences remain in CLI main.rs, api.rs, harness.rs, setup/mod.rs, setup/process.rs, and docs/API-CONTRACT.md.

The custom-adapter and API-page corrections are bounded reviewed slices, not completion of the entire six-file/source-section review. docs/source-lock.json, verifiedCommit, and verifiedAt remain unchanged. Finish the remaining source-to-public-page and any generated OpenAPI reconciliation, then truthfully advance the lock and require fresh successful canonical verification.

No runtime authority, dependency, publishing control or workflow was changed by the latest three-file commit. No release, deployment or human acceptance is inferred. Earlier raw red/green and hosted receipts remain in the PR comments and commit history, including run 35671468692 and comment 5769508439; they are historical evidence, not substitutes for the new head above.

Run documentation browser certification independently of source freshness,
while preserving the canonical fail-closed release rollup. Compare watched
paths in pinned Git trees rather than timestamp-filtered moving refs.

Add 83 dependency-free CLI and workflow regressions. Refs #89; this does
not advance the source lock or close the outstanding public-source review.
Copilot AI lite review requested due to automatic review settings September 14, 2026 06:36
@vercel

vercel Bot commented Sep 14, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
coven-docs Ready Ready Preview Sep 26, 2026 2:38pm UTC

Request Review

BunsDev commented Sep 14, 2026

Copy link
Copy Markdown
Member Author

Exact-head hosted verification receipt

Run 34814168620 exercised head 44f10e33604c2abfb659baca965006a0c2e545f7 through merge tree bbb74861642c13c562dd9e5a198a3f823860efc6 against base e0860cd68030248fb829b00bc32d684fd1720ff2.

  • Browser/build job 103881261316: completed/success. Frozen install, Chrome installation, complete pnpm verify, clean-generated-tree check and evidence upload all succeeded.
  • 83 automation regressions: passed in hosted CI, zero failures/skips/cancellations, as well as locally.
  • Freshness job 103881261446: completed/failure for four actual watched-path differences, not a malformed API response or detector crash.
  • Canonical Verify documentation release job 103881552640: completed/failure, correctly rejecting the combination of browser success and freshness failure. No bypass or merge performed.

Downloaded and independently checked both artifact ZIP SHA-256 digests:

  1. docs-certification-34814168620, artifact 10336057214, digest fdbb539361d9996e379fda7446d7132dda21b022ba4727ca477a2717d1d3ced9. Its report binds the merge tree above, returns ok:true, and records 12 HTTP-200 routes with canonical/H1/main checks, four journey categories, three 390px mobile overflow checks, and five screenshots. This is the CI-built localhost site, not production deployment evidence.
  2. docs-source-drift-34814168620, artifact 10336191598, digest d54d84e78e30b8b34e2e50768a4f1323d05b86da516432431a19b42b9b530642. It records verificationMode: git-tree-identity, exact upstream target a566c26d2bf725a0dee6b4c290fe8718ab52375b, and differences in crates/coven-cli/src/main.rs, crates/coven-cli/src/api.rs, crates/coven-cli/src/setup/process.rs, and docs/API-CONTRACT.md; all eight other watched identities match the reviewed snapshot.

This supersedes the initial PR body's pending hosted-verification status, not its source-review limit. The topology repair now has real browser execution evidence and real fail-closed rollup evidence. #89 still needs substantive review of those four source paths against affected public pages, followed by a truthful lock update and fresh successful canonical CI. Do not merge this PR by ignoring that remaining gate.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Full frozen-install, browser, and live drift gates were not run in this sandbox.

Pull request overview

This PR repairs documentation CI evidence and source-freshness detection while preserving a strict release gate.

Changes:

  • Separates freshness and browser verification jobs.
  • Pins drift checks to immutable Git tree identities.
  • Adds regression coverage and documents evidence semantics.
File summaries
File Summary
scripts/docs-release-gate.test.mjs Tests release-gate outcomes and workflow wiring.
scripts/docs-release-gate.mjs Requires both verification jobs to succeed.
scripts/check-source-drift.test.mjs Adds source-drift regression coverage.
scripts/check-source-drift.mjs Implements pinned, fail-closed tree comparison.
scripts/check-automation-syntax.mjs Runs automation regression tests.
docs/e2e-source-freshness.md Documents evidence and freshness semantics.
.github/workflows/docs.yml Adds independent verification jobs and canonical rollup.
Review details
  • Files reviewed: 7/7 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Integrate main 49c7a7d into #95 without rewriting history. Preserve current
dependencies, installation docs and pnpm action v6. Document the current
interpreter/prompt-prefix refusal contract, safe migration and build boundary.
Require those claims in the existing harness-doc check.

The scoped checker reproduced the missing-contract failure and passed after
the page repair, including 14 rejecting mutations. All 83 existing detector/
rollup regressions passed locally. Full hosted verification remains required.

Refs #89. Source-lock timestamps and revisions remain unchanged: this is the
bounded harness slice, not complete upstream contract or deployed acceptance.
An ancestor obstruction identifies the ancestor, not the watched leaf.
Reject watches unresolved in both snapshots instead of treating equal
obstructions as unchanged content. Preserve valid removal/replacement drift.

Five new subprocess regressions failed before this fix; all 88 detector
and release-rollup tests pass locally on Node 22.16.0. No source-lock
revision/timestamp, workflow authority, or publication gate is changed.
Fresh exact-head hosted verification remains required. Refs #89, #95.

Signed-off-by: Val Alexander <val@opencoven.ai>

BunsDev commented Sep 22, 2026

Copy link
Copy Markdown
Member Author

Exact-head hosted receipt for the final detector follow-up

Run 35671468692 completed for head 7279eb316adfc6a5f3df84626ddee6a50990c1b3.

  • Regression job step: success. The local red/green receipt remains five newly failing cases before the fix, then 88 total passing tests afterward.
  • Browser/build job 106568661356: success, including frozen installation, Chrome installation, full pnpm verify, clean-generated-tree verification and artifact upload.
  • Freshness job 106568661017: failure after successful source-lock validation. It reports six genuine watched-path changes, not a detector crash or API-data error.
  • Canonical release job 106568964590: failure, correctly refusing release despite the independent browser success.

Downloaded source artifact 10671251226 (docs-source-drift-35671468692) and independently verified ZIP SHA-256 6983957cf977d7d9731c3c4ae170b2f4dfad1476b391d34ad6adcb2647a08b43. The report compares reviewed upstream 9c7615a75c0a84dc9ef6f323bc542f30633189f0 with exact target 5c614c6b79faaf32e5828058d0ce4272cd7fb60e. The remaining changed inputs are CLI main.rs, api.rs, harness.rs, setup/mod.rs, setup/process.rs, and docs/API-CONTRACT.md.

Browser artifact 10672005218 was uploaded successfully; this continuation has not independently downloaded or authenticated its ZIP, and does not claim production deployment evidence.

This supersedes the PR body's pending-new-head hosted status. It does not complete #89's remaining source-to-page review, authorize a source-lock advance, or make the whole workflow green. No merge, publication, source-lock timestamp/revision change, or gate bypass occurred. Keep the current browser/detector repair evidence; finish the substantive contract review before seeking a green canonical release check.

Correct the project-root rejection example to invalid_request; the proposed
project_root_violation code remains reserved. Document negotiated 409 policy
refusals without unrestricted fallback, capability availability without grants,
and the Rust client's peer-bound renegotiation/no-automatic-mutation-replay.

Source reviewed: Coven 801e9f219b1d50336ad4111ca92ce2a3a1eafa3c,
docs/API-CONTRACT.md blob ccdc69acb5662959ffb88d02c721a795de1fa660.

Add ten documentation regressions and wire them into check:automation.
The original page fails the three positive contract checks; corrected page
passes all ten, including seven negative mutations. Combined local detector,
rollup and page suite: 98 pass, zero failures/skips. Hosted site verification
remains required. These are docs checks, not new runtime E2E acceptance.

Refs #89, #95. Source-lock boundaries, dependencies, workflows and publishing
controls remain unchanged; this bounded review does not complete all #89 work.

Signed-off-by: Val Alexander <val@opencoven.ai>

This branch was successfully deployed

1 active deployment
Preview — 571e3e7a Deployed Sep 26, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants