Skip to content

2.0: collection contract (#777) - #799

Merged
casoon merged 9 commits into
mainfrom
feat/v2-collection-contract
Oct 26, 2026
Merged

casoon merged 9 commits into
mainfrom
feat/v2-collection-contract

Conversation

@casoon

@casoon casoon commented Oct 26, 2026

Copy link
Copy Markdown
Owner

Package 2, part 1: the shared collection contract (#777) and its use in every module outside performance, security/DNS and C2PA. Those three follow as separate PRs (#769/#770, #774–#776, #789).

Contract

  • audit::collection:
    • Collected<T> with the states measured, empty, partial, failed, unsupported and not_applicable. JS, CDP, HTTP and DNS adapters produce it, never a default value.
    • CollectionLog records each collection for the page.
    • COLLECTORS is the inventory.
  • JSON: module_runs[].subchecks[] carries state, method, method_version, reason_code, scoring and gaps. The schemas now describe moduleRun/subcheckRun, and measurement_type now allows partial and not_measured (performance already emitted them).
  • Score:
    • A module that lost a scoring collection is partial and leaves the overall score, in both the module scores and the per-viewport blend.
    • A module whose every collection was lost is Failed.
    • Derived indicators are Skipped over a failed upstream module and Partial over a partial one.
  • Validation: the contract rejects a run that lost a collection but is marked Completed (collection_gap_completed), and a score built on a lost collection that is still counted (collection_gap_scored).
  • Inventory test every_collector_has_counter_cases:
    • Each COLLECTORS entry needs counter-cases. The JS contract (missing result, invalid JSON, scalar, wrong shape, valid empty) and domain cases are run.
    • In debug builds, recording an unlisted collector panics.

Collectors migrated

  • SEO: all extractors, plus:
    • page-health DOM and HTTP probes;
    • cookie inventory, broken links, service workers, web manifest, www redirect;
    • robots.txt, sitemap noindex, visible schema facts.
  • Other modules: dark mode, mobile, design quality, best practices, tech stack, journey, SSR gap, HTML conformance, help mechanisms, easy language, display modes.
  • Browser WCAG page rules: text spacing, reflow, page title, on-input, orientation.
  • Accessibility journeys: form error, add to cart.

Behaviour fixes this exposes

  • These are no longer reported when the probe did not answer: http1_only, missing_compression, www_not_consolidated, http_not_redirected_to_https, pwa_invalid_manifest, broken links, NoPrintStylesheet/NoForcedColorsSupport, print and focus layout risks without emulation, "no live region".
  • These clean results no longer stand in for a failed collection: "no cookies", "no libraries", "no console errors", "no contrast failures", 100 image-efficiency points, "no stack", "hardened".
  • The console collector is now installed whenever best practices run, so security-only audits read the console too.
  • The SSR content-gap check loads the page itself. Before, after interactive journeys it measured about:blank.
  • Page rules tell an explicit null ("nothing found") from no answer by the RemoteObject subtype.

Validation (local, all green)

  • cargo fmt --check passes, and clippy -D warnings is clean with all features and with no default features.
  • cargo test --all-features passes.
  • The ignored suites CI runs all pass: integration_test 42/42, detection_corpus_test, schema_rules_detection_corpus_test, vulnerable_libs_detection_corpus_test, easy_language_detection_test, help_mechanisms_detection_test, score_calibration_test.
  • reference_sites_test shows no deviation from the reviewed bands.

#777 stays open. Its inventory is complete only once the performance, security/DNS and C2PA collectors and the remaining infrastructure probes (navigation snapshot, frames, consent, interaction) are in.

#777)

Collected<T> carries a collector's state (measured, empty, partial, failed,
unsupported, not applicable) through the analysis; CollectionLog records it
per page and the pipeline folds the records into module_runs[].subchecks[]
with state, method, version and reason. A module that lost a scoring
collection is partial and leaves the overall score; the contract rejects a
report that still counts it.

COLLECTORS lists every collector; each needs counter-cases, run by
every_collector_has_counter_cases. First module: SEO (meta, headings,
social, technical, structured data, image efficiency, page-health DOM and
HTTP probes). Failed probes no longer report http1_only,
missing_compression, www_not_consolidated or
http_not_redirected_to_https.
…777)

Cookie inventory, broken links, service workers, web manifest, www
redirect, robots.txt, sitemap noindex and visible schema facts record
their state. Unanswered link requests are no broken links, a failed
manifest request is no invalid manifest, an unread cookie jar is not
'no cookies', an unreachable robots.txt is unknown rather than missing.
Derived indicators are skipped over a failed upstream module and partial
over a partial one; AI visibility leaves crawler signals out while
robots.txt is unknown.
Print and forced-colors detection, layout under emulation, contrast and
link-indistinct snapshots record their state. A detection without an
answer fails the analysis instead of reading as 'no print stylesheet' or
'no contrast failures'; without media emulation the print and focus
layout risks are not claimed, and the score is marked partial.
… the contract (#777)

Console and library collection record their state; a console collector
that was never installed or a malformed message list is no clean console,
a malformed library list no page without libraries. The console
collector is installed whenever best practices run, so security-only
audits read the console too. A module whose every collection failed is
Failed even with a default payload.
A detection script without an answer fails the module instead of
reporting no stack. A stack counts as audited only when one of its
hardening probes answered; unanswered probes make the collection partial
or failed, and a page without an auditable stack is not applicable.
… collectors on the contract (#777)

The SSR content-gap check loads the page itself with and without
JavaScript instead of measuring whatever the tab shows (about:blank
after interactive journeys), and a missing length is no measurement.
Help mechanisms, easy-language markers and display modes record their
state in a page-level log folded into the accessibility run.
Text spacing, reflow, page title, on-input and orientation report a
technical rule failure when the page gives no answer instead of passing;
an explicit null still means nothing found. The form-error and
add-to-cart journeys fail when a probe gives no answer instead of
reporting a missing live region or dialog.
@casoon
casoon merged commit 12c0b3c into main Oct 26, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant