Skip to content

docs: consistency audit — 41 findings across the standard, get-started, and mirrored root docs #79

Description

@maehr

Systematic consistency audit of every documentation file against the implementation, ahead of the v0.1.0 tag (#68, #4).

Baseline: staging @ 37faf37 — content-identical to what https://textrefs.org currently serves.
Ground truth used: standard/schema/*.ts, scripts/source-schema.ts, scripts/compile.ts, public/contexts/v1.jsonld, api/openapi.yaml, decisions/ADR-*.md, and the compiled output of npm run compile:data against the pinned data/ submodule.

Every item below was verified by reading both sides. Findings that turned out to be correct on inspection are not listed; notable clean results are summarised at the end. Line numbers are against the baseline commit.

Numbering is stable so items can be ticked off individually — this is deliberately one issue rather than 40, but it is meant to be worked in the sections below, and several sections are independent.


A. Normative contradictions in standard/

A1. Spec §13's flagship worked example is invalid against the schema.
specification.md:319-327 — the @graph Work is {id, key, type, preferred_label, status, created, modified}. standard/schema/work.ts:28 declares preferred_citation_system_key: FlatKey (non-optional), the spec's own §6 (specification.md:131) lists it as required, and api/openapi.yaml:203-204 agrees. The document's headline example would fail Work.safeParse at scripts/compile.ts:514. The class diagram at specification.md:67-72 has the same omission — it lists optional creators but not the required key.

A2. "Tombstone" is defined two ways, and json-ld.md states the Work projection wrongly as a result.
specification.md:279 defines a tombstone as status deprecated, withdrawn, or blocked. json-ld.md:35 then says the projection is "derived from their non-tombstoned mapping assertions", which under that definition would exclude deprecated. The compiler excludes only two: scripts/compile.ts:633 const TOMBSTONE_STATUSES = new Set(['withdrawn', 'blocked']), applied at :491. specification.md:140 and versioning.md:109 both state the narrow sense correctly. As written, json-ld.md is wrong for deprecated mappings, which the compiler does project.

A3. The advertised alternate-link media type contradicts the OpenAPI contract.
specification.md:281 (a normative SHOULD): <link rel="alternate" type="application/ld+json" href="…json">. api/openapi.yaml:13: <link rel="alternate" type="application/json" href="…json">. The implementation emits application/ld+json (src/layouts/RecordPage.astro:25), so api/openapi.yaml:13 is the incorrect side.

A4. §14's SPDX validation rule contradicts §9's published-record format.
specification.md:388 (validation requirement 10) requires a validator to check "SPDX syntax of license when present". But specification.md:233 and scripts/compile.ts:289 say published records carry the SPDX IRI (https://spdx.org/licenses/{id}). A validator applying requirement 10 to published records would reject every conforming record. The SPDX check exists only on the authoring side (scripts/source-schema.ts:48-54); standard/schema/canonical-reference.ts:10 is an unconstrained z.string().optional().

A5. One example, two license encodings.
specification.md:352 has "license": "https://spdx.org/licenses/CC-BY-4.0" and :360 has "license": "CC0-1.0" — two entries of the same resolver_targets array in the same JSON-LD document ("@context" at :317), i.e. published output, where per A4 only the IRI form occurs.

A6. §14's BCP 47 requirement is enforced nowhere.
specification.md:388 requires a validator to check "BCP 47 syntax of language and its presence for language-specific entries". standard/schema/canonical-reference.ts:6, scripts/source-schema.ts:42 and api/openapi.yaml:251 all have only minLength: 2. Neither the tag syntax nor the presence condition is checked anywhere in the pipeline the spec calls "the implementation source of truth" (specification.md:395). Either implement it or soften the requirement.

A7. license is a SHOULD in prose but a hard build failure in practice, and "current" overstates the accepted set.
specification.md:233: "license SHOULD be authored as a current SPDX license identifier". scripts/source-schema.ts:16-19,48-54: the accepted set is spdx-license-ids plus spdx-license-ids/deprecated, and a non-SPDX value throws in parseSource (:222). Deprecated ids pass; non-SPDX ids fail the build. That is a MUST, and not restricted to current ids.

A8. versioning.md misplaces one of the two "parse-time" checks.
versioning.md:115: "Two further checks run when a source file is parsed, before any record is built: a work MUST NOT declare the same citation system twice, and a locator MUST NOT contain /." Only the first is parse-time (scripts/source-schema.ts:178-190). The / check is assertValidLocator in scripts/compile.ts:199-203, called per reference during emission at :366.

B. ADR-0006 leftovers

The relation-vocabulary sweep was thorough — no exactMatch/closeMatch prose survives anywhere in the docs (the only repo occurrence is the deliberate out-of-enum rejection fixture at scripts/compile.test.ts:360). Four attribution leftovers remain:

B9. The normative-references table still credits SKOS with mapping relations.
specification.md:418: "| Concepts and mapping relations | SKOS |". After ADR-0006 the relations are prov:alternateOf / dcterms:isReferencedBy (public/contexts/v1.jsonld:41-49, standard/schema/mapping-assertion.ts:22); SKOS's remaining role is skos:prefLabel / skos:inScheme only.

B10. The §16 table has no PROV-O row at all (specification.md:405-422), although prov: is a published namespace and specification.md:140 cites prov:alternateOf normatively.

B11. Two vocabulary lists omit PROV-O.
standard/index.md:33 — "the context mapping TextRefs records onto SKOS, Dublin Core, and schema.org". get-started/index.md:20 — "published as JSON-LD against SKOS, Dublin Core and schema.org". Both omit the vocabulary that now carries the primary mapping relation (json-ld.md:9, v1.jsonld:6).

B12. MappingAssertion is still framed as "equivalence" although half the vocabulary is not.
specification.md:59 ("A curated equivalence between a Work and an external identifier"), :240, and versioning.md:90 ("reserved for genuine work-level equivalence claims"). But specification.md:264 and ADR-0006 define isReferencedBy as "a document or page about the work" — explicitly not an equivalence. The type's own summary line now excludes half its enum.

C. Examples that do not match the registry or would fail the build

C13. how-it-works.md "compiled registry record" blocks show the authoring license form.
how-it-works.md:54 "license": "CC-BY-SA-3.0", :81 "license": "CC-BY-4.0". Introduced as "The compiled registry record … is:", but real records carry the IRI (compile.ts:289, specification.md:233).

C14. The same blocks claim status: "active"; every record in data/ is draft.
how-it-works.md:59,85,103 vs data/works/plato.republic.yaml:5 and data/works/new-testament.yaml:5. The modified dates are stale too (:61 2026-06-03, :87 2026-05-31, :105 2026-06-02 vs compiled 2026-08-12/2026-08-11). Under ADR-0004 active carries the permanence promise, so this misstates the lifecycle state of live records on the page that introduces the lifecycle.

C15. The John.3.16 example is missing a resolver target and shows an unescaped URL.
how-it-works.md:68-88 shows one resolver_targets entry with …version=SBLG|reference=John.3.16 and no last_checked. data/works/new-testament.yaml:33 has %7C escaped and last_checked: '2026-06-03', and the compiled record for b6438d55-… has two targets (STEP Bible + Deutsche Bibelgesellschaft). The omission undercuts the point the passage is making — that many locations share one reference identity.

C16. Wrong Wikidata QID for the Dhammapada, in two files.
authoring.md:39 and mappings-and-resolver-targets.md:43 both show …/entity/Q220114. data/works/dhammapada.yaml:13 registers …/entity/Q748878 (mapping 7b3679c7-aafd-5db2-9fbc-98621d9aeb87). Both are presented as the work's registered mapping.

C17. Dhammapada provider/platform counts are wrong in three places.
authoring.md:24 "four different reading platforms", how-it-works.md:126 "(four providers, two languages, 423 references)", get-started/index.md:37 "three real providers". data/works/dhammapada.yaml:35,47,61 has three resolver entries across two providers (SuttaCentral ×2, ancient-buddhist-texts.net), and :56-59 records that no Wikisource resolver is wired up. The 423 references and two languages are correct.

C18. authoring.md's additional_systems example would fail the build.
authoring.md:343 uses citation_system: book-chapter; no data/systems/book-chapter.yaml exists and scripts/compile.ts:582-585 throws references unknown citation_system. authoring.md:334 also shows status: active for plato.republic, which is draft (data/works/plato.republic.yaml:5).

C19. authoring.md documents an examples key the schema rejects.
authoring.md:20 ("declares the locator regex and examples") and :417 ("regex, examples, references"). scripts/source-schema.ts:194-207SystemSource is a strictObject with no examples field, so an unknown key is an authoring error. No data/systems/*.yaml has one, and src/pages/id/system/[key]/index.astro:89-108 renders none.

D. Docs vs implementation drift

D20. related-systems.md tells implementers to read a mappings array that no record has.
related-systems.md:39: "Read external identifiers and resolver targets from the mappings array". Published Works carry alternateOf / isReferencedBy (standard/schema/work.ts:30-32); resolver targets live on CanonicalReference (canonical-reference.ts:27) and never were in mappings. This is the page's integration advice.

D21. use-cases.md attributes two nonexistent fields to CitationSystem.
use-cases.md:32: "a documented locator regex, a normalisation rule, and a list of valid reference types". standard/schema/citation-system.ts:5-11 and specification.md:162: id, key, type, preferred_label, description, locator_regex + admin metadata. Normalisation lives in prose inside description; "valid reference types" has no counterpart at all.

D22. Work.creators is missing from the OpenAPI schema.
Documented at specification.md:131,136, implemented at standard/schema/work.ts:29, in the context at v1.jsonld:18-22 — but api/openapi.yaml:205-231 lists only id, key, type, preferred_label, preferred_citation_system_key, alternateOf, isReferencedBy.

D23. url_by misses are described as silent; the compiler warns.
authoring.md:176: "References for chapters not in the map are silently skipped for this provider". scripts/compile.ts:263-272,371-374: a url_by miss returns null and increments warnings exactly like a missing template variable. This also contradicts authoring.md:158 on the same mechanism.

D24. The json-ld.md dcterms row omits two terms the context uses.
json-ld.md:23 lists created, modified, source, language, license, conformsTo, isReferencedBy — missing dcterms:description (v1.jsonld:17) and dcterms:isReplacedBy (v1.jsonld:80-83).

D25. A /reg/ route that does not exist.
authoring.md:420 "browse paginated reference lists from work/system pages"; url-layout.md:13 "from those record pages". src/pages/reg/ contains only index.astro and work/[key]/refs/[page].astro; src/pages/id/system/[key]/index.astro:93-108 lists works with counts only. There is no per-system reference list.

D26. verify and verify:fast are described as not running tests, but both do.
package.json:21-22 — both chains include npm test. Contradicted in three places: README.md:82 ("Prettier + astro check + production build — the CI gate"), AGENTS.md:12-13, and CONTRIBUTING.md:69 plus its site mirror.

E. Internal contradictions in get-started/

E27. mappings-and-resolver-targets.md contradicts itself and the spec on edition-level identifiers.
:120 — "If an identifier points to a whole edition, scan, or digital object …, it is not a MappingAssertion target at all". Same file :32 lists "DOIs, Handles, ARKs, PURLs, or URN:NBNs for editions or digital objects" as common mapping targets, and specification.md:262 normatively allows an edition as a target. The edge-case bullet forbids what both the same page and the spec permit.

E28. The TEI row claims passage-level mapping targets the spec forbids.
related-systems.md:26: "TextRefs MappingAssertions can point at a specific TEI anchor in a published edition". specification.md:261: per-passage external identifiers "MUST NOT be stored as separate MappingAssertion records"; standard/schema/mapping-assertion.ts:6-11 requires a Work IRI as subject.

E29. The PURL row inverts the /id/ vs /cite/ distinction.
related-systems.md:21: "TextRefs IDs are themselves HTTP URIs that redirect". url-layout.md:12 says /id/ is "the canonical, persistent URL of every record"; only /cite/ redirects (src/pages/cite/[...alias].astro:21). This is the opposite of url-layout.md's "/id/ is the registry. … /cite/ shortcuts to it."

E30. Prose calls reading URLs "mappings".
get-started/index.md:14 lists "the reading URL of the archive that holds the text" among "curated mappings", against mappings-and-resolver-targets.md:22 ("A MappingAssertion is about work-level equivalence. A resolver_targets entry is about dereferencing one passage."). A Stephanus locator, also in that list, is not a mapping either.

E31. "Two sources" introduces a five-item list.
authoring.md:127 — "Variables are drawn from two sources:" followed by items 1–5 at :129-134. All five are real (compile.ts:141-187, :232-252); only the count is wrong.

F. Mirror-pair divergence

AGENTS.md:50 and ROADMAP.md:41 require these pairs be edited in the same commit.

F32. The published contributing page gives the wrong branch. ⚠️ Most user-facing item in this issue.
CONTRIBUTING.md:110,114 — "Branch from staging for content/docs/blog; branch from main for infra …", "Open the PR against the branch you started from". src/content/docs/community/contributing.md:102,106 — "Branch from main." / "Open the PR against main."
The site version also omits the entire ## Branching model section (CONTRIBUTING.md:95-107, 13 lines) with no counterpart anywhere. And the guidance conflicts with the actual infrastructure: .github/dependabot.yml sets target-branch: staging, and data/.github/workflows/validate.yml:4-10,29 checks out parent/staging because "staging (not main) is the integration branch".

F33. The mirrored release checklist diverges, 9 steps vs 7, and the site's step 2 is a documented footgun.
CONTRIBUTING.md:125-133 vs src/content/docs/community/contributing.md:118-124. The published page prescribes bare npm run changelog, which the canonical file's step 3 explicitly warns "would file the commits under ## [Unreleased]" (it says to pass --tag explicitly). The site version also drops the CITATION.cff step and the Pages-dispatch step — the latter being the one that explains why a release PR stays blocked.

F34. AGENTS.md's list of mirrored files omits ROADMAP.md.
AGENTS.md:50 names CODE_OF_CONDUCT.md, CONTRIBUTING.md, SECURITY.md. ROADMAP.md:41 declares itself a mirror pair too. An agent following AGENTS.md alone would not know — plausibly how F32/F33 arose.

F35. The mirrored roadmap page links to itself.
ROADMAP.md:3 "mirrored on the docs site at https://textrefs.org/community/roadmap/" is carried verbatim into src/content/docs/community/roadmap.md:8, where it is a self-link. It should point at ROADMAP.md in the repo. (Otherwise the pair is byte-identical apart from frontmatter — the maintenance rule holds.)

F36. English association pages drop the legal-reservation notice the binding German originals carry.
de/association/statutes.md:9 (and governance.md:9, expenses.md:9, mission.md:9): "Dies ist eine Arbeitsfassung … Sie steht unter rechtlichem Vorbehalt bis zur Annahme durch die Gründungsversammlung." The English pages (association/statutes.md:8-10 etc.) carry only the :::caution[Non-binding translation] aside. An English reader learns the translation is non-binding but not that the underlying document is an unadopted draft. Otherwise all four pairs match exactly (Art. 1–22, all figures).

G. Stale or unreachable

G37. README.md:58 lists docs-internal/, which does not exist (and is not gitignored). The same block omits src/pages/, src/layouts/ and src/lib/, all documented in AGENTS.md:26-28.

G38. The conformance-failure issue template requires test IDs from a suite that does not exist.
.github/ISSUE_TEMPLATE/conformance_failure.yml:26-30 has a validations: required: true field asking to "List the conformance test IDs that fail". ROADMAP.md:35 lists the conformance suite as "planned".

G39. ROADMAP.md:21 "in progress: Replace remaining community placeholder pages" is stale — all seven files in src/content/docs/community/ carry substantive prose (203–1218 words); the only occurrence of "placeholder" in that directory is the roadmap line itself.

G40. versioning.md:34 attributes the -draft suffix to the wrong artefact.
"0.x releases stay working-draft regardless of any -draft suffix on the tag." The tag is v0.1.0, with no suffix; -draft is on the spec document version (specification.md:9). The spec-version-vs-release-tag distinction is deliberate (#68 §4) but is currently explained nowhere on the site — versioning.md's three-train table (:11-15) is where it belongs. Without it, a reader citing release v0.1.0 and landing on a spec headed 0.1.0-draft has nothing to reconcile the two.

G41. standard/index.md:38 — "A frozen, citable release will be tagged once the core stabilises" — reads oddly next to package.json/CITATION.cff at 0.1.0 and CHANGELOG.md dated 2026-08-12. Related to, but distinct from, the release-state claims already tracked in #68 (ROADMAP.md:27, CITATION.cff date vs the absent tag); not restating those here.


Checked and clean

Worth recording, so this ground is not re-audited:

  • All internal anchors resolve. All 27 /standard/…#… cross-references point at headings that exist; no get-started/ page uses a cross-file anchor.
  • Both example UUIDs recompute from their documented v5 seeds and namespaces (dc799d4b-…, b6438d55-…).
  • All reference counts are right: Iliad 15,693 / Analects 517 / Dhammapada 423 / Genesis 1,533 / Republic 1,475.
  • ADR-0006 sweep is otherwise complete — no exactMatch/closeMatch/skos: mapping prose survives in any doc; data/AGENTS.md:14 is correct.
  • Draft-lifecycle, tombstone, identifier-persistence and alias-minting rules agree across specification.md, identifier-syntax.md, versioning.md, ADR-0004 and the compiler. The six compiler invariants at versioning.md:108-113 match compile.ts:658-737 one for one.
  • Status / access / relation enums, the flat-key regex, UUID namespaces and seeds are consistent across docs, Zod, context and OpenAPI.
  • Bare-vs-qualified alias rules and the asymmetric reference_status default match compile.ts:405-408,595-596 and ADR-0005.
  • SECURITY.md and CODE_OF_CONDUCT.md are byte-identical to their mirrors; every npm run … command named in any doc exists in package.json; all root-relative links resolve except G37.

Suggested handling before the tag

A3, A4, D22 touch the published API contract and A1 is the spec's headline example — those are the ones that are cheapest to fix now and most expensive to have frozen into a citable baseline. F32/F33 mislead contributors today. The rest is safe to land after v0.1.0.


Audit performed with Claude Code.

Metadata

Metadata

Assignees

No one assigned

    Labels

    addressed-in-v0.1.0Addressed in the v0.1.0 release (PR #4)documentationImprovements or additions to documentationstandardThe published specification and schemas

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions