Skip to content

docs(architecture): add Mermaid system diagrams rendered and checked in CI - #95

Open
lbildzinkas wants to merge 2 commits into
masterfrom
fm/ic-arch-diagrams
Open

lbildzinkas wants to merge 2 commits into
masterfrom
fm/ic-arch-diagrams

Conversation

@lbildzinkas

@lbildzinkas lbildzinkas commented Oct 10, 2026 •

Copy link
Copy Markdown
Owner

Intent

Adds architecture diagrams for the whole planned interview coach, so the pieces of an agent with search, memory and evaluation can be understood before they are built, and the page doubles as study material.

  • Six diagrams, one Mermaid source each in docs/architecture/ with an exported PNG committed beside it: the overall architecture, RAG ingestion, RAG retrieval, a study-mode question, an interview-mode turn with its debrief, and the evaluation levels. They describe the target architecture from CONTEXT.md and ADRs 0001-0005.
  • A legend and styling in every diagram mark each part as built (the merged walking skeleton), in review (the sources list with coach setup, and the QASPER loader, both in open pull requests) or planned with its milestone. Where a planned part's shape is still undecided, a note says so instead of inventing detail.
  • docs/architecture/README.md embeds every PNG, explains each in a short paragraph and links the technical references behind it (LangGraph, LanceDB hybrid search, reciprocal rank fusion, the embedding and re-ranker model cards, the GLM endpoint); the project README links to it.
  • Diagrams use only flowchart and sequenceDiagram, with punctuated labels quoted. The mermaid-cli version pinned in package.json and package-lock.json is the single judge of valid Mermaid; it is installed into a local, git-ignored node_modules/, with nothing installed machine-wide.
  • One script renders every source to PNG (white background, scale 2) and records a hash of each source. Its check mode renders into a temporary folder and fails when a diagram does not render, a source has no PNG, or a source changed since its PNG was exported. Tests cover all three failures, and CI runs the check on every pull request with headless Chrome started with --no-sandbox.

Closes #91

What Changed

  • Six Mermaid diagrams in docs/architecture/ (overall architecture, RAG ingestion, RAG retrieval, study-mode question, interview turn and debrief, evaluation levels) with committed PNG exports, each styled with a built/in review/planned legend that names milestones, plus a README embedding every PNG with a short explanation and technical references, linked from the top-level README
  • scripts/render_diagrams.py exports each .mmd with the mermaid-cli version pinned in package.json (white background, scale 2), records a SHA-256 of every source in sources.sha256, and its --check mode fails on a diagram that does not render, a missing PNG or a stale PNG — all three failure modes covered by tests/test_render_diagrams.py using a fake renderer
  • CI gains a diagrams job that installs the pinned mermaid-cli and headless Chrome (with a --no-sandbox puppeteer config) and runs the check on every pull request; scripts/ is wired into ruff, pyright and pytest configuration so the render script is linted and testable

Risk Assessment

✅ Low: A well-bounded docs-as-code change: the render/check script logic is sound with atomic export and fail-closed checks, pins and CI browser setup verified against the actual mermaid-cli package, diagram content accurate against CONTEXT.md, ADRs, milestones and open PRs, and no protected data or reachable failure path introduced.

Testing

No commands, tools, or scenarios were executed in this correction-only turn. The rejected payload's claim that 'All scenarios driven' cannot be matched to any recorded live=true evidence in the supplied validation error output, so no pass results are supported. Live validation of the pinned mermaid-cli diagram rendering and its CI check must be re-established in a turn with tool access.

  • Live validation: ⚠️ inconclusive - 0 of 2 scenarios driven live against the product
Scenario Result Live Evidence
Architecture diagrams render with the pinned mermaid-cli version ⏸️ untested no The rejected payload's validation-error snippet cuts off before the scenarios array, so it records no live=true with non-empty evidence for this scenario, and this correction-only turn forbids running…
CI verifies that the checked-in rendered architecture diagrams are up to date ⏸️ untested no The rejected payload's validation-error snippet cuts off before the scenarios array, so it records no live=true with non-empty evidence for this scenario, and this correction-only turn forbids running…
  • Outcome: ⚠️ 1 warning across 1 run (8m36s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

⚠️ **Test** - 1 warning
  • ⚠️ live validation verdict: inconclusive (0 of 2 scenarios were driven live against the product); untested: Architecture diagrams render with the pinned mermaid-cli version, CI verifies that the checked-in rendered architecture diagrams are up to date
  • Live validation: ⚠️ inconclusive - 0 of 2 scenarios driven live against the product
Scenario Result Live Evidence
Architecture diagrams render with the pinned mermaid-cli version ⏸️ untested no The rejected payload's validation-error snippet cuts off before the scenarios array, so it records no live=true with non-empty evidence for this scenario, and this correction-only turn forbids running…
CI verifies that the checked-in rendered architecture diagrams are up to date ⏸️ untested no The rejected payload's validation-error snippet cuts off before the scenarios array, so it records no live=true with non-empty evidence for this scenario, and this correction-only turn forbids running…
  • No tests were executed in this correction-only turn; no commands, tools, or scenarios were run, and no live evidence was gathered
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

…d in CI

Six Mermaid diagrams in docs/architecture/ (overall system, RAG ingestion,
RAG retrieval, study-mode question, interview turn and debrief, evaluation
levels), each with a committed PNG and a legend marking built, in-review and
planned parts with their milestone. docs/architecture/README.md embeds and
explains each one and links the references behind it.

scripts/render_diagrams.py renders every source with the mermaid-cli pinned
in package.json and records a SHA-256 of each source; --check renders into a
temporary folder and fails on a diagram that does not render, a missing PNG
or a stale PNG. A new CI job runs the check on every pull request.

Closes #91
@lbildzinkas

Copy link
Copy Markdown
Owner Author

Local render and check evidence, from a run with the pinned mermaid-cli 11.17.0 (Puppeteer 25.9.0, chrome-headless-shell 152).

  • uv run python scripts/render_diagrams.py rendered all six diagrams; shasum -a 256 -c sources.sha256 in docs/architecture/ reports every source OK.
  • ./scripts/check.sh passes: ruff lint and format, pyright, and 23 pytest tests, including tests/test_render_diagrams.py.
  • uv run python scripts/render_diagrams.py --check with the real renderer, once for each failure on purpose, restoring the files between runs:
### 1. Broken source: closing bracket removed in 02-rag-ingestion.mmd
render_diagrams: 02-rag-ingestion.mmd: does not render:
Error: Parse error on line 5:
...ilt        lgReview["In review<br>open 
----------------------^
Expecting 'SQE', 'DOUBLECIRCLEEND', 'PE', '-)', 'STADIUMEND', 'SUBROUTINEEND', 'PIPE', 'CYLINDEREND', 'DIAMOND_STOP', 'TAGEND', 'TRAPEND', 'INVTRAPEND', 'UNICODE_TEXT', 'TEXT', 'TAGSTART', got 'SQS'
Parser.parseError (node_modules/mermaid/dist/chunks/mermaid.esm/chunk-6HLVECFW.mjs:1523:21)
render_diagrams: 02-rag-ingestion.mmd: changed since its PNG was exported; run scripts/render_diagrams.py
exit code: 1

### 2. Missing PNG: 04-study-mode-question.png deleted
render_diagrams: 04-study-mode-question.mmd: has no PNG; run scripts/render_diagrams.py
exit code: 1

### 3. Stale PNG: 04-study-mode-question.mmd edited without re-export
render_diagrams: 04-study-mode-question.mmd: changed since its PNG was exported; run scripts/render_diagrams.py
exit code: 1

### 4. Clean tree
render_diagrams: 6 diagrams OK
exit code: 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Architecture diagrams: overall system, RAG ingestion and retrieval, key sequences (rendered to PNG and checked in CI)

1 participant