Skip to content

doc: make canonical URLs absolute and list root pages in llms.txt - #39913

Merged
bosconi merged 6 commits into
mainfrom
jc/docs-agent-followups
Oct 19, 2026
Merged

bosconi merged 6 commits into
mainfrom
jc/docs-agent-followups

Conversation

@bosconi

@bosconi bosconi commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Follow-ups promised in review of #39470.

Part of: EDU-101

What changed

  • Canonical and og:url are absolute. head.html printed .Permalink for both, which under --baseURL /docs is a path such as /docs/sql/create-source/. og:url must be an absolute URL. Both now use the absolute-permalink partial that the sitemap already uses.
  • Root-level pages are listed in the root llms.txt. License and Support have no section, so they were published in about/llms.txt, a directory with nothing else in it. The root index now lists them under their sidebar header ("About"), and about/llms.txt is no longer generated. The root index's orienting note says that pages outside any section follow the section list.
  • One partial per "which build is this" decision. The templates make two different decisions, and checked them three ways:
    • Is this Markdown published on the website, so that links in it must be absolute URLs, or is it the bin/gen-claude-skill bundle, where root-relative links name bundle directories? markdown-link-url checked ne Path "/", and markdown-body checked for a non-empty trimmed path. markdown-links-absolute.html now owns this. The two checks only disagreed for a host-only --baseURL, which no build uses.
    • Does the website publish a Markdown tree for this build at all? self-managed-docs/* branches build the HTML alone. has-markdown and index.llmstxt.txt each matched "self-managed" in the base path. site-has-markdown.html now owns this, and has-markdown calls it.
  • A comment next to the Hugo pin in ci/builder/Dockerfile says to compare doc/user/layouts/sitemap.xml with the built-in template on each Hugo bump. This changes the ci-builder image hash, so CI rebuilds the builder image for this PR.

Verification

Built with Hugo 0.152.2 before and after: the production HTML and Markdown passes (--baseURL /docs), the skill bundle (no --baseURL, as bin/gen-claude-skill runs it), the preview passes, the lint-docs.sh passes with a host in the base URL, and a self-managed HTML build. Each build was byte-identical across two runs of the same commit, so every difference below comes from this PR.

  • Partials refactor alone (first commit): all six builds byte-identical to the base branch.
  • Whole PR:
    • Every Markdown tree, production, preview, lint and skill bundle, is byte-identical.
    • In the production, preview and self-managed HTML, the only changed lines are og:url and canonical, on all 570 pages, and all 570 of each are now absolute. Production uses https://materialize.com/docs/..., self-managed https://materialize.com/docs/self-managed/<version>/..., preview https://preview.materialize.com/materialize/<PR>/.... The lint build's HTML is unchanged, because its base URL already has a host.
    • llms.txt changes as described above in the production, preview and lint builds. The self-managed build still publishes none.
  • check-docs-markdown.py --fail-on-links passes on the production Markdown, and test_check_docs_markdown.py passes.

Not changed: the 542 alias pages, rendered by layouts/alias.html, keep a path-only canonical. In production, website.sh replaces each with an S3 redirect, so their HTML is never served.

🤖 Generated with Claude Code

@linear-code

linear-code Bot commented Oct 16, 2026

Copy link
Copy Markdown

EDU-101

Base automatically changed from jc/mz-docs-optimization to main October 16, 2026 23:54
bosconi and others added 5 commits October 16, 2026 19:55
The Markdown templates told the website build from the skill bundle with
two different base-path checks, and the HTML templates told builds with a
Markdown tree from self-managed builds with a substring match repeated in
two places. markdown-on-website.html and site-has-markdown.html now own
those two decisions, and every caller uses them.

The production HTML and Markdown builds and the skill bundle are
byte-identical before and after.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With the bare-path baseURL "/docs", .Permalink has no scheme or host, so
both were paths. og:url must be an absolute URL, and an absolute
canonical leaves nothing for a crawler to resolve. Both now use the
absolute-permalink partial that the sitemap uses.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The pages at the docs root, License and Support, were published in an
about/llms.txt index, a directory that holds nothing else, since no
"about" section exists. The root llms.txt now lists them itself, under
their sidebar header, and about/llms.txt is no longer generated.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
doc/user/layouts/sitemap.xml copies Hugo's built-in sitemap template, so
a Hugo upgrade can leave it behind the version it overrides.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@bosconi
bosconi force-pushed the jc/docs-agent-followups branch from d8b3b3e to a3189ae Compare October 16, 2026 23:57
@bosconi
bosconi marked this pull request as ready for review October 17, 2026 00:12
@bosconi
bosconi requested review from a team as code owners October 17, 2026 00:12

@antiguru antiguru left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Some nits: markdown-on-website could have a more specific name markdown-links-absolute.

@antiguru
antiguru added this pull request to stack #39921 October 17, 2026 11:48
The partial decides whether links in the Markdown output must be absolute
URLs, so name it for that, per review.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@bosconi

bosconi commented Oct 19, 2026

Copy link
Copy Markdown
Member Author

@antiguru Thanks! Renamed markdown-on-website to markdown-links-absolute in 3316e05, and updated its doc comment to match.

@bosconi
bosconi merged commit c1ebe1a into main Oct 19, 2026
85 checks passed
@bosconi
bosconi deleted the jc/docs-agent-followups branch October 19, 2026 06:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants