ci: platform-checks compiles cold on every main push, unlike sdk on the same runner role

The platform-checks slice declares cache_mode: "none", so every run of it compiles from scratch, on main as well as on pull requests. The sdk slice sits on the same runner_role: platform-build and declares pr-isolated. I cannot find anything in ci/ci.md that explains the difference, so this is a question before it is a proposal.

What it costs today

Platform checks (windows-unit) on the ten most recent pushes to main, 2026-09-22:


durations	18, 19, 19, 20, 21, 23, 23, 23, 24, 24 min
mean	21.4 min
sccache on one of those jobs	1005 compile requests, 2 cache hits, 807 compilations, hit rate 0.25%

Platform checks (macos-unit) is the same slice and also cold, 12m43s on a recent run.

Pushes to main that trigger Main · Windows: 9 on 09-19, 14 on 09-20, 18 on 09-21, 10 on 09-22 by 11:01Z. At roughly fourteen pushes on a full day that is about five hours of Windows runner per day for this one row, plus about three for the macOS one, all of it recompiling a dependency graph that did not change.

Why it looks like one word

scripts/plan-ci.py:564 reads:

def _planned_cache_mode(definition: dict[str, Any], profile: str) -> str:
    mode = definition["cache_mode"]
    if mode != "pr-isolated":
        return mode
    if profile == "main":
        return "trusted-readwrite"
    if profile == "manual-full":
        return "trusted-readonly"
    return mode

So a slice declared pr-isolated is already PR-isolated on pull requests and becomes trusted-readwrite on main, which is what sdk gets on the same runners. A slice declared none stays none everywhere, including on main where the trust boundary is not the same as on a PR.

Changing platform-checks to pr-isolated would therefore be one word in ci/slices.yml, with no new authority on pull requests.

What I am not claiming

I have not measured what a warm cache actually buys here, and I will not guess: a 0.25% hit rate says there is nothing to read from, not how much a populated store would save. The compile is 807 crates, most of which do not change between pushes, so the ceiling looks high, but that is an expectation and not a number.

I also do not know whether none is deliberate. It may encode a trust decision about these runners that sdk does not need, or it may simply be what the slice was born with. If it is deliberate, this issue is answered by saying so, and it would be worth a line in ci/ci.md so the next person does not ask again.

Offer

If the answer is "try it", I can prepare the catalog commit in the shape 947222559 took, watch the first few main runs, and report the before and after durations with the sccache statistics. If the row gets the nine crates measured on #1985, a cold build will only get longer, so the question is worth settling before rather than after.

Measured from the public run logs; nothing here needs a Windows machine to reproduce.

Update with a narrower diagnosis and a concrete proposal.

Current cost. Platform checks (windows-unit) on the last 10 successful Main · Windows runs: 22.9 to 28.4 min, mean 26.1 min (21.4 min when I opened this; the row has grown since).

It is not only cache_mode. Comparing the two Windows slice workflows on main (b380e140c):

ci-windows-host-slice.yml sets SCCACHE_GHA_ENABLED: "true" and RUSTC_WRAPPER: sccache, then runs ./.github/actions/configure-sccache-gha after mozilla-actions/sccache-action, fed by the allow_native_github_cache / allow_depot_remote_cache outputs of its runner policy.
ci-platform-checks-slice.yml installs sccache-action too, and the repository's .cargo/config.toml makes cargo use it, but no step gives it a backend. Its runner_policy already computes the same two allow_* outputs and the job already runs audit-depot-pr-isolation with them; they are only used for the macOS llama cache. So sccache runs with a job-local disk cache on a fresh runner every time, which is the 0.25% hit rate above.

Proposal, if none is not a deliberate trust decision for these runners:

In ci-platform-checks-slice.yml, for the Windows unit job: configure-sccache-gha after sccache-action with the two allow_* inputs, plus the same SCCACHE_GHA_ENABLED / RUSTC_WRAPPER env the host slice uses. PR runs keep the isolation the host slice already has, since the same policy action decides what a PR may read or write.
In ci/slices.yml, platform-checks declares pr-isolated like sdk, so the plan states what the job does (trusted-readwrite on main, isolated on PRs, per _planned_cache_mode).

Part 2 is a protected-catalog change, so it would be a catalog-only maintainer merge like #2068, and part 1 is a lane workflow change that PR lanes load from main, so neither can prove itself before landing. I have not measured the warm-cache gain and will not guess it. Happy to open both if you confirm the direction.

Update: the Windows half of this is fixed. #2374 (Oct 12) wired configure-sccache-gha into ci-platform-checks-slice.yml for Windows. On main's latest Platform checks (windows-unit) (job 118444921110): 2120 compile requests, 1431 hits, 80.8% hit rate, against 0.25% when I opened this. The job still takes 24 to 41 min on main (last 12 runs), but that time is now Run Windows unit tests itself (27 min), not a cold build.

macOS is unchanged. The two sccache steps are gated on platform == 'windows', so setup-macos-lld starts sccache with a local disk cache on a fresh runner, and nothing restores Rust build state between runs (only the static Metal ABI is cached). Run macOS unit tests takes 22 min on main (job 118449867228). The macOS host slice handles this with Swatinem/rust-cache behind allow_native_github_cache; the unit row could do the same, or drop the Windows gate on the sccache steps. I can't run macOS CI here to measure what either buys, so I'd rather leave that choice to you than guess.

Evidence — reproduced on current base (configuration-level)

Verified against fix/megabugs @ 973ba7faf0 (identical on main). No fix applied yet.

1. The cited owner has moved. scripts/plan-ci.py no longer exists. The planner is Rust: tools/xtask/src/ci_plan/plan.rs:33 owns planned_cache_mode, and tools/xtask/src/ci_plan.rs:104 stores it per slice. Semantics are unchanged from the Python version the report quotes.

2. The divergence is present in the checked-in catalog. ci/slices.yml declares both slices on the same runner role with different cache authority:

slice	line	`runner_role`	`cache_mode`
`platform-checks`	248	`platform-build`	`none`
`sdk`	251	`platform-build`	`pr-isolated`

platform-checks is the only platform-build-role slice with no cache. native-abi/native-ffi also declare none, but they run on the separate self-hosted native-abi role, so they are not the comparison.

3. The planner preserves it in every profile. planned_cache_mode maps pr-isolated → trusted-readwrite (main), trusted-readonly (manual-full), pr-isolated (all pr-*), and returns none unchanged for none. The committed expected fixtures show the asymmetry on the current base:

.github/fixtures/ci_plan/control-main.plan.json — "platform-checks":"none" alongside "sdk":"trusted-readwrite"
.github/fixtures/ci_plan/manual-full-force-all.plan.json — "platform-checks":"none" alongside "sdk":"trusted-readonly"
PR fixtures (native-ffi, reverse-dependency-log-store, windows-log-store) — "platform-checks":"pr-isolated"

So on main the Windows/macOS hosted compile slice runs with no cache while the SDK slice on the identical runner role gets a read-write cache.

4. PR isolation does not depend on the catalog value. .github/workflows/ci-runner-contract-slice.yml:48 fails any pr-* plan containing a trusted-readwrite mode, and ci/ci.md §11.1 requires PR jobs to restore only and forbids PR writes to the main namespace. pr-isolated is therefore the correct declaration for this slice; none is not protecting the PR boundary, it is only removing restore on main. The sdk slice is the existing proof that a platform-build slice can declare pr-isolated and stay PR-restore-only.

5. Scope check against the Cargo-archive rule. ci/ci.md §11.1 forbids restoring per-shard Cargo target archives and per-object GHA publication for the Linux Clippy, Rust tests, host, and runtime families. platform-checks is the Windows/macOS hosted family, so aligning it does not touch that rule.

Not re-verified here: the 0.25% sccache hit rate and ~21 min figures are the reporter's measurements. I confirmed the configuration and planner behavior that produces a cold compile, not the historical hit-rate number, and I did not rerun the expensive hosted workflows.

Planned fix and its merge constraint. Smallest correct change is catalog-only: platform-checks cache_mode: "none" → "pr-isolated", with the affected ci_plan fixtures regenerated. Note the sequencing constraint from ci/ci.md §11.1 ("Catalog evolution is a sequenced maintainer merge"): the protected planner compares the PR's ci/slices.yml byte-identically against the default-branch copy, so a branch that edits the catalog cannot pass its own Plan gate until that catalog change lands on the protected default branch. The PR will therefore show a red Plan gate by design until the catalog commit is landed; the compare must not be relaxed to work around it.

Correction to the paths and one value in the comment above

Verified by reading the fixture files directly.

Wrong path. The planner goldens are at tools/xtask/tests/fixtures/ci_plan/expected/, not .github/fixtures/ci_plan/. Correct references:

tools/xtask/tests/fixtures/ci_plan/expected/control-main.plan.json
tools/xtask/tests/fixtures/ci_plan/expected/manual-full-force-all.plan.json
tools/xtask/tests/fixtures/ci_plan/expected/windows-log-store.plan.json, .../native-ffi.plan.json, .../reverse-dependency-log-store.plan.json

Wrong value. I wrote that the PR-profile fixtures show "platform-checks":"pr-isolated". They do not. Read from the committed goldens:

fixture	`profile`	`platform-checks`	`sdk`
`control-main`	`main`	`none`	`trusted-readwrite`
`manual-full-force-all`	`manual-full`	`none`	`trusted-readonly`
`windows-log-store`	`pr-ready`	`none`	not selected
`native-ffi`	`pr-ready`	`none`	not selected
`reverse-dependency-log-store`	`pr-ready`	`none`	not selected

This makes the gap wider than I stated, not narrower: platform-checks resolves to none in every profile, so the slice gets no cache on PRs either, while sdk on the same platform-build role gets trusted-readwrite on main, trusted-readonly on manual-full, and restore-only isolation on PRs. Points 1, 2, 4, and 5 above are unchanged.

One implementation consequence found while preparing the fix. The planner contract cases do not read the live ci/slices.yml; tools/xtask/tests/migration_ci_plan/support.rs materializes each case's manifests from tools/xtask/tests/fixtures/ci_plan/manifests/<case>/slices.json. So the catalog change has to be mirrored into those fixture manifests and the affected goldens regenerated, otherwise the contract test would keep asserting the old none value from a stale fixture copy.

Conclusion: NOT-REPRODUCIBLE on current `main` — no fix applied

The reported behavior does not occur, and the stated mechanism is not connected to the runtime path. No code change and no PR.

1. Current `main` run shows a warm, read-write cache on this exact slice

main_windows.yml run 40248201661 (branch main, 2026-10-26T00:00:01Z, success), job 121337069666 Platform checks (windows-unit):

00:24:35Z  Baked sccache is using job-local disk with a read_write GitHub Actions cache.
00:51:37Z  Compile requests                   2127
00:51:37Z  Cache hits                         1438
00:51:37Z  Cache misses                        340
00:51:37Z  Cache hits rate                   80.88 %
00:58:13Z  Cache hits rate                   80.00 %

Selector job 121337069602: allow_native_github_cache: true, allow_depot_remote_cache: false.

So the Windows platform compile job on main already runs with a read-write GitHub Actions compiler cache at an 80.88% hit rate, not 0.25%.

2. `cache_mode` is planner metadata; nothing in the runtime path reads it

plan.cache_modes has exactly one consumer in the entire CI surface:

.github/workflows/ci-runner-contract-slice.yml:48 — jq -e 'all(.cache_modes[]; . != "trusted-readwrite")', a pull-request-profile assertion.

No workflow or action reads it. The actual cache decision for this lane comes from .github/actions/configure-sccache-gha/action.yml, which keys off the event name (pull_request* → read-only, disk-only; otherwise disk,gha with read_write) plus allow_native_github_cache / allow_depot_remote_cache from .github/actions/select-ci-runners, wired in .github/workflows/ci-platform-checks-slice.yml:142-144. Changing platform-checks from none to pr-isolated in ci/slices.yml would change only the input to that one assertion — zero runtime effect.

3. What is actually true

The catalog inconsistency is real: ci/slices.yml line 248 declares platform-checks with cache_mode: "none" while line 251 declares sdk with cache_mode: "pr-isolated", both on runner_role: "platform-build". It is inert metadata, not a cause.
The macOS half of the report is a deliberate design choice, not a cache-mode consequence: in .github/workflows/ci-platform-checks-slice.yml both the sccache install (line 140) and the cache configuration (line 142) are gated if: ${{ matrix.check.platform == 'windows' }}, so macOS jobs never had sccache regardless of the catalog value.
If compile time on this slice is still a concern, the owning levers are the runner-policy outputs and the sccache cache-key scope inside configure-sccache-gha, not the slice catalog. That is a different investigation from the one this issue describes.

4. Corrections to my earlier comments on this issue

Restated here so the final record is self-contained.

Fixture path: the planner goldens are tools/xtask/tests/fixtures/ci_plan/expected/, not .github/fixtures/ci_plan/.
PR-profile value: platform-checks resolves to none in the pr-ready goldens too, not pr-isolated.

5. Disposition

No source, test, workflow, or catalog change. A regression test asserting platform-checks plans as trusted-readwrite on main was drafted and discarded: it would have pinned planner metadata that no runtime consumer reads, presenting an unused field as a behavioral contract. Catalog and cache policy are unchanged.

Accuracy amendment to the comment above

I posted that before reading the two maintainer updates on this issue. They are consistent with my measurements, and one of my sentences overreached.

What my evidence does establish. Independently of the earlier measurement, main_windows.yml run 40248201661 (2026-10-26) logs read_write GitHub Actions cache and Cache hits rate 80.88 % on Platform checks (windows-unit). That reproduces the maintainer's 80.8% on job 118444921110 and confirms the Windows cold-build / 0.25%-hit-rate condition no longer occurs on main. It also confirms the mechanism claim is wrong on current source: plan.cache_modes is consumed only by the pull-request-profile assertion in ci-runner-contract-slice.yml, so the cache_mode: "none" catalog entry cannot have caused the original cold build either — the real cause was the missing configure-sccache-gha wiring, which #2374 added at ci-platform-checks-slice.yml:140-144.

What my evidence does not establish. I wrote "the reported behavior does not occur." That is too broad. The duration concern is not disproven by a hit-rate number: the maintainer's measured timings stand — windows-unit 22.9–28.4 min over 10 runs, 24–41 min over the last 12, and Run macOS unit tests ~22 min on main. What changed is where that time goes: test execution, not a cold build.

Correction on macOS. I called the Windows-only sccache gate "a deliberate design choice." I had no evidence for intent. The gate is real (matrix.check.platform == 'windows' at lines 140 and 142), but per the maintainer update the macOS cache question is an open, unmeasured decision — setup-macos-lld starts sccache on a job-local disk with nothing restored between runs, and the two candidate fixes (a Swatinem/rust-cache step behind allow_native_github_cache, or dropping the Windows gate) have not been measured. This issue still has that live half.

On the catalog entry. platform-checks still declares cache_mode: "none" while sdk declares pr-isolated on the same platform-build role. Aligning it is a truthful-plan/documentation change that needs a catalog-only maintainer merge, and it will not change any runtime behavior or recover the macOS time.

Disposition unchanged: no source, test, workflow, or catalog change from me. Ledger status is recorded as NOT-REPRODUCIBLE for the cache_mode mechanism and the Windows cold-build claim, with the macOS half left open.

docs: scope the #2003 finding to the cache_mode premise

Closing, this is fixed on main. Today's Main · Windows run (41291374619) has Platform checks (windows-unit) on a read_write cache at 90.48% hits, and the last runs take 7 to 8 min against 21 min when I opened this. Thanks Nick for tracking it down.

Navigation Menu

Repository navigation

ci: platform-checks compiles cold on every main push, unlike sdk on the same runner role #2003

What it costs today

Why it looks like one word

What I am not claiming

Offer

Evidence — reproduced on current base (configuration-level)

Correction to the paths and one value in the comment above

Conclusion: NOT-REPRODUCIBLE on current `main` — no fix applied

1. Current `main` run shows a warm, read-write cache on this exact slice

2. `cache_mode` is planner metadata; nothing in the runtime path reads it

3. What is actually true

4. Corrections to my earlier comments on this issue

5. Disposition

Accuracy amendment to the comment above

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

ci: platform-checks compiles cold on every main push, unlike sdk on the same runner role #2003

Description

What it costs today

Why it looks like one word

What I am not claiming

Offer

Activity

Virgile-pct commented on Sep 28, 2026

Virgile-pct commented on Oct 20, 2026

ndizazzo commented on Oct 26, 2026

Evidence — reproduced on current base (configuration-level)

ndizazzo commented on Oct 26, 2026

Correction to the paths and one value in the comment above

ndizazzo commented on Oct 26, 2026

Conclusion: NOT-REPRODUCIBLE on current main — no fix applied

1. Current main run shows a warm, read-write cache on this exact slice

2. cache_mode is planner metadata; nothing in the runtime path reads it

3. What is actually true

4. Corrections to my earlier comments on this issue

5. Disposition

ndizazzo commented on Oct 26, 2026

Accuracy amendment to the comment above

Virgile-pct commented on Nov 1, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Conclusion: NOT-REPRODUCIBLE on current `main` — no fix applied

1. Current `main` run shows a warm, read-write cache on this exact slice

2. `cache_mode` is planner metadata; nothing in the runtime path reads it