ci: skip scaffolding tests on Changesets release PRs

[mdamjanic7]

TL;DR: Changesets release PRs always fail the test_scaffolding job, because it installs a scaffolded app from npm and the release bumps workspace packages to versions that are not published yet. This skips that job on release PRs so their CI can go green instead of relying on merging past red checks.

Why

The test_scaffolding job scaffolds a Hydrogen app and installs its dependencies from the public npm registry. The skeleton template pins @shopify/hydrogen, @shopify/hydrogen-codegen, and @shopify/mini-oxygen with workspace:*, and the CLI build (packages/cli/tsup.config.ts → replaceWorkspaceProtocolVersions) bakes those into exact versions at build time.

On a Changesets release PR, those exact versions are the new versions being released — which are not published to npm until the PR merges. So npm install fails with ETARGET (e.g. No matching version found for @shopify/mini-oxygen@4.2.0), no lockfile is produced, and the "Verify lockfile" step fails.

This is not specific to any one PR. It reproduces on every release PR:

• The current release PR [ci] release semver #3824 fails all 5 scaffolding checks.
• Earlier release PRs [ci] release 2026.4.0 #3676 ([ci] release 2026.4.0) and [ci] release 2026.4.4 #3791 ([ci] release 2026.4.4) also failed all 5 and were merged anyway.

It became a problem once two changes combined: the scaffolding job was added (#3458) and the skeleton switched to workspace:* with build-time exact-version baking (#3475). Before that, the skeleton used caret ranges (e.g. ^4.0.0) that any already-published version could satisfy, so release PRs passed.

What this changes

• Adds a job-level if: guard to test_scaffolding that skips it when the PR comes from a changeset-release/ branch.

test_scaffolding:
  name: ⬣ Scaffolding tests (${{ matrix.os }} / ${{ matrix.pm }})
  if: ${{ !startsWith(github.head_ref, 'changeset-release/') }}

github.head_ref is the source branch on pull_request events and empty on push to main, so the job still runs on normal PRs and on pushes to main (after the release merges and the versions are published).

Risk

• This is a job-level skip, so the job reports a "Skipped" conclusion (the workflow still runs). GitHub treats a skipped required status check as satisfied, so this does not leave the check "Pending" and does not block merges. This is different from filtering at the on: level, which would leave a required check pending forever.
• We lose scaffolding coverage on release PRs only, where it can never pass pre-publish anyway. Coverage is unchanged on every normal PR (against published deps) and on pushes to main.

How to Test

1. This can't be exercised via a normal local run; it is verified by CI behavior on branch names.
2. On the next Changesets release PR (branch changeset-release/main), confirm the ⬣ Scaffolding tests (...) checks show Skipped rather than failing.
3. On this PR and any other feature PR, confirm the scaffolding checks still run.

[shopify]

Oxygen deployed a preview of your ci/skip-scaffolding-on-release-prs branch. Details:

Storefront	Status	Preview link	Deployment details	Last update (UTC)
Skeleton (skeleton.hydrogen.shop)	✅ Successful (Logs)	Preview deployment	Inspect deployment	July 1, 2026 2:35 PM

Learn more about Hydrogen's GitHub integration.

[Copilot]

Pull request overview

This PR updates the CI workflow to avoid running the test_scaffolding job on Changesets-generated release PRs, where scaffolding is expected to fail because the scaffolded app installs exact package versions that have not been published to npm yet.

Changes:

• Add a job-level if: condition to skip test_scaffolding when the PR source branch starts with changeset-release/.
• Document (inline) why the job is skipped for those PRs.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.