fix(remote-react-components): scope cross-version refs per test file - #2688
Merged
Conversation
The in-process cross-version harness keyed each ephemeral reference HTML file only on the test name + description. Two reused visual tests share a name — `Checkbox.browser.test.tsx` and `CheckboxButton.browser.test.tsx` both have a `Checkbox edge cases` test — so both read and wrote the SAME `.refs/*.html` file. They render structurally different trees (a lone `<Checkbox>` vs a `Flex` with two `<CheckboxButton>`s), so whichever test wrote the reference last made the other's version comparison mismatch. Since the cross-version job was added (2026-07-21) it failed on its first scheduled run and every one since: deterministic, not a flake, and not a real version incompatibility (current and the compared old version render identically once keyed apart). Include the source-file basename in the reference key, mirroring how vitest scopes browser screenshots per test file. This resolves the collision and prevents any future same-name collision. No impact on the visual suite or committed screenshots; the `.refs` are regenerated each run. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mfal
requested review from
Jan-Eimertenbrink,
Lisa18289,
ins0 and
maaaathis
as code owners
July 23, 2026 09:14
Contributor
Coverage Report for ./packages/components/
File CoverageNo changed files found. |
… edge cases" The test was mis-named identically to Checkbox's own "Checkbox edge cases" test. The preceding commit made the cross-version reference key file-scoped so the shared name is no longer harmful, but the name is still confusing. Rename it for clarity; the render is unchanged, so the committed visual baselines are moved (git mv) rather than regenerated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
🚀 Preview DeploymentPreview environments are ready:
Images:
|
mfal
enabled auto-merge (squash)
July 23, 2026 10:21
Lisa18289
approved these changes
Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The scheduled Run scheduled visual tests workflow's
cross-versionjob has failed on every run since it was introduced (2026-07-21) — specifically the in-process harness, with aCheckbox edge cases(CrossVersion)structural-snapshot mismatch against the previous published version.It is not a version incompatibility. The in-process harness keyed each ephemeral reference HTML file only on
testName + description:`${REF_DIR}/${slugify(`${testName}-${description}`)}.html`Two reused visual tests share the exact name
"Checkbox edge cases":Checkbox.browser.test.tsx→ renders one<Checkbox>(7-tag tree)CheckboxButton.browser.test.tsx→ renders aFlexwith two<CheckboxButton>s (18-tag tree) — the test is mis-named"Checkbox edge cases"Both therefore read and write the same
.refs/*.htmlfile. Whichever test writes the reference last makes the other's comparison mismatch. This is deterministic (it failed on the job's very first scheduled run and every one since), and unrelated to the actual old version — current and the compared version render identically once the tests are keyed apart.The normal visual screenshot suite does not hit this because vitest scopes screenshots per test file (
__screenshots__/<file>/…); only the cross-version harness used a flat, non-file-scoped key.Fix
Include the source-file basename in the reference key, mirroring vitest's own per-file screenshot scoping. Small, self-contained change to
crossVersionEnvironment.tsx. This resolves the collision and prevents any future same-name collision.Validation
@mittwald/flow-remote-react-components@0.2.0-alpha.932locally and reproduced the failure via the full 83-file corpus.test:compile(tsc), ESLint, and Prettier all clean..refsare wiped and regenerated on every run.Notes for reviewers
alpha.933, a docs-only release that renders identically to932), so it would not durably fix the failure.CheckboxButtontest from"Checkbox edge cases"to"CheckboxButton edge cases"for clarity. Left out here because the description also names that test's committed visual screenshot baselines (CheckboxButton.browser.test.tsx/Checkbox-edge-cases-*.png), so renaming would require regenerating those (including the Linux baseline via theupdate-screenshotslabel). The file-scoping fix makes the duplicate name harmless regardless.Buttonscenarios intermittently hit ~180–240s@quilted/threadsconnection-timeout hangs; the workflow's retry loop usually absorbs it.🤖 Generated with Claude Code