BUG: SessionStorageCache_Chrome fails in CI because the client makes no json post

[jwrosewell]

What fails

SessionStorageCache_Chrome, in both its (True) and (False) rows,
fails the Examples workflow on ubuntu with:

Assert.IsTrue failed. 'condition' expression: 'CountJsonPosts() >= 1'.
the first page must call the json endpoint at least once

It fails on every pull request opened since 28 August 2026, and it blocks
pull request 21 and
pull request 22, neither of
which causes it.

What has been established

Neither open pull request caused it. Pull request 21 changes no
.mustache file, and it fails the identical two tests with the identical
assertion as pull request 22, which does. Both runs also carry the same
surrounding log lines.

The test arrived from the shared suite rather than from this
repository. The Examples workflow last passed on 27 July 2026, run
30250692221. SessionStorageCacheTests was added to
51Degrees/selenium-api-tests
on 30 July in commit 4fd1f13 and hardened on 6 August in f296259,
6ca6fe7 and 1e3cdea. It carries TestCategory("Contract"), which is
the filter this repository runs, and
ci/run-integration-tests.ps1 clones the
suite with git clone --depth 1 and no ref, so it always takes the
default branch tip. The first Examples run after the July gap, on 28
August, went red without anything changing here.

The rust integration satisfies the contract when a real browser drives
it. Running dd-web-getting-started-cloud from the pull request 22
branch against the live cloud on 30 August 2026, behind a copy of the
suite's proxy and driven by headless Chrome, produced exactly one
POST /51dpipeline/json and rendered a real device id. Running the
actual SessionStorageCache_Chrome test locally, with the suite at the
same commit CI clones (578ea63), passes the CountJsonPosts assertion
and stops later for an unrelated reason, which is
issue 23. That holds with
the paid key and with the free key, and with --headless and with
--headless=new.

It is not the headless mode. The runner image carries Google Chrome
151, and Chrome removed the old headless implementation in version 132,
so --headless on the runner already means the new mode. Both spellings
send byte-identical headers, being a HeadlessChrome user agent together
with sec-ch-ua client hints naming Google Chrome.

The remaining question

The client only posts when the served payload carries a JavaScript
property with a body to run. Measured against the running example:

headers sent	JavaScript bodies returned
plain Chrome user agent	3
HeadlessChrome user agent, no client hints	none
HeadlessChrome user agent, with client hints	3

So the payload a browser gets depends on whether the client hints reach
the cloud. On a workstation they do, which is why the post happens and
the test passes. What the same request returns from a GitHub runner is
not known, and that is the gap.

What is being done about it

Branch ci/diagnose-selenium-json-post adds a temporary block to
ci/run-integration-tests.ps1 that prints the Chrome and ChromeDriver
versions and, for each of the three header sets above, how many runnable
JavaScript bodies the cloud returns to that runner. It is branched from
main rather than from either pull request, so the run also shows
whether main alone fails.

Remove that block once it has answered the question. No resource key is
printed by it, because the key stays inside the example rather than in
the requests it makes.

Written with AI assistance under James Rosewell's direction and needs
human review.

[jwrosewell]

Answer

The diagnostic runs have found it, and it is not in this repository.

The browser on the runner receives an include carrying no JavaScript
to run, so the client correctly completes without posting and
CountJsonPosts() >= 1 fails. Run
33319243453:

  page state: complete
  device id rendered: '15364-18110-96032-132572'
  requests the browser made:
    GET /page1
    GET /51Degrees.core.js
  posts to the json endpoint: 0
  include as delivered to the browser: bytes=14506 runnable=NONE

The cause is one request header. The runner's Chrome sends:

sec-ch-ua: "Not=A?Brand";v="99", "Google Chrome";v="151", "Chromium";v="151"
user-agent: ... HeadlessChrome/151.0.0.0 ...

Replaying those exact headers against the example reproduces it every
time, and changing one thing at a time isolates it to the version on the
GREASE brand, which is the deliberately meaningless entry Chrome adds so
that parsers do not assume a fixed shape. Holding everything else equal
and varying only that number:

user agent	GREASE version	browser detected	version	JavaScript returned
ordinary Chrome	8	Chrome	151	yes
ordinary Chrome	24	Chrome	151	yes
ordinary Chrome	99	Chrome	151	yes
HeadlessChrome	8	Chrome	151	yes
HeadlessChrome	24	Chrome	134	yes
HeadlessChrome	99	Chrome Headless	Unknown	no

So with a HeadlessChrome user agent the GREASE version leaks into the
version resolution. At 99 the brands stop resolving altogether and
detection falls back to the user agent, giving "Chrome Headless" with an
unknown version, and no client-side JavaScript is offered for that. At
24 it resolves to Chrome 134 rather than Chrome 151, which is also
wrong, though it still returns the JavaScript so nothing visibly breaks.
An ordinary Chrome user agent is unaffected at every value, so ordinary
traffic does not hit this.

That also explains why this is stable rather than flaky, and why it
passes on a workstation. Chrome picks the GREASE version per build, and
the two builds involved pick different ones. The runner carries Chrome
151, which sends 99 and fails. The workstation carries Chrome 152, which
sends 24 and passes. The same test therefore passes locally and fails in
CI with no difference in the code.

What follows

Neither open pull request is involved, confirmed twice over. This
branch is cut from main and carries no product change, and it fails
identically, two failures with three passing and four skipped. Nothing
in rust decides the include's contents, so any language's example on
that runner would behave the same.

Two separate things now need owners.

1. The detection result for a HeadlessChrome user agent depends on the
   GREASE version, and at 99 it loses the browser entirely while at 24
   it reports the wrong version. That is a device detection question
   rather than an SDK one and needs the Data team, not this repository.
2. The shared suite asserts that the first page must post, which a
   client cannot do when no JavaScript comes back. It should skip in
   that case, the way it already skips when the include carries no
   session id, so the suite reports "nothing to test here" instead of a
   failure. That belongs in
   51Degrees/selenium-api-tests.

Until one of those is done, the Examples workflow stays red on every
pull request in this repository, including
21 and
22, for a reason that has
nothing to do with the changes in them.

The diagnostic branch has answered its question. The block it adds
should not be merged, and
pull request 25 is closed,
with the branch kept so the run can be repeated if needed.

Written with AI assistance under James Rosewell's direction and needs
human review.