Run example-app browser tests against the current SDK - #1500
Merged
Merged
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
brianr
added this pull request to stack #1501
September 28, 2026 17:09
AI Agent Review LGTM (openai, openai-astra)LGTM. No blocking findings were found. Review: run example-app browser tests against the current SDKLGTM. I found nothing to fix in the changed lines. What I checked in the checkout
Outside the diff (not findings)
I could not run anything, so I have not confirmed that any test or CI step passes. |
brianr
force-pushed
the
fix/example-tests-use-current-sdk
branch
from
September 28, 2026 18:25
63ad672 to
17285ac
Compare
Base automatically changed from
deps/examples-drop-webpack-dev-server
to
master
September 28, 2026 18:34
test/examples/react.test.ts and webpack.test.ts loaded committed bundles of rollbar.js 2.7.1 and 2.13.0 from 2019, so they passed or failed regardless of the SDK in the repo. Build those apps fresh instead. - Add `npm run build:test-examples`: packs the SDK's current dist/ and installs and builds react-16 and webpack via validate-examples.js, which now accepts example names. - Run it in CI before the browser tests, and at the start of `npm test`. `test:wtr` on its own fails with a hint to run it if a build is missing. - Remove and gitignore both apps' dist/. - validate-examples.js now removes an example's gitignored lockfile and installed rollbar before installing. The tarball keeps its version across SDK rebuilds and the lockfile pins its old integrity, so npm otherwise reinstalls the cached copy and local runs test a stale SDK. - Update the tests for 3.x: requests are sent a tick later, and the access token is sent in the X-Rollbar-Access-Token header rather than the body (or a stored JSON payload). Drop a 2019 Travis Chrome 62 workaround. - Update the example READMEs, CONTRIBUTING.md, CLAUDE.md and AGENTS.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brianr
force-pushed
the
fix/example-tests-use-current-sdk
branch
from
September 28, 2026 18:34
17285ac to
39f0cbd
Compare
waltjones
approved these changes
Sep 28, 2026
devtools-agent Bot
pushed a commit
that referenced
this pull request
Sep 28, 2026
Resolve the conflict in scripts/validate-examples.js between master's named-example filter (#1500) and this branch's engines.node skip: an example skipped for the running Node version is no longer reported as unknown when named, and its skip line is only printed when it was requested (or when no names are given). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
devtools-agent Bot
pushed a commit
that referenced
this pull request
Sep 28, 2026
Picks up #1500, whose CI step `npm run build:test-examples` failed here: lint-action checks out the PR head before later steps run, so the job used master's ci.yml with this branch's older package.json, which lacks that script. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
test/examples/react.test.tsandtest/examples/webpack.test.tsload committed bundles that were last rebuilt in 2019. They contain rollbar.js 2.7.1 and 2.13.0, so these tests passed or failed regardless of the SDK in the repo. This PR removes those bundles and builds both apps against the current SDK before the browser tests.How the builds get made
npm run build:test-examples. It runsnpm run pack(tarball of the SDK's currentdist/), thenvalidate-examples.js -p react-16 webpack. That's the same install-and-build CI already does for every example, limited to these two.validate-examples.jsnow accepts example names; with none it validates all examples, as before.npm testruns it first, so the documented command works with no extra setup (about 9s with warmnode_modules).npm run test:wtron its own doesn't build anything. If a build is missing, the example tests fail with, for example,Failed to load examples/react-16/dist/index.html: HTTP 404. Run npm run build:test-examples to build the example apps.Before, a missing bundle failed later with a confusing assertion error.examples/react-16/dist/andexamples/webpack/dist/are removed and gitignored in each example's.gitignore.Stale-tarball fix in
validate-examples.jsThe tarball keeps version
3.1.0across SDK rebuilds, and an example'spackage-lock.jsonfrom an earlier install pins the old tarball's integrity. npm then reinstalls the old copy from its cache, even afternode_modules/rollbaris deleted. I reproduced this locally: after repacking, the example still installed the old SDK. Local runs would have silently tested a stale SDK. The script now removes the example's lockfile and installedrollbarbeforenpm install. Example lockfiles are gitignored and none are committed. CI always installs fresh, so it wasn't affected. This also fixes the same stale install for local runs ofvalidate:examples.Test updates for the 3.x SDK
Against fresh builds, all 6 tests failed for two reasons, both intended SDK behavior:
api.postItemschedules the request withsetTimeout(0), so the tests now waitsetTimeoutAsync(1)beforeserver.respond().test/examples/universalBrowser.test.tsalready does the same.X-Rollbar-Access-Tokeninstead of the body, so the tests check the request header.buildJsonPayloadno longer embeds it, so the stored-payload test now asserts the token is absent from the stored JSON. The header is added whensendJsonPayloadsends it.I also removed the webpack test's Travis Chrome 62 workaround from 2019, which parsed the user agent to pick an expected message.
Docs
The example READMEs' "commit a new bundle" steps are replaced with how the builds are made now.
CONTRIBUTING.md,CLAUDE.mdandAGENTS.mdmentionbuild:test-examples.Validation (Node 22, npm 10)
npm run build:test-examplesbuilds both apps in about 9s. The resulting bundles contain3.1.0, andgit statusstays clean.npm test, starting with no example builds: both apps built, then 52/52 browser test files and 621 tests passed, then the server suite passed. Exit 0, 27s.npm run lint(--max-warnings 0),npm run format:checkandnpm run typecheck:testsall pass.validate-examples.js -n -pstill lists all 16 examples,-p 4 react-16 webpackstill sets 4 jobs, and an unknown name fails with the list of valid names.git merge-tree).🤖 Generated with Claude Code