Version
v26.10.0
Platform
Subsystem
test_runner
What steps will reproduce the bug?
repro.test.mjs:
import { test } from 'node:test';
test('registered first', () => {});
null.x; // throws while the file loads
test('never registered', () => {});
node --test --test-isolation=none repro.test.mjs; echo "exit $?"
How often does it reproduce? Is there a required condition?
Every time. The file must register at least one test before it throws. If the throw comes before any test() call, the run fails as expected.
What is the expected behavior? Why is that the expected behavior?
The file fails to load, so the run should fail, as it does with the default process isolation:
$ node --test repro.test.mjs
TypeError: Cannot read properties of null (reading 'x')
ℹ tests 1
ℹ pass 0
ℹ fail 1
What do you see instead?
The error is never printed, the rest of the file is silently skipped, and the run passes:
$ node --test --test-isolation=none repro.test.mjs; echo "exit $?"
✔ registered first (0.212625ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 4.442542
exit 0
Additional information
Found while running a mutation tester (Stryker) with --test-isolation=none to speed it up: a mutant that made a rule throw on one input survived, because the table-driven test file that computes its cases at load time stopped partway through and the run still exited 0. With several files, the other files' tests run and the broken file's later tests just disappear from the count (129 of 198 in that case), so nothing looks wrong unless you compare totals.
Possibly related: #64833 (--test-force-exit with concurrency loses verdicts and exits 0).
Version
v26.10.0
Platform
Subsystem
test_runner
What steps will reproduce the bug?
repro.test.mjs:How often does it reproduce? Is there a required condition?
Every time. The file must register at least one test before it throws. If the throw comes before any
test()call, the run fails as expected.What is the expected behavior? Why is that the expected behavior?
The file fails to load, so the run should fail, as it does with the default process isolation:
What do you see instead?
The error is never printed, the rest of the file is silently skipped, and the run passes:
Additional information
Found while running a mutation tester (Stryker) with
--test-isolation=noneto speed it up: a mutant that made a rule throw on one input survived, because the table-driven test file that computes its cases at load time stopped partway through and the run still exited 0. With several files, the other files' tests run and the broken file's later tests just disappear from the count (129 of 198 in that case), so nothing looks wrong unless you compare totals.Possibly related: #64833 (
--test-force-exitwith concurrency loses verdicts and exits 0).