Skip to content

feat: programmatic API with getSuites(), executeSuite(), executeTest() #5518

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
DavertMik wants to merge 3 commits into
base: 4.x
Choose a base branch
from
Open

feat: programmatic API with getSuites(), executeSuite(), executeTest() #5518

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c8a7be8
feat: add programmatic API with getSuites(), executeSuite(), executeT…
Apr 12, 2026
77fc33b
refactor: extract Runner class from Codecept for all Mocha interactions
Apr 12, 2026
cd81b57
refactor: clean up workers.js — remove dead code, extract Config.appl…
Apr 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
189 changes: 157 additions & 32 deletions docs/internal-api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -225,42 +225,167 @@ Step events provide step objects with following fields:

Whenever you execute tests with `--verbose` option you will see registered events and promises executed by a recorder.

## Custom Runner
## Programmatic API

You can run CodeceptJS tests from your script.
CodeceptJS can be imported and used programmatically from your scripts. The main entry point is the `Codecept` class, which provides methods to list and execute tests.

### Setup

```js
const { codecept: Codecept } = require('codeceptjs');

// define main config
const config = {
helpers: {
WebDriver: {
browser: 'chrome',
url: 'http://localhost'
}
}
import { Codecept, container } from 'codeceptjs';

const config = {
helpers: {
Playwright: { browser: 'chromium', url: 'http://localhost' }
},
tests: './*_test.js',
};

const opts = { steps: true };

// run CodeceptJS inside async function
(async () => {
const codecept = new Codecept(config, options);
codecept.init(__dirname);

try {
await codecept.bootstrap();
codecept.loadTests('**_test.js');
// run tests
await codecept.run(test);
} catch (err) {
printError(err);
process.exitCode = 1;
} finally {
await codecept.teardown();
}
})();
const codecept = new Codecept(config, { steps: true });
await codecept.init(__dirname);
```

### Listing Tests

Use `getSuites()` to get all parsed suites with their tests without executing them:

```js
const suites = codecept.getSuites();

for (const suite of suites) {
console.log(suite.title, suite.tags);
for (const test of suite.tests) {
console.log(' -', test.title, test.tags);
}
}
```

`getSuites()` accepts an optional glob pattern. If `loadTests()` hasn't been called yet, it will be called internally.

Each suite contains:

| Property | Type | Description |
|----------|------|-------------|
| `title` | `string` | Feature/suite title |
| `file` | `string` | Absolute path to the test file |
| `tags` | `string[]` | Tags (e.g. `@smoke`) |
| `tests` | `Array` | Tests in this suite |

Each test contains:

| Property | Type | Description |
|----------|------|-------------|
| `title` | `string` | Scenario title |
| `uid` | `string` | Unique test identifier |
| `tags` | `string[]` | Tags from scenario and suite |
| `fullTitle` | `string` | `"Suite: Test"` format |

### Executing Suites

Use `runSuite()` to run all tests within a suite:

```js
await codecept.bootstrap();

const suites = codecept.getSuites();
for (const suite of suites) {
await codecept.runSuite(suite);
}

const result = container.result();
console.log(result.stats);
console.log(`Passed: ${result.passedTests.length}`);
console.log(`Failed: ${result.failedTests.length}`);

await codecept.teardown();
```

> Also, you can run tests inside workers in a custom scripts. Please refer to the [parallel execution](/parallel) guide for more details.
### Executing Individual Tests

Use `runTest()` to run a single test:

```js
await codecept.bootstrap();

const suites = codecept.getSuites();
for (const test of suites[0].tests) {
await codecept.runTest(test);
}

const result = container.result();
await codecept.teardown();
```

### Result Object

The `Result` object returned by `container.result()` provides:

| Property | Type | Description |
|----------|------|-------------|
| `stats` | `object` | `{ passes, failures, tests, pending, failedHooks, duration }` |
| `tests` | `Test[]` | All collected tests |
| `passedTests` | `Test[]` | Tests that passed |
| `failedTests` | `Test[]` | Tests that failed |
| `skippedTests` | `Test[]` | Tests that were skipped |
| `hasFailed` | `boolean` | Whether any test failed |
| `duration` | `number` | Total duration in milliseconds |

### Full Lifecycle (Low-Level)

For full control, you can orchestrate the lifecycle manually:

```js
const codecept = new Codecept(config, opts);
await codecept.init(__dirname);

try {
await codecept.bootstrap();
codecept.loadTests('**_test.js');
await codecept.run();
} catch (err) {
console.error(err);
process.exitCode = 1;
} finally {
await codecept.teardown();
}
```

### Runner

All Mocha interactions are handled by the `Runner` class (`lib/runner.js`). After calling `init()`, it's available as `codecept.runner`:

```js
const runner = codecept.runner;

// Same as codecept.getSuites() / runSuite() / runTest()
const suites = runner.getSuites();
await runner.runSuite(suites[0]);
await runner.runTest(suites[0].tests[0]);
```

The `Codecept` methods (`getSuites`, `runSuite`, `runTest`, `run`) delegate to the Runner.

### Codecept Methods Reference

| Method | Description |
|--------|-------------|
| `new Codecept(config, opts)` | Create codecept instance |
| `await init(dir)` | Initialize globals, container, helpers, plugins, runner |
| `loadTests(pattern?)` | Find test files by glob pattern |
| `getSuites(pattern?)` | Load and return parsed suites with tests |
| `await bootstrap()` | Execute bootstrap hook |
| `await run(test?)` | Run all loaded tests (or filter by file path) |
| `await runSuite(suite)` | Run a specific suite from `getSuites()` |
| `await runTest(test)` | Run a specific test from `getSuites()` |
| `await teardown()` | Execute teardown hook |

### Runner Methods Reference

| Method | Description |
|--------|-------------|
| `getSuites(pattern?)` | Parse suites using a temporary Mocha instance |
| `run(test?)` | Run tests, optionally filtering by file path |
| `runSuite(suite)` | Run all tests in a suite |
| `runTest(test)` | Run a single test by fullTitle |

> Also, you can run tests inside workers in a custom script. Please refer to the [parallel execution](/parallel) guide for more details.
99 changes: 37 additions & 62 deletions lib/codecept.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,13 @@ const __filename = fileURLToPath(import.meta.url)
const __dirname = dirname(__filename)

import Helper from '@codeceptjs/helper'
import Runner from './runner.js'
import container from './container.js'
import Config from './config.js'
import event from './event.js'
import runHook from './hooks.js'
import ActorFactory from './actor.js'
import output from './output.js'
import { emptyFolder } from './utils.js'
import { initCodeceptGlobals } from './globals.js'
import { validateTypeScriptSetup, getTSNodeESMWarning } from './utils/loaderCheck.js'
import recorder from './recorder.js'
import store from './store.js'

import storeListener from './listener/store.js'
Expand Down Expand Up @@ -103,6 +100,7 @@ class Codecept {
await this.requireModules(this.requiringModules)
// initializing listeners
await container.create(this.config, this.opts)
this.runner = new Runner(this)
await this.runHooks()
}

Expand Down Expand Up @@ -256,71 +254,48 @@ class Codecept {
return testFiles.slice(startIndex, endIndex)
}

/**
* Returns parsed suites with their tests.
* Creates a temporary Mocha instance to avoid polluting container state.
* Must be called after init(). Calls loadTests() internally if testFiles is empty.
*
* @param {string} [pattern] - glob pattern for test files
* @returns {Array<{title: string, file: string, tags: string[], tests: Array<{title: string, uid: string, tags: string[], fullTitle: string}>}>}
*/
getSuites(pattern) {
return this.runner.getSuites(pattern)
}

/**
* Run all tests in a suite.
* Must be called after init() and bootstrap().
*
* @param {{file: string}} suite - suite object returned by getSuites()
* @returns {Promise<void>}
*/
async runSuite(suite) {
return this.runner.runSuite(suite)
}

/**
* Run a single test by its fullTitle.
* Must be called after init() and bootstrap().
*
* @param {{fullTitle: string}} test - test object returned by getSuites()
* @returns {Promise<void>}
*/
async runTest(test) {
return this.runner.runTest(test)
}

/**
* Run a specific test or all loaded tests.
*
* @param {string} [test]
* @returns {Promise<void>}
*/
async run(test) {
await container.started()

// Check TypeScript loader configuration before running tests
const tsValidation = validateTypeScriptSetup(this.testFiles, this.requiringModules || [])
if (tsValidation.hasError) {
output.error(tsValidation.message)
process.exit(1)
}

// Show warning if ts-node/esm is being used
const tsWarning = getTSNodeESMWarning(this.requiringModules || [])
if (tsWarning) {
output.print(output.colors.yellow(tsWarning))
}

// Ensure translations are loaded for Gherkin features
try {
const { loadTranslations } = await import('./mocha/gherkin.js')
await loadTranslations()
} catch (e) {
// Ignore if gherkin module not available
}

return new Promise((resolve, reject) => {
const mocha = container.mocha()
mocha.files = this.testFiles

if (test) {
if (!fsPath.isAbsolute(test)) {
test = fsPath.join(store.codeceptDir, test)
}
const testBasename = fsPath.basename(test, '.js')
const testFeatureBasename = fsPath.basename(test, '.feature')
mocha.files = mocha.files.filter(t => {
return fsPath.basename(t, '.js') === testBasename || fsPath.basename(t, '.feature') === testFeatureBasename || t === test
})
}

const done = async (failures) => {
event.emit(event.all.result, container.result())
event.emit(event.all.after, this)
// Wait for any recorder tasks added by event.all.after handlers
await recorder.promise()
// Set exit code based on test failures
if (failures) {
process.exitCode = 1
}
resolve()
}

try {
event.emit(event.all.before, this)
mocha.run(async (failures) => await done(failures))
} catch (e) {
output.error(e.stack)
reject(e)
}
})
return this.runner.run(test)
}

/**
Expand Down
13 changes: 4 additions & 9 deletions lib/command/check.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,15 +70,10 @@ export default async function (options) {
if (codecept) {
try {
codecept.loadTests()
const files = codecept.testFiles
const mocha = Container.mocha()
mocha.files = files
mocha.loadFiles()

for (const suite of mocha.suite.suites) {
if (suite && suite.tests) {
numTests += suite.tests.length
}
const suites = codecept.getSuites()

for (const suite of suites) {
numTests += suite.tests.length
}

if (numTests > 0) {
Expand Down
Loading