diff --git a/docs/06-concepts/02-endpoints-and-apis/02-sessions.md b/docs/06-concepts/02-endpoints-and-apis/02-sessions.md index 23332f72..5823abde 100644 --- a/docs/06-concepts/02-endpoints-and-apis/02-sessions.md +++ b/docs/06-concepts/02-endpoints-and-apis/02-sessions.md @@ -141,7 +141,7 @@ session.log('Something looks off', level: LogLevel.warning); ## Test with sessions -The test tools provide a `sessionBuilder` for calling endpoints in tests and simulating session state such as authentication. See [Get started with testing](../testing/get-started) and [the basics](../testing/the-basics) for the patterns. +The test tools provide a `sessionBuilder` for calling endpoints in tests and simulating session state such as authentication. See [Get started with testing](../testing/get-started) and [Writing tests](../testing/writing-tests) for the patterns. ## Related diff --git a/docs/06-concepts/03-data-and-the-database/02-database/10-transactions.md b/docs/06-concepts/03-data-and-the-database/02-database/10-transactions.md index cda4a45b..d6c72777 100644 --- a/docs/06-concepts/03-data-and-the-database/02-database/10-transactions.md +++ b/docs/06-concepts/03-data-and-the-database/02-database/10-transactions.md @@ -26,7 +26,7 @@ var result = await session.db.transaction((transaction) async { In the example we insert a company and an employee in the same transaction. If any of the operations fail, the entire transaction will be rolled back and no changes will be made to the database. If the transaction is successful, the return value will be `true`. :::tip -The Serverpod test tools use this rollback behavior to isolate test cases: by default, each test runs inside a transaction that is rolled back when the test ends. See [rollback configuration](../../testing/the-basics#rollback-database-configuration) in the testing docs. +The Serverpod test tools use this rollback behavior to isolate test cases: by default, each test runs inside a transaction that is rolled back when the test ends. See [rollback configuration](../../testing/configuration#rollbackdatabase) in the testing docs. ::: ## Cancel a transaction diff --git a/docs/06-concepts/08-testing/01-get-started.md b/docs/06-concepts/08-testing/01-get-started.md index 6b17e8c9..4c35f15a 100644 --- a/docs/06-concepts/08-testing/01-get-started.md +++ b/docs/06-concepts/08-testing/01-get-started.md @@ -1,229 +1,63 @@ --- sidebar_label: Get started -description: Integration testing in Serverpod uses the withServerpod helper to call endpoints directly, seed the database, and simulate authenticated sessions without running a separate server. +description: The Serverpod test tools call your endpoints like ordinary Dart functions, running a real server and a real database that resets between tests. --- # Get started with testing -Serverpod's test tools let you call endpoints directly, seed the database, and simulate authenticated sessions, all without spinning up a separate server. +Serverpod generates test tools that let you call your endpoints from a test the same way your app calls them, as ordinary Dart functions. Behind that call the tools run a real server against a real database, so a test exercises your endpoint, your models, and your queries together. - -
-Project created with Serverpod 2.1 or earlier? Complete these steps first. -

-If your project was created with Serverpod 2.1 or earlier, the test infrastructure is not yet set up. Complete these steps before continuing: -1. Add the `server_test_tools_path` key with the value `test/integration/test_tools` to `config/generator.yaml`: +That means no separate server to start, no HTTP client to write, and no mocked database. It also means these are integration tests: they check your code end to end rather than one class in isolation. Plain unit tests still work as they always do, and they need none of this. -```yaml -server_test_tools_path: test/integration/test_tools -``` - - Without this key, the test tools file is not generated. With the above config the location of the test tools file is `test/integration/test_tools/serverpod_test_tools.dart`, but this can be set to any folder (though should be outside of `lib` as per Dart's test conventions). - -2. New projects now come with a test postgres and redis instance in `docker-compose.yaml`. This is not strictly mandatory, but is recommended to ensure that the testing state is never polluted. Add the snippet below to the `docker-compose.yaml` file in the server directory: - -```yaml -# Add to the existing services -postgres_test: - image: pgvector/pgvector:pg16 - ports: - - '9090:5432' - environment: - POSTGRES_USER: postgres - POSTGRES_DB: _test - POSTGRES_PASSWORD: "" - volumes: - - _test_data:/var/lib/postgresql/data -redis_test: - image: redis:6.2.6 - ports: - - '9091:6379' - command: redis-server --requirepass 'REDIS_TEST_PASSWORD' - environment: - - REDIS_REPLICATION_MODE=master -volumes: - # ... - _test_data: -``` - -

-Or copy the complete file here. -

- -```yaml -services: - # Development services - postgres: - image: pgvector/pgvector:pg16 - ports: - - '8090:5432' - environment: - POSTGRES_USER: postgres - POSTGRES_DB: - POSTGRES_PASSWORD: "" - volumes: - - _data:/var/lib/postgresql/data - redis: - image: redis:6.2.6 - ports: - - '8091:6379' - command: redis-server --requirepass "" - environment: - - REDIS_REPLICATION_MODE=master - - # Test services - postgres_test: - image: pgvector/pgvector:pg16 - ports: - - '9090:5432' - environment: - POSTGRES_USER: postgres - POSTGRES_DB: _test - POSTGRES_PASSWORD: "" - volumes: - - _test_data:/var/lib/postgresql/data - redis_test: - image: redis:6.2.6 - ports: - - '9091:6379' - command: redis-server --requirepass "" - environment: - - REDIS_REPLICATION_MODE=master - -volumes: - _data: - _test_data: -``` - -

-
-3. Create a `test.yaml` file and add it to the `config` directory: - -```yaml -# This is the configuration file for your test environment. -# All ports are set to zero in this file which makes the server find the next available port. -# This is needed to enable running tests concurrently. To set up your server, you will -# need to add the name of the database you are connecting to and the user name. -# The password for the database is stored in the config/passwords.yaml. - -# Configuration for the main API test server. -apiServer: - port: 0 - publicHost: localhost - publicPort: 0 - publicScheme: http - -# Configuration for the Insights test server. -insightsServer: - port: 0 - publicHost: localhost - publicPort: 0 - publicScheme: http - -# Configuration for the web test server. -webServer: - port: 0 - publicHost: localhost - publicPort: 0 - publicScheme: http - -# This is the database setup for your test server. -database: - host: localhost - port: 9090 - name: _test - user: postgres - dataPath: .serverpod/test/pgdata - -# This is the setup for your Redis test instance. -redis: - enabled: false - host: localhost - port: 9091 - -sessionLogs: - persistentEnabled: true - consoleEnabled: true -``` +Two things follow from that: -4. Add this entry to `config/passwords.yaml` - -```yaml -test: - database: '' - redis: '' -``` - -5. Add a `dart_test.yaml` file to the `server` directory (next to `pubspec.yaml`) with the following contents: - -```yaml -tags: - integration: {} - -``` +- **Tests stay independent.** Anything a test writes to the database is rolled back once that test finishes, so tests never see each other's data and you do not clean up after yourself. +- **Each test sets its own scene.** A session builder decides what the server state looks like for a call, such as who is signed in. -6. Finally, add the `test` and `serverpod_test` packages as dev dependencies in `pubspec.yaml`: +## Set up -```yaml -dev_dependencies: - serverpod_test: # Should be same version as the `serverpod` package - test: ^1.24.2 -``` +New projects are ready to test. If you have `test/integration/test_tools/serverpod_test_tools.dart` in your server package, skip to [writing a test](#write-a-test). -That's it, the project setup should be ready to start using the test tools! -

-
- +:::info +Projects created with Serverpod 2.1 or earlier need a few one-time setup steps first. See [Upgrade to 2.2](../../upgrading/archive/upgrade-to-two-point-two). +::: -Go to the server directory and generate the test tools: +The test tools are generated along with the rest of your code. With `serverpod start` running, saving a file regenerates them. Outside a session, run `serverpod generate` from your server directory. -```bash -serverpod generate -``` +Either way you get `test/integration/test_tools/serverpod_test_tools.dart`. The `test/integration` folder keeps these apart from your unit tests, which makes a project easier to navigate. To generate the file somewhere else, change `server_test_tools_path` in `config/generator.yaml`. -The default location for the generated file is `test/integration/test_tools/serverpod_test_tools.dart`. The folder name `test/integration` is chosen to differentiate from unit tests (see the [best practices section](./best-practises#unit-and-integration-tests) for more information on this). +## Write a test -The generated file exports a `withServerpod` helper that enables you to call your endpoints directly like regular functions: +Import the generated file rather than the `serverpod_test` package: it carries the helpers and your endpoints together. ```dart import 'package:test/test.dart'; -// Import the generated file, it contains everything you need. import 'test_tools/serverpod_test_tools.dart'; void main() { - withServerpod('Given Example endpoint', (sessionBuilder, endpoints) { - test('when calling `hello` then should return greeting', () async { - final greeting = await endpoints.example.hello(sessionBuilder, 'Michael'); - expect(greeting, 'Hello Michael'); + withServerpod('Given Greeting endpoint', (sessionBuilder, endpoints) { + test('when calling `hello` then it returns a greeting', () async { + final greeting = await endpoints.greeting.hello(sessionBuilder, 'Bob'); + expect(greeting.message, 'Hello Bob'); }); }); } ``` -A few things to note from the above example: +The `withServerpod` callback gives you the two things every test uses. The `sessionBuilder` describes the server state a call runs under, and `endpoints` holds your endpoints. -- The test tools should be imported from the generated test tools file and not the `serverpod_test` package. -- The `withServerpod` callback takes two parameters: `sessionBuilder` and `endpoints`. - - `sessionBuilder` is used to build a `session` object that represents the server state during an endpoint call and is used to set up scenarios. - - `endpoints` contains all your Serverpod endpoints and lets you call them. - -:::tip - -The location of the test tools can be changed by changing the `server_test_tools_path` key in `config/generator.yaml`. If you remove the `server_test_tools_path` key, the test tools will stop being generated. - -::: - -Before the test can be run the Postgres and Redis also have to be started: +Run it: ```bash -docker compose up --build --detach +dart test ``` -Now the test is ready to be run: +New projects set `dataPath` under `database` in `config/test.yaml`, which makes the test server start and manage its own PostgreSQL. There is nothing to launch first. If your `test.yaml` has no `dataPath`, the server connects to the database that file points at, and that one has to be running. -```bash -dart test -``` +## Next -Happy testing! +- [Writing tests](writing-tests): the session builder, authentication, and seeding data. +- [Configuration](configuration): the options `withServerpod` accepts. +- [Advanced examples](advanced-examples): streams, future calls, and business logic. +- [Best practices](best-practices): conventions worth following. diff --git a/docs/06-concepts/08-testing/02-the-basics.md b/docs/06-concepts/08-testing/02-the-basics.md deleted file mode 100644 index cef7a338..00000000 --- a/docs/06-concepts/08-testing/02-the-basics.md +++ /dev/null @@ -1,271 +0,0 @@ ---- -description: The Serverpod test tools basics, set up scenarios with sessionBuilder, seed the database, control rollback behavior, and simulate authenticated and unauthenticated sessions. ---- - -# The basics - -This page covers the core building blocks of the test tools: setting up a session, seeding the database, controlling rollback, and simulating authenticated and unauthenticated sessions. - -## Set up a test scenario - -The `withServerpod` helper provides a `sessionBuilder` that helps with setting up different scenarios for tests. To modify the session builder's properties, call its `copyWith` method. It takes the following named parameters: - -|Property|Description|Default| -|:---|:---|:---:| -|`authentication`|See section [Setting authenticated state](#setting-authenticated-state).|`AuthenticationOverride.unauthenticated()`| -|`enableLogging`|Whether logging is turned on for the session.|`false`| - -The `copyWith` method creates a new unique session builder with the provided properties. This can then be used in endpoint calls (see section [Setting authenticated state](#setting-authenticated-state) for an example). - -To build out a `Session` (to use for [database calls](#seeding-the-database) or [pass on to functions](./advanced-examples#test-business-logic-that-depends-on-session)), call the `build` method: - -```dart -Session session = sessionBuilder.build(); -``` - -Given the properties set on the session builder through the `copyWith` method, this returns a Serverpod `Session` that has the corresponding state. - -### Setting authenticated state - -To control the authenticated state of the session, the `AuthenticationOverride` class can be used. - -To create an unauthenticated override (this is the default value for new sessions), call `AuthenticationOverride unauthenticated()`: - -```dart -static AuthenticationOverride unauthenticated(); -``` - -To create an authenticated override, call `AuthenticationOverride.authenticationInfo(...)`: - -```dart -static AuthenticationOverride authenticationInfo( - String userIdentifier, - Set scopes, { - String? authId, -}) -``` - -Pass these to `sessionBuilder.copyWith` to simulate different scenarios. Below follows an example for each case: - -```dart -withServerpod('Given AuthenticatedExample endpoint', (sessionBuilder, endpoints) { - // Corresponds to an actual user id - final userId = '550e8400-e29b-41d4-a716-446655440000'; - - group('when authenticated', () { - var authenticatedSessionBuilder = sessionBuilder.copyWith( - authentication: - AuthenticationOverride.authenticationInfo(userId, {Scope('user')}), - ); - - test('then calling `hello` should return greeting', () async { - final greeting = await endpoints.authenticatedExample - .hello(authenticatedSessionBuilder, 'Michael'); - expect(greeting, 'Hello, Michael!'); - }); - }); - - group('when unauthenticated', () { - var unauthenticatedSessionBuilder = sessionBuilder.copyWith( - authentication: AuthenticationOverride.unauthenticated(), - ); - - test( - 'then calling `hello` should throw `ServerpodUnauthenticatedException`', - () async { - final future = endpoints.authenticatedExample - .hello(unauthenticatedSessionBuilder, 'Michael'); - await expectLater( - future, throwsA(isA())); - }); - }); -}); -``` - -### Seeding the database - -To seed the database before tests, `build` a `session` and pass it to the database call just as in production code. - -:::info - -By default `withServerpod` does all database operations inside a transaction that is rolled back after each `test` case. See the [rollback database configuration](#rollback-database-configuration) for how to configure this behavior. - -::: - -```dart -withServerpod('Given Products endpoint', (sessionBuilder, endpoints) { - var session = sessionBuilder.build(); - - setUp(() async { - await Product.db.insert(session, [ - Product(name: 'Apple', price: 10), - Product(name: 'Banana', price: 10) - ]); - }); - - test('then calling `all` should return all products', () async { - final products = await endpoints.products.all(sessionBuilder); - expect(products, hasLength(2)); - expect(products.map((p) => p.name), containsAll(['Apple', 'Banana'])); - }); -}); -``` - -## Environment - -By default `withServerpod` uses the `test` run mode and the database settings will be read from `config/test.yaml`. - -It is possible to override the default run mode by setting the `runMode` setting: - -```dart -withServerpod( - 'Given Products endpoint', - (sessionBuilder, endpoints) { - /* test code */ - }, - runMode: ServerpodRunMode.development, -); -``` - -## Configuration - -The following optional configuration options are available to pass as named arguments to `withServerpod`: - -|Property|Description|Default| -|:-----|:-----|:---:| -|`applyMigrations`|Whether pending migrations should be applied when starting Serverpod.|`true`| -|`configOverride`|A function that overrides values in the loaded Serverpod config before the test server starts.|`null`| -|`enableSessionLogging`|Whether session logging should be enabled.|`false`| -|`experimentalFeatures`|Optionally specify experimental features used by Serverpod.|`null`| -|`rollbackDatabase`|Options for when to rollback the database during the test lifecycle (or disable it). See detailed description [here](#rollback-database-configuration).|`RollbackDatabase.afterEach`| -|`runMode`|The run mode that Serverpod should be running in.|`test`| -|`runtimeParametersBuilder`|A callback for providing runtime parameters to Serverpod before startup.|`null`| -|`serverpodLoggingMode`|The logging mode used when creating Serverpod.|`ServerpodLoggingMode.normal`| -|`serverpodStartTimeout`|The timeout to use when starting Serverpod, which connects to the database among other things. Defaults to `Duration(seconds: 30)`.|`Duration(seconds: 30)`| -|`testServerOutputMode`|Options for controlling test server output during test execution.|`TestServerOutputMode.normal`| -|`testGroupTagsOverride`|By default Serverpod test tools tags the `withServerpod` test group with `"integration"`. This is to provide a simple way to only run unit or integration tests. This property allows this tag to be overridden to something else. Defaults to `['integration']`.|`['integration']`| - -### `rollbackDatabase` {#rollback-database-configuration} - -By default `withServerpod` does all database operations inside a transaction that is rolled back after each `test` case. Just like the following enum describes, the behavior of the automatic rollbacks can be configured: - -```dart -/// Options for when to rollback the database during the test lifecycle. -enum RollbackDatabase { - /// After each test. This is the default. - afterEach, - - /// After all tests. - afterAll, - - /// Disable rolling back the database. - disabled, -} -``` - -There are a few reasons to change the default setting: - -1. **Scenario tests**: when consecutive `test` cases depend on each other. While generally considered an anti-pattern, it can be useful when the set up for the test group is very expensive. In this case `rollbackDatabase` can be set to `RollbackDatabase.afterAll` to ensure that the database state persists between `test` cases. At the end of the `withServerpod` scope, all database changes will be rolled back. - -2. **Concurrent transactions in endpoints**: when concurrent calls are made to `session.db.transaction` inside an endpoint, it is no longer possible for the Serverpod test tools to do these operations as part of a top level transaction. In this case this feature should be disabled by passing `RollbackDatabase.disabled`. - -```dart -Future concurrentTransactionCalls( - Session session, -) async { - await Future.wait([ - session.db.transaction((tx) => /*...*/), - // Will throw `InvalidConfigurationException` if `rollbackDatabase` - // is not set to `RollbackDatabase.disabled` in `withServerpod` - session.db.transaction((tx) => /*...*/), - ]); -} -``` - -When setting `rollbackDatabase.disabled` to be able to test `concurrentTransactionCalls`, remember that the database has to be manually cleaned up to not leak data: - -```dart -withServerpod( - 'Given ProductsEndpoint when calling concurrentTransactionCalls', - (sessionBuilder, endpoints) { - tearDownAll(() async { - var session = sessionBuilder.build(); - // If something was saved to the database in the endpoint, - // for example a `Product`, then it has to be cleaned up! - await Product.db.deleteWhere( - session, - where: (_) => Constant.bool(true), - ); - }); - - test('then should execute and commit all transactions', () async { - var result = - await endpoints.products.concurrentTransactionCalls(sessionBuilder); - // ... - }); - }, - rollbackDatabase: RollbackDatabase.disabled, -); -``` - -Additionally, when setting `rollbackDatabase.disabled`, it may also be needed to pass the `--concurrency=1` flag to the dart test runner. Otherwise multiple tests might pollute each others database state: - -```bash -dart test -t integration --concurrency=1 -``` - -For the other cases this is not an issue, as each `withServerpod` has its own transaction and will therefore be isolated. - - -3. **Database exceptions that are quelled**: There is a specific edge case where the test tools behavior deviates from production behavior. See example below: - -```dart -var transactionFuture = session.db.transaction((tx) async { - var data = UniqueData(number: 1, email: 'test@test.com'); - try { - await UniqueData.db.insertRow(session, data, transaction: tx); - await UniqueData.db.insertRow(session, data, transaction: tx); - } on DatabaseException catch (_) { - // Ignore the database exception - } -}); - -// ATTENTION: This will throw an exception in production -// but not in the test tools. -await transactionFuture; -``` - -In production, the transaction call will throw if any database exception happened during its execution, _even_ if the exception was first caught inside the transaction. However, in the test tools this will not throw an exception due to how the nested transactions are emulated. Quelling exceptions like this is not best practice, but if the code under test does this setting `rollbackDatabase` to `RollbackDatabase.disabled` will ensure the code behaves like in production. - - -## Test exceptions - -The following exceptions are exported from the generated test tools file and can be thrown by the test tools in various scenarios, see below. - -|Exception|Description| -|:-----|:-----| -|`ServerpodUnauthenticatedException`|Thrown during an endpoint method call when the user was not authenticated.| -|`ServerpodInsufficientAccessException`|Thrown during an endpoint method call when the authentication key provided did not have sufficient access.| -|`ConnectionClosedException`|Thrown during an endpoint method call if a stream connection was closed with an error. For example, if the user authentication was revoked.| -|`InvalidConfigurationException`|Thrown when an invalid configuration state is found.| - -## Test helpers - -### `flushEventQueue` - -Test helper to flush the event queue. -Useful for waiting for async events to complete before continuing the test. - -```dart -Future flushEventQueue(); -``` - -For example, if depending on a generator function to execute up to its `yield`, then the -event queue can be flushed to ensure the generator has executed up to that point: - -```dart -var stream = endpoints.someEndpoint.generatorFunction(session); -await flushEventQueue(); -``` - -See also [this complete example](./advanced-examples#multiple-users-interacting-with-a-shared-stream). diff --git a/docs/06-concepts/08-testing/02-writing-tests.md b/docs/06-concepts/08-testing/02-writing-tests.md new file mode 100644 index 00000000..01124fdc --- /dev/null +++ b/docs/06-concepts/08-testing/02-writing-tests.md @@ -0,0 +1,151 @@ +--- +description: The pieces every Serverpod test uses, the session builder, simulated authentication, seeding the database, and the exceptions the test tools throw. +--- + +# Writing tests + +Every test you write with the test tools uses the same few pieces. The `withServerpod` callback hands you two of them: + +- **`sessionBuilder`** describes the server state a call runs under, such as who is signed in. You pass it to endpoint calls, and you can build it into a `Session` when you need one directly. +- **`endpoints`** is your generated endpoints, called the same way your app calls them. + +```dart +withServerpod('Given Greeting endpoint', (sessionBuilder, endpoints) { + test('then hello returns a greeting', () async { + var greeting = await endpoints.greeting.hello(sessionBuilder, 'Bob'); + expect(greeting.message, 'Hello Bob'); + }); +}); +``` + +The generated file carries the test helpers and your endpoints. Serverpod's own types, such as `Session` and `Scope`, and your models come from their own packages: + +```dart +import 'package:serverpod/serverpod.dart'; +import 'package:test/test.dart'; +import 'package:my_project_server/src/generated/protocol.dart'; + +import 'test_tools/serverpod_test_tools.dart'; +``` + +## Set up a scenario + +Call `copyWith` on the session builder to change the state a call runs under. It returns a new builder, leaving the original untouched, so each scenario in a test file can have its own. + +| Property | What it does | If you omit it | +| --- | --- | --- | +| `authentication` | Who is signed in for calls made with this builder. | Inherits from the builder you called `copyWith` on. | +| `enableLogging` | Whether the session writes log entries. | Inherits from the builder you called `copyWith` on. | + +A fresh builder from `withServerpod` starts out unauthenticated, and logs only if you passed [`enableSessionLogging`](configuration#options). + +When you need a `Session` rather than a builder, for a database call or to pass into [your own business logic](advanced-examples#test-business-logic-that-depends-on-session), build one: + +```dart +Session session = sessionBuilder.build(); +``` + +## Simulate authentication + +The `AuthenticationOverride` class sets who the call runs as. It offers two: + +```dart +static AuthenticationOverride unauthenticated(); + +static AuthenticationOverride authenticationInfo( + String userIdentifier, + Set scopes, { + String? authId, +}); +``` + +The `userIdentifier` is the signed-in user's id, which is a UUID string when you use Serverpod's auth module. Scopes are the permissions the call should carry. See [Authentication](../authentication/basics) for both. + +Pass an override through `copyWith` to test each case: + +```dart +withServerpod('Given AuthenticatedExample endpoint', (sessionBuilder, endpoints) { + final userId = '550e8400-e29b-41d4-a716-446655440000'; + + group('when authenticated', () { + var authenticated = sessionBuilder.copyWith( + authentication: + AuthenticationOverride.authenticationInfo(userId, {Scope('user')}), + ); + + test('then calling `hello` returns a greeting', () async { + final greeting = + await endpoints.authenticatedExample.hello(authenticated, 'Michael'); + expect(greeting, 'Hello, Michael!'); + }); + }); + + group('when unauthenticated', () { + var unauthenticated = sessionBuilder.copyWith( + authentication: AuthenticationOverride.unauthenticated(), + ); + + test('then calling `hello` throws', () async { + final future = + endpoints.authenticatedExample.hello(unauthenticated, 'Michael'); + await expectLater( + future, throwsA(isA())); + }); + }); +}); +``` + +## Seed the database + +Build a session and use it for database calls exactly as your server code would: + +```dart +withServerpod('Given Products endpoint', (sessionBuilder, endpoints) { + var session = sessionBuilder.build(); + + setUp(() async { + await Product.db.insert(session, [ + Product(name: 'Apple', price: 10), + Product(name: 'Banana', price: 10), + ]); + }); + + test('then calling `all` returns all products', () async { + final products = await endpoints.products.all(sessionBuilder); + expect(products, hasLength(2)); + expect(products.map((p) => p.name), containsAll(['Apple', 'Banana'])); + }); +}); +``` + +Everything a test writes to the database is rolled back when the test ends, so each test starts from the same state and you do not have to clean up after yourself. You can change when that happens, or turn it off, with [`rollbackDatabase`](configuration#rollbackdatabase). + +## Exceptions the test tools throw + +The generated file exports these, so you can name them in a test without importing anything else. + +| Exception | Thrown when | +| --- | --- | +| `ServerpodUnauthenticatedException` | An endpoint call was made without authentication and the endpoint requires it. | +| `ServerpodInsufficientAccessException` | The authentication used did not carry the scopes the endpoint requires. | +| `ConnectionClosedException` | A stream connection closed with an error, for example because authentication was revoked. | +| `InvalidConfigurationException` | The endpoint ran database calls or transactions in parallel, which the automatic rollback cannot wrap. See [`rollbackDatabase`](configuration#rollbackdatabase). | + +The test server also throws when it fails to start or takes longer than the start timeout. That error type is not exported, so you cannot catch it by name, but the message says which of the two happened. + +## Wait for async work + +The `flushEventQueue` helper lets already-queued async work run before the test carries on. Reach for it when a call has to get as far as its first `yield` before you assert on it: + +```dart +var stream = endpoints.someEndpoint.generatorFunction(sessionBuilder); +await flushEventQueue(); +``` + +For a complete case, see [multiple users interacting with a shared stream](advanced-examples#multiple-users-interacting-with-a-shared-stream). + +## Related + +- [Configuration](configuration): every option `withServerpod` accepts. +- [Advanced examples](advanced-examples): streams, future calls, and business logic. +- [Sessions](../endpoints-and-apis/sessions): what a session carries in production. diff --git a/docs/06-concepts/08-testing/03-configuration.md b/docs/06-concepts/08-testing/03-configuration.md new file mode 100644 index 00000000..cf46f507 --- /dev/null +++ b/docs/06-concepts/08-testing/03-configuration.md @@ -0,0 +1,151 @@ +--- +description: Every option withServerpod accepts, including run mode, database rollback behavior, server output, timeouts, and test tags. +--- + +# Configuration + +The `withServerpod` helper takes optional named arguments that control how the test server starts and how tests are isolated from each other. + +Some of the values below, such as `ServerpodRunMode` and `Constant`, come from `package:serverpod/serverpod.dart` rather than the generated test tools, so import it when you use them. The `serverDirectory` option takes a `Directory` from `dart:io`. + +```dart +withServerpod( + 'Given Products endpoint', + (sessionBuilder, endpoints) { + /* test code */ + }, + rollbackDatabase: RollbackDatabase.afterAll, +); +``` + +## Options + +| Option | What it does | Default | +| --- | --- | --- | +| `applyMigrations` | Whether pending migrations are applied when the test server starts. | `true` | +| `configOverride` | A function that changes values in the loaded config before the server starts. | `null` | +| `enableSessionLogging` | Whether session logging is on. | `false` | +| `experimentalFeatures` | Experimental features to enable, such as [diagnostic event handlers](../operations/exception-monitoring). | `null` | +| `rollbackDatabase` | When database changes are rolled back. See [below](#rollbackdatabase). | `RollbackDatabase.afterEach` | +| `runMode` | The run mode the server starts in. | `test` | +| `runtimeParametersBuilder` | Runtime parameters to apply before startup. See [runtime parameters](../data-and-the-database/database/runtime-parameters). | `null` | +| `serverDirectory` | The directory that `config/.yaml`, `config/passwords.yaml`, and `migrations/` are resolved against. | The directory the test process runs in | +| `serverpodLoggingMode` | The logging mode used when creating Serverpod. | `ServerpodLoggingMode.normal` | +| `serverpodStartTimeout` | How long to wait for the server to start. | 30 seconds | +| `testGroupTagsOverride` | The tags applied to the generated test group. See [below](#test-tags). | `['integration']` | +| `testServerOutputMode` | How much server output reaches your terminal. See [below](#server-output). | `TestServerOutputMode.normal` | + +:::info +In a project without a database, `applyMigrations`, `rollbackDatabase`, and `runtimeParametersBuilder` are not generated at all, so passing one is a compile error. Rollback is off in that case and there is nothing to migrate. +::: + +Set `serverDirectory` when tests run from somewhere other than the server package, such as the workspace root. Without it the server resolves those paths against the current directory, misses your config, and falls over later on whatever it needed from it. + +## Run mode + +Tests run in the `test` run mode, so the server reads `config/test.yaml`. Point them at a different mode when you need its configuration instead: + +```dart +withServerpod( + 'Given Products endpoint', + (sessionBuilder, endpoints) { + /* test code */ + }, + runMode: ServerpodRunMode.development, +); +``` + +## Database rollback {#rollbackdatabase} + +By default each test runs inside a transaction that is rolled back when that test ends, so tests cannot see each other's data. Three values control this: + +| Value | Rolls back | +| --- | --- | +| `RollbackDatabase.afterEach` | After every test. The default. | +| `RollbackDatabase.afterAll` | Once the whole `withServerpod` group finishes. | +| `RollbackDatabase.disabled` | Never. You clean up yourself. | + +Three situations call for changing it. + +**Tests that build on each other.** When one test depends on what the previous one left behind, `afterAll` keeps state across the group and still rolls everything back at the end. Tests that share state are usually worth avoiding, but this helps when the setup is expensive. + +**Concurrent database work in an endpoint.** The rollback wraps each test in one transaction, and concurrent calls cannot be nested inside it. An endpoint that runs database calls or transactions in parallel throws `InvalidConfigurationException` unless you pass `RollbackDatabase.disabled`: + +```dart +Future concurrentTransactionCalls(Session session) async { + await Future.wait([ + session.db.transaction((tx) => /*...*/), + // Throws InvalidConfigurationException unless rollbackDatabase + // is RollbackDatabase.disabled. + session.db.transaction((tx) => /*...*/), + ]); +} +``` + +With rollback disabled you clean up yourself, or later tests inherit the data: + +```dart +withServerpod( + 'Given ProductsEndpoint when calling concurrentTransactionCalls', + (sessionBuilder, endpoints) { + tearDownAll(() async { + var session = sessionBuilder.build(); + await Product.db.deleteWhere(session, where: (_) => Constant.bool(true)); + }); + + test('then it commits all transactions', () async { + var result = + await endpoints.products.concurrentTransactionCalls(sessionBuilder); + // ... + }); + }, + rollbackDatabase: RollbackDatabase.disabled, +); +``` + +Tests that share a database this way also need to run one at a time, since the test runner runs files in parallel by default: + +```bash +dart test -t integration --concurrency=1 +``` + +This applies only when rollback is disabled. With rollback on, each `withServerpod` group has its own transaction and is already isolated from the others. + +**Caught database exceptions.** This is the one case where the test tools behave differently from production: + +```dart +var transactionFuture = session.db.transaction((tx) async { + var data = UniqueData(number: 1, email: 'test@test.com'); + try { + await UniqueData.db.insertRow(session, data, transaction: tx); + await UniqueData.db.insertRow(session, data, transaction: tx); + } on DatabaseException catch (_) { + // Ignored. + } +}); + +// Throws in production, but not under the test tools. +await transactionFuture; +``` + +In production a transaction fails if any database exception happened inside it, even one your code caught. The test tools emulate nested transactions and do not, so code that swallows database exceptions passes a test it would fail in production. Set `RollbackDatabase.disabled` to get production behavior. + +## Server output + +The `testServerOutputMode` option controls what the test server prints, which is the setting to reach for when you cannot see why a server failed: + +| Value | Shows | +| --- | --- | +| `TestServerOutputMode.normal` | Errors only. The default. | +| `TestServerOutputMode.verbose` | Everything the server prints. | +| `TestServerOutputMode.silent` | Nothing. | + +## Test tags + +Test groups created by `withServerpod` are tagged `integration`, which is what lets you run integration and unit tests separately. Change the tag with `testGroupTagsOverride`. See [running tests separately](advanced-examples#run-unit-and-integration-tests-separately). + +## Related + +- [Writing tests](writing-tests): the pieces every test uses. +- [Advanced examples](advanced-examples): the options above put to work. +- [Server configuration](../server-fundamentals/configuration): the config files the test server reads. diff --git a/docs/06-concepts/08-testing/03-advanced-examples.md b/docs/06-concepts/08-testing/04-advanced-examples.md similarity index 71% rename from docs/06-concepts/08-testing/03-advanced-examples.md rename to docs/06-concepts/08-testing/04-advanced-examples.md index 6ed74d7b..78521493 100644 --- a/docs/06-concepts/08-testing/03-advanced-examples.md +++ b/docs/06-concepts/08-testing/04-advanced-examples.md @@ -1,10 +1,10 @@ --- -description: Advanced Serverpod testing examples, run integration and unit tests separately, test business logic with sessions, test multi-user stream interactions, and manage database connections. +description: Advanced Serverpod testing examples, separate unit and integration runs, test business logic, shared streams, future calls, and exception monitoring. --- # Advanced examples -These examples build on [the basics](./the-basics) and cover less common testing needs: separating unit and integration tests, testing business logic directly, multi-user stream interactions, and managing database connections. +These examples build on [Writing tests](writing-tests) and cover less common needs: separating unit and integration tests, testing business logic directly, multi-user stream interactions, future calls, connection limits, and exception monitoring. ## Run unit and integration tests separately @@ -21,7 +21,7 @@ dart test -t integration dart test -x integration ``` -To change the name of this tag, see the [`testGroupTagsOverride`](./the-basics#configuration) configuration option. +To change the name of this tag, see the [`testGroupTagsOverride`](configuration#test-tags) configuration option. ## Test business logic that depends on `Session` @@ -120,43 +120,44 @@ withServerpod('Given CommunicationExampleEndpoint', (sessionBuilder, endpoints) }); ``` -## Optimizing the number of database connections +## Run a future call -By default, Dart's test runner runs tests concurrently. The number of concurrent tests depends on the running hosts' available CPU cores. If the host has a lot of cores it could trigger a case where the number of connections to the database exceeds the maximum connections limit set for the database, which will cause tests to fail. +The generated test tools expose your [future calls](../scheduling/future-calls) alongside your endpoints, so a test can invoke one immediately instead of waiting for its scheduled time. -Each `withServerpod` call will lazily create its own Serverpod instance which will connect to the database. Specifically, the code that causes the Serverpod instance to be created is `sessionBuilder.build()`, which happens at the latest in an endpoint call if not called by the test before. - -If a test needs a session before the endpoint call (e.g. to seed the database), `sessionBuilder.build()` has to be called which then triggers a database connection attempt. - -If the max connection limit is hit, there are two options: - -- Raise the max connections limit on the database. -- Build out the session in `setUp`/`setUpAll` instead of the top level scope: +Given a future call class named `ReminderFutureCall` with a `send` method, the accessor is `reminder`, following the same [naming rule](../scheduling/future-calls#schedule-a-call) as scheduling: ```dart -withServerpod('Given example test', (sessionBuilder, endpoints) { - // Instead of this - var session = sessionBuilder.build(); - - - // Do this to postpone connecting to the database until the test group is running +withServerpod('Given the reminder future call', (sessionBuilder, endpoints) { late Session session; - setUpAll(() { + + setUp(() { session = sessionBuilder.build(); }); - // ... + + test('when invoked then it records a reminder', () async { + await endpoints.futureCalls.reminder.send(sessionBuilder, 'user-42'); + + final reminders = await Reminder.db.find(session); + expect(reminders, hasLength(1)); + }); }); ``` -:::info +This runs the future call's method directly. It does not exercise scheduling, so use it to test what the call does rather than when it runs. + +## Too many database connections + +Dart's test runner runs test files in parallel, and each `withServerpod` group starts its own server with its own connection pool. On a machine with many cores, enough files running at once can exceed the database's connection limit and fail the run. -This case should be rare and the above example is not a recommended best practice unless this problem is anticipated, or it has started happening. +This is uncommon, and worth addressing only once you hit it. When you do, raise the limit on the database or cap how many files run at once: -::: +```bash +dart test -t integration --concurrency=4 +``` ## Testing exception monitoring -`withServerpod` accepts the same `experimentalFeatures` argument as the server, so you can register a [diagnostic event handler](../operations/exception-monitoring) in a test and assert that your code reports the exceptions you expect. +The `withServerpod` helper accepts the same `experimentalFeatures` argument as the server, so you can register a [diagnostic event handler](../operations/exception-monitoring) in a test and assert that your code reports the exceptions you expect. Write a handler that records what it receives, so the test can wait for an event: @@ -221,3 +222,9 @@ void main() { ``` Handlers run asynchronously and are not awaited by the code that triggers them, so wait for the event rather than asserting immediately after the call. + +## Related + +- [Writing tests](writing-tests): the pieces these examples build on. +- [Configuration](configuration): the options used above, such as `experimentalFeatures` and `rollbackDatabase`. +- [Best practices](best-practices): conventions worth following. diff --git a/docs/06-concepts/08-testing/04-best-practises.md b/docs/06-concepts/08-testing/05-best-practices.md similarity index 66% rename from docs/06-concepts/08-testing/04-best-practises.md rename to docs/06-concepts/08-testing/05-best-practices.md index a7f4ea51..35456707 100644 --- a/docs/06-concepts/08-testing/04-best-practises.md +++ b/docs/06-concepts/08-testing/05-best-practices.md @@ -7,12 +7,12 @@ toc_max_heading_level: 2 ## Imports -While it's possible to import types and test helpers from the `serverpod_test` package, it's completely redundant. The generated file exports everything that is needed. Adding an additional import is unnecessary noise and will likely also be flagged as duplicated imports by the Dart linter. +The generated test tools file re-exports the test helpers, so importing `serverpod_test` as well brings in the same names twice and adds noise for no gain. ### Don't ```dart -import 'serverpod_test_tools.dart'; +import 'test_tools/serverpod_test_tools.dart'; // Don't import `serverpod_test` directly. import 'package:serverpod_test/serverpod_test.dart'; ❌ ``` @@ -20,14 +20,15 @@ import 'package:serverpod_test/serverpod_test.dart'; ❌ ### Do ```dart -// Only import the generated test tools file. -// It re-exports all helpers and types that are needed. -import 'serverpod_test_tools.dart'; ✅ +// The generated file carries the test helpers and your endpoints. +import 'test_tools/serverpod_test_tools.dart'; ✅ ``` +Serverpod's own types are a separate matter. The generated file does not re-export them, so a test that uses `Session`, `Scope`, `Constant`, `ServerpodRunMode`, `ExperimentalFeatures`, or a Serverpod exception type needs `package:serverpod/serverpod.dart` as well. Your own models come from your project's generated protocol. + ## Database clean up -Unless configured otherwise, by default `withServerpod` does all database operations inside a transaction that is rolled back after each `test` (see [the configuration options](./the-basics#rollback-database-configuration) for more info on this behavior). +Unless configured otherwise, by default `withServerpod` does all database operations inside a transaction that is rolled back after each `test` (see [the configuration options](configuration#rollbackdatabase) for more info on this behavior). ### Don't @@ -75,9 +76,9 @@ While it's technically possible to instantiate an endpoint class and call its me ```dart void main() { // ❌ Don't instantiate endpoints directly - var exampleEndpoint = ExampleEndpoint(); + var greetingEndpoint = GreetingEndpoint(); - withServerpod('Given Example endpoint', ( + withServerpod('Given Greeting endpoint', ( sessionBuilder, _ /* not using the provided endpoints */, ) { @@ -85,8 +86,8 @@ void main() { test('when calling `hello` then should return greeting', () async { // ❌ Don't call an endpoint method directly on the endpoint class. - final greeting = await exampleEndpoint.hello(session, 'Michael'); - expect(greeting, 'Hello, Michael!'); + final greeting = await greetingEndpoint.hello(session, 'Bob'); + expect(greeting.message, 'Hello Bob'); }); }); } @@ -96,12 +97,11 @@ void main() { ```dart void main() { - withServerpod('Given Example endpoint', (sessionBuilder, endpoints) { + withServerpod('Given Greeting endpoint', (sessionBuilder, endpoints) { test('when calling `hello` then should return greeting', () async { // ✅ Use the provided `endpoints` to call the endpoint that should be tested. - final greeting = - await endpoints.example.hello(sessionBuilder, 'Michael'); - expect(greeting, 'Hello, Michael!'); + final greeting = await endpoints.greeting.hello(sessionBuilder, 'Bob'); + expect(greeting.message, 'Hello Bob'); }); }); } @@ -121,3 +121,9 @@ It is significantly easier to navigate a project if the different types of tests - `test/unit`: Unit tests. - `test/integration`: Tests for endpoints or business logic modules using the `withServerpod` helper. + +## Related + +- [Advanced examples](advanced-examples): patterns for streams, future calls, and business logic. +- [Configuration](configuration): the options `withServerpod` accepts. +- [Deploy to Serverpod Cloud](../../deployments/deploy-to-serverpod-cloud): ship the server your tests cover.