# Testing Guide Choose the tier by the behavior under test. [Gradle recipes](#gradle-recipes) lists the commands. JUnit tags select tests independently of filenames. Use descriptive `*Test` and `*IntegrationTest` names for readers. ## Unit tests * Extend `BaseUnitTest` for isolated component tests. * Use Mockito (`@Mock`, `@InjectMocks`) for dependencies. * Keep unit tests independent of Spring contexts and Testcontainers. ## Integration tests * Extend `BaseIntegrationTest` for cross-component behavior. * Reuse the shared PostgreSQL Testcontainer. * For webhook tests, deserialize recorded fixtures into the handler's event DTO. ## Controller integration tests Test controllers at the HTTP boundary so access-control configuration is exercised. Reuse the domain's integration-test base and fixture factories. * Extend `AbstractWorkspaceIntegrationTest` or a service-specific base. Each controller IT then cleans the database automatically. It provides helper factories such as `persistUser` and exposes `workspaceService` for setup. * Use `WebTestClient`. The full Spring Boot context already runs. Thus, tests verify access-control rules exactly as the frontend receives them (`401` unauthenticated versus `403` non-admin). * Reach for the custom security annotations (`@WithAdminUser`, `@WithUser`) together with `TestAuthUtils.withCurrentUser()` so the generated JWT matches our resource-server configuration. * Assert on status code, payload shape, and repository side effects. A minimal test covers "happy path", "validation failure", and at least one access-control branch. * Keep factories close to the domain. Prefer `createWorkspace("slug", "Display", ...)` to manual repository seeds. This keeps each test focused on behavior instead of boilerplate. Example skeleton: ``` @AutoConfigureWebTestClient class FooControllerIntegrationTest extends AbstractWorkspaceIntegrationTest { @Autowired private WebTestClient webTestClient; @Test @WithAdminUser void endpointHappyPath() { User owner = persistUser("owner"); Workspace workspace = createWorkspace("slug", "Display", "login", AccountType.ORG, owner); webTestClient .get() .uri("/workspaces/{workspaceSlug}", workspace.getWorkspaceSlug()) .headers(TestAuthUtils.withCurrentUser()) .exchange() .expectStatus() .isOk(); } } ``` ## Cross-tenant isolation tests Tenancy bugs silently leak data. The endpoint answers `200` with plausible data, so nothing fails visibly. Every **new workspace-scoped read path** gets an isolation case in `CrossTenantIsolationIntegrationTest` (in `workspace/`). Its class Javadoc documents the fixture and reason: two workspaces share an `account_login` and have one overlapping member. Checklist for a new read path: * **List endpoint** — assert A's collection under A's slug never contains B's rows (`length()` + value, not just "my row is present"). * **Detail endpoint (anti-IDOR)** — assert that B's entity id through A's slug returns `404`. Add a positive control: A's own id returns `2xx`. Thus, the `404` cannot pass against a broken endpoint. * **Query keyed on anything but `workspace_id`** is a warning sign. Examples are an org login, a login string, or an external native id. Before shipment, scope by `workspace_id` or add the provider/workspace predicate. * **SCM work scoped through `repository_to_monitor`?** A monitor stores only the repository name, and another provider can host the same name. Match the name, then call `monitor_covers_repository(...) = true` in JPQL or append `MonitoredRepositoryScope.NATIVE_RTM_R` in native SQL. Test two workspaces on different instances of the same provider kind. Give both monitors the same name and native id. Include the unconnected inverse and verify that removal preserves the other instance's repository. * **`@WorkspaceAgnostic` repository?** The SQL statement inspector is **off** for it. Thus, the integration test is the only automated guard. Do not skip it. The `hephaestus.tenancy.enforcement: throw` setting in `application-test.yml` fails tests that read a `workspace_id`-bearing table without a `workspace_id` predicate. This safety net cannot see `@WorkspaceAgnostic` queries or string-keyed scopes. Thus, HTTP-level assertions remain necessary. ## Webhook fixtures and tooling Reusable webhook JSON lives in `server/application/src/test/resources/github` and `…/gitlab`, named after the event it records. Load one with a `ClassPathResource`, or extract new samples with: ``` vp run schema:nats ``` Available examples include `label.created`, `repository.created`, `create`, `push`, and more – consult the folder before recording new payloads. Before you commit a recording, replace each person's identity with a synthetic one of the same shape. This includes the name, username, email address, numeric user ID and avatar URL. Use one synthetic identity for each person across all fixtures, so that a lookup that the tests exercise still matches. The test organization's own accounts (`HephaestusTest` and the maintainer account that recorded the payloads, `FelixTJDietrich`) can stay. `vp run schema:nats` only sees events an environment already subscribes to. To record any other GitHub event, add a temporary repository webhook in the `HephaestusTest` organization that subscribes to that event only. Its URL does not need to answer. Cause the event, and then save each delivery's `request.payload` from `gh api repos///hooks//deliveries/`. Delete the webhook and whatever you created when you are done. The `issue_dependencies` fixtures were recorded this way. A fixture ending in `.derived.json` is not a recording. It starts with a recorded payload. For fields without a recordable event in the test organization, it uses the provider's documented payload. Replace it with a recording when one exists. Each row lists every field the fixture adds, changes or removes against its recorded base. | Fixture | Recorded base | Fields that differ | | -------------------------------------------------------------- | ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `github/pull_request.review_requested.reviewer.derived.json` | `pull_request.review_requested`, a team request | `requested_team` replaced by `requested_reviewer`, a user, since GitHub sends one or the other ([octokit/webhooks example](https://github.com/octokit/webhooks/blob/main/payload-examples/api.github.com/pull_request/review_requested.payload.json)). `pull_request.requested_reviewers` lists that user and the Copilot reviewer bot as [`/users/copilot-pull-request-reviewer[bot]`](https://api.github.com/users/copilot-pull-request-reviewer%5Bbot%5D) reports it. `pull_request.requested_teams` is as recorded. | | `gitlab/merge_request.update.reviewers.derived.json` | `merge_request.update` | Adds `reviewers` with each `state` (`approved`, `requested_changes`, `reviewed`, `unreviewed`) and `re_requested: false`, `assignees`, and `changes.assignees` (none before). Removes `changes.labels` and `changes.updated_by_id`. Moves `object_attributes.updated_at` and `changes.updated_at` on to 19:05:00. No `changes.reviewers`: nobody's state changes in this hook. It carries all four states. Before 19.3, a review or request for changes runs no merge request hook of its own. | | `gitlab/merge_request.update.review_rerequested.derived.json` | `merge_request.update` | The re-request's own hook: adds `reviewers` as in `reviewers.derived` but with `user1` `unreviewed` and `re_requested: true`, `assignees`, and `changes.reviewers` from the `reviewers.derived` list to this one, as the only `changes` entry. Moves `object_attributes.updated_at` on to 19:08:00, which the re-request's system note touches before the hook runs ([`request_review_service.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/v18.6.0-ee/app/services/merge_requests/request_review_service.rb#L10-28)). | | `gitlab/merge_request.update.reviewers_removed.derived.json` | `merge_request.update` | Adds `changes.reviewers` from the `review_rerequested` list to none and `changes.assignees` from `ab12cde` to none, with no `reviewers` or `assignees` key, as GitLab leaves them out when there are none ([`issuable.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/v18.6.0-ee/lib/gitlab/data_builder/issuable.rb#L28-34)). Removes `changes.labels` and `changes.updated_by_id`. Moves `object_attributes.updated_at` on to 19:15:00, `changes.updated_at` from 19:08:00. | | `gitlab/merge_request.update.reviewers_stateless.derived.json` | `merge_request.update` | A hook from GitLab before 18.6, whose reviewers carry `User#hook_attrs` only ([`issuable.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/v18.4.0-ee/lib/gitlab/data_builder/issuable.rb#L31)): adds `reviewers` as in `reviewers.derived` without `state` and `re_requested` and without `user3`, `assignees`, and `changes.reviewers` from the four to the three. Removes `changes.labels` and `changes.updated_by_id`. Moves `object_attributes.updated_at` on to 19:06:00 and `changes.updated_at` from 19:05:00 to 19:06:00. | The GitLab fixtures other than `reviewers_stateless` follow GitLab 18.6 and later, which states each reviewer's `state` and `re_requested` ([`reviewers_hook_attrs`](https://gitlab.com/gitlab-org/gitlab/-/blob/v18.6.0-ee/app/models/concerns/issuable.rb#L589-597)) and lists `changes.reviewers` whenever those attributes change ([`hook_reviewer_changes`](https://gitlab.com/gitlab-org/gitlab/-/blob/v18.6.0-ee/app/models/concerns/issuable.rb#L599-609)). From 19.3 a submitted review runs merge request hooks of its own as well ([`update_reviewer_state_service.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/v19.3.0-ee/app/services/merge_requests/update_reviewer_state_service.rb#L67-75)). The GitLab fields follow [merge request events](https://docs.gitlab.com/user/project/integrations/webhook_events/#merge-request-events). ## Gradle recipes From `server/`: ``` ./gradlew :application:test # Unit tests ./gradlew :application:architectureTest # ArchUnit and Modulith ./gradlew :application:verification # Unit, architecture and coverage floors ./gradlew :application:integrationTest # Spring and Testcontainers ./gradlew :application:test --tests SomeTest # One unit test class ./gradlew :application:test --tests 'SomeTest.method' # One unit test method ./gradlew :application:bootJar # Executable JAR, without running tests ``` Generation and compilation dependencies are automatic. Test tasks always execute when requested. Compilation remains incremental and cacheable. `--no-build-cache` prevents output restoration. Combine it with `clean` when measuring a cold build. The shared integration PostgreSQL container runs the `docker/postgres` image. Locally, each test JVM builds that image from its Dockerfile. CI integration jobs instead pull the image that CI published for the commit their images alias from. They name it in `HEPHAESTUS_TEST_POSTGRES_IMAGE`. When a change edits the image, or no published image exists, CI builds the Dockerfile once per job before the tests start. That build downloads packages, as a local build does. Contexts built on `BaseIntegrationTest` reach GitHub's GraphQL endpoint through `ScriptedGitHub`. The application's own client, senders, schedulers and listeners run unchanged, and only the HTTP exchange answers from the script. A test reads what the application posted from `ScriptedGitHub.threads()`, keyed by repository and number. A GitHub document that the script does not answer fails as a GraphQL error. No JVM client in a test resolves a real host. `OfflineNetwork` replaces `InetAddress` host-name resolution in every test JVM. A child process such as `git`, a native DNS client or a container uses its own resolver, so the guard does not cover it. Loopback names and the names the host gives its own addresses resolve. A name reserved by RFC 2606 or RFC 6761 fails as an unknown host. Any other name is refused and recorded, and `OfflineNetworkExtension` fails the test class that tried it. A reverse lookup of a public address fails too. It is recorded unless one of the host's own interfaces carries the address. A test whose code only classifies an address declares the name with `OfflineNetwork.answer`. The live tier resolves every name. The test database starts with silent mode engaged, as a new instance does. A test that asserts a posted copy turns silent mode off and restores it afterwards. The shared integration PostgreSQL container keeps its ephemeral data directory on tmpfs. Database cleanup still truncates every application table transactionally. SQL constraints and isolation checks remain enabled. This container does not prove storage durability across container restarts. `vp run test:server:selection` runs dedicated Gradle/JUnit inventory tasks without executing tests. They use the same tier and shard selectors as the execution tasks. The command proves that non-live tests belong to a required tier and that integration shards are disjoint and complete. Required PR verification and `vp run verify` run this proof before the real suites. Discovery writes only to `application/build/test-selection/`, leaving execution reports and coverage untouched. Those discovery reports are not evidence that tests passed. Execution tasks do not accept the removed `-PtestSelection` property. Use the selection command for discovery. ## Flaky tests A flaky test fails and then passes on the same commit. CI does not retry Gradle or Vitest tests. Playwright retries a failed test once to record a trace, and then it fails the run as flaky. You can rerun a failed job, but the rerun does not hide the flake. The weekly [flaky test report](/contributor/ci-cd.md#flaky-tests) lists every job that passed only on a rerun. Maintainers read it each week. When a test flakes: 1. Open an issue for the test and assign an owner. 2. The owner finds the cause and fixes it, or deletes a test that asserts nothing new. Do not add a retry, a longer timeout, or a sleep to hide it. 3. Quarantine the test only while the flake blocks merges and the cause is not fixed yet. The skip names the issue, the owner's GitHub login and an end date at most 14 days ahead. In Java, write `@Disabled("quarantine #123 @owner until 2026-11-07: free-port race")`. In TypeScript, put `// quarantine #123 @owner until 2026-11-07: free-port race` on the line directly above `it.skip(`. 4. On the end date, the owner fixes or deletes the test. `vp run check` fails on a skip that names no issue, owner and end date, and on an expired quarantine. A skip that depends on the environment, such as missing live credentials, is not a quarantine. ## Live external-service tests Live tests use real provider APIs, LLM endpoints or sandbox containers. They carry `@Tag("live")` and are excluded from every other tier. From `server/`, explicitly opt in with: ``` ./gradlew :application:liveTest ``` Use `--tests ClassName` to select a suite. An unmatched filter fails. To send only some catalogue criteria cases to the model, name their `id` values from `src/test/resources/practices/criteria-cases.json`: ``` ./gradlew :application:liveTest -PcriteriaCases=dependency-label,dependency-import-without-declaration-edit ``` The property first runs the selected fixture-staging checks, then those cases of `PracticeRunnerLiveLlmTest` and no other live test. Each case is reported under its `id`, not its position in the file. A blank, repeated or unknown `id` fails the run instead of sending every case. Without the property, the live tier runs every case. To also limit the upstream model requests that each selected case sends, add the optional `criteriaRequestLimit` property: ``` ./gradlew :application:liveTest -PcriteriaCases=dependency-label -PcriteriaRequestLimit=12 ``` The limit applies to each case. Each forwarding attempt reserves one slot, including retries and compaction requests. Failed or cancelled attempts keep their slots. A request past the limit is refused, and that case fails, even if its result matches. The limit needs `criteriaCases` and a positive integer. Without the property, this harness adds no request-count limit. Record provider token usage separately. To measure the same cases with other model settings, add one or more of these optional properties: ``` ./gradlew :application:liveTest -PcriteriaCases=dependency-label \ -PcriteriaReasoningEffort=HIGH -PcriteriaMaxOutputTokens=32768 -PcriteriaContextWindow=131072 ``` | Property | Value | Default | | ------------------------- | --------------------------------------------------------------------------------------------------------------- | --------- | | `criteriaReasoningEffort` | A reasoning effort: `NONE`, `MINIMAL`, `LOW`, `MEDIUM`, `HIGH`, `XHIGH` or `MAX`. The names are case-sensitive. | none sent | | `criteriaMaxOutputTokens` | A positive integer, less than the context window | 4096 | | `criteriaContextWindow` | A positive integer | 131072 | Each property needs `criteriaCases`. Gradle refuses a token count that is not a positive integer before any test starts. The live test refuses a lower-case or unknown reasoning effort, and an output allowance that does not fit the context window, before any model request. A live test that is skipped because no credentials are set checks none of these values. Each case prints the model settings it uses. First check the same selection without credentials or a model: ``` ./gradlew :application:integrationTest -PcriteriaCases=dependency-label,dependency-import-without-declaration-edit ``` This runs only the selected cases in `PracticeCriteriaCaseStagingIntegrationTest`. Each invocation stages the evidence and shipped criteria used by the live harness. It proves selection and fixture staging, not model outcomes. Gradle's test dry run does not execute a parameterized method's argument source. Use the staging results to count selected case IDs. A dry-run report cannot prove that count. The live credential gate remains in effect. Do not combine `criteriaCases` with `HEPHAESTUS_INTEGRATION_SHARD`. The tasks reject that combination instead of adding unrelated shard tests. ### Meta-annotation cheat sheet Use one of these composite annotations on the test class (or method) rather than wiring `@Tag("live") + @EnabledIfEnvironmentVariable` by hand. They keep gates uniform and put the env-var contract in one place. | Annotation | Gate | Use for | | ----------------- | ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `@LiveLlmTest` | `HEPHAESTUS_LIVE_LLM_API_KEY` | Tests that drive a real OpenAI-compatible endpoint. Set `HEPHAESTUS_LIVE_LLM_BASE_URL`, `HEPHAESTUS_LIVE_LLM_API_KEY`, and `HEPHAESTUS_LIVE_LLM_MODEL` explicitly. Read through `LiveLlmCredentials.fromEnv()`. | | `@LiveGitHubTest` | `GH_APP_INSTALLATION_ID` | Tests that hit github.com through the installed GitHub App. `BaseGitHubLiveIntegrationTest` runs an extra runtime check on private-key material. | | `@LiveGitLabTest` | `HEPHAESTUS_LIVE_GITLAB_TOKEN` | Reserved for GitLab-side replay/live-sync tests. | | `@LiveDockerTest` | none — runtime probe in `@BeforeAll` | Tests that need a Docker daemon. Combine with `@LiveLlmTest` when the sandbox payload also needs LLM credentials. | The annotations use `Execution(SAME_THREAD)`. Gradle also runs one test JVM at a time. ### Authoring a new live test 1. Pick the correct meta-annotation. If the environment variable does not exist yet, add an annotation alongside the existing four. Do not reuse one with a misleading name. 2. Put the test under `server/application/src/test/java/.../live/`. The directory marker keeps live and non-live tests in different code-review buckets. 3. Read credentials from `LiveLlmCredentials.fromEnv()` or the provider helpers. Never inline secrets or commit local credential files. 4. Per-test wall-clock timeouts: 90s is reasonable for a single conversation turn on a warm host. The first turn also starts the conversation host. 5 minutes for a full practice run. ### Conversation turn live test `HephTurnLiveTest` runs one conversation turn through the real path: the ask endpoint, the worker's claim, the conversation host container, the LLM proxy and a model. It needs Docker and the agent image built from `docker/agents/pi`: ``` docker buildx build -f docker/agents/pi/Dockerfile -t hephaestus-agent-pi:local --load docker/agents HEPHAESTUS_AGENT_IMAGE_REFERENCE=hephaestus-agent-pi:local \ HEPHAESTUS_LIVE_LLM_BASE_URL=... HEPHAESTUS_LIVE_LLM_API_KEY=... HEPHAESTUS_LIVE_LLM_MODEL=... \ ./gradlew :application:liveTest --tests HephTurnLiveTest ``` The host reaches the test's sandbox gateway at `host.docker.internal`. The gateway binds a free port unless `SANDBOX_API_PORT` names one. If a host firewall admits containers to one port only, set `SANDBOX_API_PORT` to that port. ### Hephaestus voice eval `server/application/src/test/resources/agent/heph-voice.live.ts` sends nine questions to a real model through the conversation host. The eval composes the prompt as `HephHostSpec` does, from `system.md` and the writing guide, and a fake worker serves a small context folder. Each case asserts what an answer must never do and prints one line: `eval outputTokens= words= fieldsRead=`. The file name does not match `*.spec.ts`, so `vp run test:agents` and `vp run check` do not run it. It reads the `@LiveLlmTest` credentials in the [meta-annotation cheat sheet](#meta-annotation-cheat-sheet). Without them, every case skips. From the repository root: ``` HEPHAESTUS_LIVE_LLM_BASE_URL=… HEPHAESTUS_LIVE_LLM_API_KEY=… HEPHAESTUS_LIVE_LLM_MODEL=… \ node --test server/application/src/test/resources/agent/heph-voice.live.ts ``` To compare another prompt, set `HEPHAESTUS_LIVE_HEPH_SYSTEM_PROMPT_FILE` to a file with the whole system prompt. It replaces `system.md` and the writing guide. ### Live models for practice scripts Two live tests run a practice program that uses every direct model slot against real models. | Test | Runs | What it proves | | --------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `docker/agents/practice-scripts/live.test.ts` | Custom programs through the program host, `docker/agents/practice-scripts/lib/core/program-host.ts`. The proxy stand-in forwards each call to the real endpoint. | Each bound slot answered every call, and one review request went through IPC. A program can return its own result. Each call names its practice and carries the bearer token. | | `PracticeScriptLiveLlmTest` | The same fixture behind the real LLM proxy on the sandbox gateway, with the job token and the models file of the job | The proxy routes each slot and records the usage of the attempt. The end of the attempt adds one ledger row for each bound decision, embedding and reranking model. | The server test carries `@LiveLlmTest`. Both tests read the same variables: | Variable | Slot | Required | | --------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | | `HEPHAESTUS_LIVE_LLM_BASE_URL`, `HEPHAESTUS_LIVE_LLM_API_KEY`, `HEPHAESTUS_LIVE_LLM_MODEL` | `chat` | Yes. Without them, the test skips itself. | | `HEPHAESTUS_LIVE_EMBEDDING_MODEL` | `embedding`, on the chat endpoint | No | | `HEPHAESTUS_LIVE_RERANKING_MODEL` | `reranking`, on the chat endpoint | No | | `HEPHAESTUS_LIVE_DECISION_BASE_URL`, `HEPHAESTUS_LIVE_DECISION_API_KEY`, `HEPHAESTUS_LIVE_DECISION_MODEL` | `decision`, on an OpenAI Decisions endpoint | No. Without them, the host test lets the chat model serve the decision slot through its log probabilities. The server test leaves the slot unbound. | A slot without its variables is not bound. The host test does not assert it. The server test asserts that the program reports it as not available. The chat model must return `top_logprobs` when no decision endpoint is set. The tests assert what the host guarantees, never the wording of a model. To run the host test from the repository root: ``` HEPHAESTUS_LIVE_LLM_BASE_URL=… HEPHAESTUS_LIVE_LLM_API_KEY=… HEPHAESTUS_LIVE_LLM_MODEL=… \ node --test docker/agents/practice-scripts/live.test.ts ``` To run the server test, from `server/`: ``` ./gradlew :application:liveTest --tests PracticeScriptLiveLlmTest ``` `vp run test:agents` also runs `live.test.ts`, which skips itself without the chat variables. ### GitHub live API tests We ship a focused suite that exercises the live GitHub App installation and verifies end-to-end sync behavior (repository metadata, labels, milestones, and teams). ### Prerequisites 1. **Sandbox installation** – the `Hephaestus IntegrationTests` GitHub App must be installed in a sandbox organization you control. The tests create and delete repositories, milestones, labels, and teams on each run. 2. **Credentials** – provide both a GitHub App private key and a Personal Access Token with the following scopes: * `repo` (full) * `admin:org` * `read:packages` 3. **Local config file** – copy the template that lives alongside the tests: ``` cd server/application/src/test/resources cp application-live-local.example.yml application-live-local.yml ``` Fill in the placeholders with the sandbox organization slug, the installation id, and either an inline PEM key (`github.app.privateKey`) or a readable `privateKeyLocation`. Keep this file out of version control – it is already listed in `.gitignore`. Alternatively, export the matching environment variables: ``` export GH_APP_ID=2250297 export GH_APP_INSTALLATION_ID=93512943 export GH_APP_PRIVATE_KEY="$(cat /path/to/private-key.pem)" export GH_APP_PAT=ghp_xxx... # PAT with the scopes above export GH_APP_ORGANIZATION=HephaestusTest ``` `GH_APP_INSTALLATION_ID` must be exported even when the other credentials come from the local file: `@LiveGitHubTest` checks the environment before Spring loads that file. Missing credentials skip the suite. A successful Gradle exit alone does not prove live tests executed. The run prints the GitHub artifacts it provisions. Clean-up is handled automatically, but if a failure interrupts execution you can safely delete any `hephaestus-it-*` repositories, milestones, or teams that remain in the sandbox. ### Authoring new GitHub sync tests 1. Extend `AbstractGitHubLiveSyncIntegrationTest` (or fall back to `BaseGitHubLiveIntegrationTest` when repositories are not needed). These bases set up credential checks, provide the `workspaceRepository`, and expose helpers such as `createEphemeralRepository`, `registerRepositoryToMonitor`, `createEphemeralTeam`, and `seedOrganizationMembers`. 2. Use the supplied helpers to create and track temporary GitHub artifacts. They automatically register clean-up handlers through `@AfterEach`, so add new resources to the provided lists instead of implementing manual deletion logic. 3. Keep tests deterministic. Rely on `databaseTestUtils.cleanDatabase()` in `@BeforeEach`, which the base already invokes. Generate unique slugs through `nextEphemeralSlug("suffix")` for repositories, branches, or teams. 4. If a scenario needs extra Spring configuration, extend `application-live-local.yml` in `server/application/src/test/resources/`. The checked-in `.example` file documents every property. Copy it on demand and keep secrets out of version control. ### Troubleshooting * **Skipping because of missing credentials** – export `GH_APP_INSTALLATION_ID`, then check the console for the base class's App id, private-key and PAT checks. * **GitHub API rate-limit failures** – the suite creates several entities per run. Prefer a dedicated sandbox organization so you do not clash with production automation limits. * **Longer runtimes** – each suite bootstraps a Testcontainers PostgreSQL instance and provisions GitHub resources. Expect higher runtimes than the pure Testcontainers integration tests. Avoid running them on every PR and instead use them before releases or when touching the GitHub sync layer. ## Practice cutover proof Database-tier tests seed the complete released schema and realistic retained data before applying the cutover. Fresh-schema tests cannot replace that upgrade proof. Focused integration tests prove account claim and absorption, result ordering, mark reverses and uncertain-write recovery. Scan tests distinguish withheld from unreviewed pairs and preserve reused membership after job deletion. They test complete inventory, the 20-percent removal boundary and comparison under a changed method. The [migration procedure](/contributor/database-migration.md#prove-the-released-state-upgrade) owns the database proof steps.