Testing Guide

Testing keeps regression risk low across services. The application server uses Maven with dedicated base classes for unit and integration tests.

Quick start

1. Decide whether you need a fast unit test or a Spring-backed integration test.

2. Run the appropriate command:

   # Unit tests only (fast)
   cd server && ./mvnw test
   
   # Unit and integration tests
   cd server && ./mvnw verify

   Generated clients have their own module, and the reactor restores their compiled artifact from Maven Build Cache when its inputs are unchanged. After one reactor build, repeated application or test edits can use the application-only recipes below without touching generated clients.

3. Extend the correct base class:

   Unit tests extend BaseUnitTest; Spring-backed tests extend BaseIntegrationTest or a feature-specific subclass.

:::warning Naming matters Use *Test.java for unit tests and *IntegrationTest.java or *LiquibaseTest.java for integration tests. Maven's Surefire and Failsafe plugins use these patterns. :::

Unit tests

• Extend BaseUnitTest for isolated component tests.
• Use Mockito (@Mock, @InjectMocks) for dependencies.
• Keep unit tests independent of Spring contexts and Testcontainers.

Integration tests

• Extend BaseIntegrationTest for cross-component behaviour.
• Relies on a shared PostgreSQL Testcontainers instance (significantly faster than per-test containers).
• Reads recorded webhook payloads from the fixture folder and deserialises them into the handler's event DTO.

Controller integration tests

Two rules, and everything below follows from them: test a controller at the HTTP boundary, because that is where a security misconfiguration shows; and keep fixtures reusable, so a new endpoint needs no bespoke bootstrapping.

• Extend AbstractWorkspaceIntegrationTest (or a service-specific base) so every controller IT automatically cleans the database, provides helper factories such as persistUser, and exposes workspaceService for setup.
• Use WebTestClient. The full Spring Boot context is already running, so access-control rules (401 unauthenticated vs 403 non-admin) are verified exactly as the frontend receives them.
• Reach for the custom security annotations (@WithAdminUser, @WithUser) together with TestAuthUtils.withCurrentUser() so the generated JWT matches our resource-server configuration.
• Assert on status code, payload shape, and repository side effects. A minimal test covers "happy path", "validation failure", and at least one access-control branch.
• Keep factories close to the domain: prefer createWorkspace("slug", "Display", ...) over manually seeding repositories, which keeps each test focused on behaviour instead of boilerplate.

Example skeleton:

@AutoConfigureWebTestClient
class FooControllerIntegrationTest extends AbstractWorkspaceIntegrationTest {

    @Autowired
    private WebTestClient webTestClient;

    @Test
    @WithAdminUser
    void endpointHappyPath() {
        User owner = persistUser("owner");
        Workspace workspace = createWorkspace("slug", "Display", "login", AccountType.ORG, owner);

        webTestClient
            .get()
            .uri("/workspaces/{workspaceSlug}", workspace.getWorkspaceSlug())
            .headers(TestAuthUtils.withCurrentUser())
            .exchange()
            .expectStatus()
            .isOk();
    }
}

Cross-tenant isolation tests

Tenancy bugs are silent data leaks: the endpoint answers 200 with plausible data, so nothing fails loudly. Every new workspace-scoped read path gets an isolation case in CrossTenantIsolationIntegrationTest (in workspace/), whose fixture and rationale are documented in its class Javadoc — two workspaces sharing an account_login, one overlapping member.

Checklist for a new read path:

• List endpoint — assert A's collection under A's slug never contains B's rows (length() + value, not just "my row is present").
• Detail endpoint (anti-IDOR) — assert B's entity id through A's slug returns 404, and pair it with a positive control (A's own id returns 2xx) so the 404 can't pass against a broken endpoint.
• Query keyed on anything but workspace_id (an org login, a login string, an external native id) is a red flag — scope by workspace_id, or add the provider/workspace predicate, before shipping. The #1372 leaks (leaderboard roster by org-login, team lists by org-string) were exactly this shape.
• @WorkspaceAgnostic repository? The SQL statement inspector is off for it, so the integration test is the only automated guard — do not skip it.

The hephaestus.tenancy.enforcement: throw setting in application-test.yml fails any test that reads a workspace_id-bearing table without a workspace_id predicate — a free safety net, but it cannot see @WorkspaceAgnostic queries or string-keyed scoping, which is why the HTTP-level assertions still matter.

Webhook fixtures and tooling

Reusable webhook JSON lives in server/application/src/test/resources/github, named after the event it records. Load one with a ClassPathResource, or extract new samples with:

pnpm run nats:extract-examples

Available examples include label.created, repository.created, create, push, and more – consult the folder before recording new payloads.

Maven recipes

From server/:

./mvnw test                         # Unit tests
./mvnw test -Parchitecture-tests     # ArchUnit + Modulith verification
./mvnw verify                       # Unit + integration tests and packaging
./mvnw -f application/pom.xml test -Dtest=SomeTest         # Single test class
./mvnw -f application/pom.xml test -Dtest=SomeTest#method  # Single test method

Run a reactor command first after a fresh checkout or after changing a schema, generator, generated-client dependency, or generated-clients/pom.xml. The application-only commands then preserve Maven's incremental compilation; they fail immediately if the generated-client artifact has not been installed.

The tag filter is a property, not the -Dgroups user property: surefire is configured with <groups>${surefire.includedGroups}</groups>, which an explicitly-configured plugin parameter wins over. -Dgroups=… is silently ignored; use -Dsurefire.includedGroups=… when you need to change the tag set directly, or the profiles above, which set it for you.

Live external-service tests

Some regressions can only be validated against real systems — the GitHub API, a real LLM endpoint, a Docker daemon. Live tests are tagged @Tag("live") and excluded from every other tier; they only run under cd server && ./mvnw test -Plive-tests. The profile flips Surefire's group filter and adds failIfNoSpecifiedTests=false so single-test invocations don't fail when the gating env var is unset.

Meta-annotation cheat sheet

Use one of these composite annotations on the test class (or method) rather than wiring @Tag("live") + @EnabledIfEnvironmentVariable by hand. They keep gates uniform and put the env-var contract in one place.

Annotation	Gate	Use for
@LiveLlmTest	HEPHAESTUS_LIVE_LLM_API_KEY	Tests that drive a real OpenAI-compatible endpoint. Set HEPHAESTUS_LIVE_LLM_BASE_URL, HEPHAESTUS_LIVE_LLM_API_KEY, and HEPHAESTUS_LIVE_LLM_MODEL explicitly. Read via LiveLlmCredentials.fromEnv().
@LiveGitHubTest	GH_APP_INSTALLATION_ID	Tests that hit github.com via the installed GitHub App. BaseGitHubLiveIntegrationTest runs an extra runtime check on private-key material.
@LiveGitLabTest	HEPHAESTUS_LIVE_GITLAB_TOKEN	Reserved for GitLab-side replay/live-sync tests.
@LiveDockerTest	none — runtime probe in @BeforeAll	Tests that need a Docker daemon. Combine with @LiveLlmTest when the sandbox payload also needs LLM credentials.

Each annotation pins JUnit to Execution(SAME_THREAD) so two live tests in the same module never race on rate limits, log interleaving, or shared temp dirs.

Authoring a new live test

1. Pick the right meta-annotation. If the env var doesn't exist yet, add a new annotation alongside the existing four — don't reuse one with a misleading name.
2. Put the test under server/application/src/test/java/.../live/. The directory marker keeps live and non-live tests in different code-review buckets.
3. Read credentials from LiveLlmCredentials.fromEnv() (or the GitHub helpers); never inline values. Secrets stay in env vars only.
4. Per-test wall-clock timeouts: 90s is reasonable for a single mentor turn; 5 minutes for a full practice run.

GitHub live API tests

We ship a focused suite that exercises the live GitHub App installation and verifies end-to-end sync behaviour (repository metadata, labels, milestones, and teams).

Prerequisites

1. Sandbox installation – the Hephaestus IntegrationTests GitHub App must be installed in a sandbox organisation you control. The tests create and delete repositories, milestones, labels, and teams on each run.

2. Credentials – provide both a GitHub App private key and a Personal Access Token with the following scopes:

   • repo (full)
   • admin:org
   • read:packages

3. Local config file – copy the template that lives alongside the tests:

   cd server/application/src/test/resources
   cp application-live-local.example.yml application-live-local.yml

   Fill in the placeholders with the sandbox organisation slug, the installation id, and either an inline PEM key (github.app.privateKey) or a readable privateKeyLocation. Keep this file out of version control – it is already listed in .gitignore.

   Alternatively, export the matching environment variables:

   export GH_APP_ID=2250297
   export GH_APP_INSTALLATION_ID=93512943
   export GH_APP_PRIVATE_KEY="$(cat /path/to/private-key.pem)"
   export GH_APP_PAT=ghp_xxx...             # PAT with the scopes above
   export GH_APP_ORGANIZATION=HephaestusTest

   Use either the config file or environment variables; the test suite checks both and aborts if key material is missing.

Running the suite

From server/ run:

./mvnw test -Plive-tests

The live-tests profile tells Maven to run only tests tagged with @Tag("live"). This is the single guard for live tests – if you don't pass the profile, the tests simply won't run.

Live tests never run during normal CI. They are excluded from the other tiers by tag filtering in the Surefire and Failsafe plugins.

The run takes roughly two minutes and prints the GitHub artefacts it provisions. Clean-up is handled automatically, but if a failure interrupts execution you can safely delete any hephaestus-it-* repositories, milestones, or teams that remain in the sandbox.

Authoring new GitHub sync tests

1. Extend AbstractGitHubLiveSyncIntegrationTest (or fall back to BaseGitHubLiveIntegrationTest when repositories are not needed). These bases set up credential checks, provide the workspaceRepository, and expose helpers such as createEphemeralRepository, registerRepositoryToMonitor, createEphemeralTeam, and seedOrganizationMembers.
2. Use the supplied helpers to create and track temporary GitHub artefacts. They automatically register clean-up handlers via @AfterEach, so add new resources to the provided lists instead of implementing manual deletion logic.
3. Keep tests deterministic: rely on databaseTestUtils.cleanDatabase() in @BeforeEach (already invoked by the base), and generate unique slugs via nextEphemeralSlug("suffix") when naming repositories, branches, or teams.
4. If a scenario needs extra Spring configuration, extend application-live-local.yml in server/application/src/test/resources/. The checked-in .example file documents every property; copy it on demand and keep secrets out of version control.

Troubleshooting

• Skipping because of missing credentials – check the console output; the base test class verifies that the App id, private key, PAT, and installation id are all present before executing.
• GitHub API rate-limit failures – the suite creates several entities per run. Prefer a dedicated sandbox organisation so you do not clash with production automation limits.
• Longer runtimes – each suite bootstraps a Testcontainers PostgreSQL instance and provisions GitHub resources. Expect higher runtimes than the pure Testcontainers integration tests; avoid running them on every PR and instead use them before releases or when touching the GitHub sync layer.