Sizing and scaling

This page owns the resource budget of the self-hosted Compose stack. The Install guide gives the host requirements.

Initial budget

4 vCPUs, 24 GB RAM, 40 GB SSD is a conservative initial budget for one concurrent review sandbox. It is not a measured requirement. It covers these memory limits, which are ceilings, not reservations:

Container	Memory limit	Setting
application-server	5 GiB	APPLICATION_SERVER_MEM_LIMIT
application-worker	3 GiB	APPLICATION_WORKER_MEM_LIMIT
webhook-server	2 GiB	WEBHOOK_SERVER_MEM_LIMIT
postgres	4 GiB	POSTGRES_MEM_LIMIT
each concurrent review sandbox	3 GiB	SANDBOX_MEMORY_BYTES, SANDBOX_MAX_CONCURRENT (1 in the self-hosted Compose file)
the conversation host, one per worker	2 GiB	Fixed. Runtime roles gives the limits of the conversation host

With the defaults of the self-hosted Compose file, these limits add up to 19 GiB, about 20.4 GB. During an upgrade, the one-shot database-migration container runs beside the previous roles for a few minutes. Keep 1 GiB free for it (DATABASE_MIGRATION_MEM_LIMIT). NATS, Traefik, and the webapp have no limit. The rest of the 24 GB is for them and the operating system.

A limit caps a container but reserves nothing. Each JVM sizes its heap from its own limit and can grow toward it. The host must hold simultaneous use by all containers and sandboxes, plus the operating system and spare capacity. If demand exceeds memory, the host swaps where swap is configured. The kernel can kill a container or sandbox.

To use a smaller memory budget, lower the limits in .env before the first start.

A worker runs no more review sandboxes at once than SANDBOX_MAX_CONCURRENT and its review capacity allow. A review whose sandbox has ended writes its feedback without a sandbox, so it does not count against these limits. The application default of SANDBOX_MAX_CONCURRENT is 5. The self-hosted Compose file sets 1. The review capacity (HEPHAESTUS_WORKER_REVIEW_MAX) defaults to auto: the number of CPUs minus 1.

Each worker also runs turn sandboxes for Hephaestus, beside the review sandboxes and the conversation host. Their limits are fixed and do not count against SANDBOX_MAX_CONCURRENT.

PostgreSQL memory

The bundled PostgreSQL image sizes its memory settings from the memory limit of its container when it starts. shared_buffers is the starting value that the PostgreSQL manual gives. The other values are common starting points:

Setting	Value
shared_buffers	25 % of the limit
effective_cache_size	75 % of the limit
maintenance_work_mem	1/16 of the limit, from 64 MB to 1 GB
max_wal_size	half of the limit, at least 1 GB
max_connections	one for each 16 MB, from 100 to 400

A container without a limit keeps the PostgreSQL defaults, because the memory of the host is shared. A -c flag in the command of the postgres service overrides a value. The image also records statement statistics. To give PostgreSQL more memory, raise POSTGRES_MEM_LIMIT and restart the postgres service.

Review sandbox limits

Each review sandbox has a limit of 3 GiB of memory (SANDBOX_MEMORY_BYTES), 1 CPU (SANDBOX_CPUS), and 256 processes (SANDBOX_PIDS_LIMIT). On Kubernetes, the worker sizes each sandbox Pod by its kind and ignores SANDBOX_MEMORY_BYTES (Kubernetes § Sandbox sizes). The CPU limit is a share of CPU time that the sandbox can use, not a reserved core. The memory limit holds the sandbox's temporary file systems, which can grow to 2.06 GiB, and its processes. Hephaestus refuses to start if SANDBOX_MEMORY_BYTES leaves less than 512 MiB beside those file systems.

These limits come from one measurement on release 0.82.0. Later releases are not re-measured. On that release, a review of a small course repository reached a cgroup memory.peak of 194 MiB. It used at most 19 processes and 0.2 CPU, in samples taken once per second. The workspace initializer used less than 1 MiB of its own memory for a 437 MB workspace. The rest of its memory was file cache, which the kernel reclaims under the limit.

A larger repository or a review that runs heavy commands can need more memory. If the kernel stops a review sandbox at its memory limit, the review fails with exit code 137. Then increase SANDBOX_MEMORY_BYTES.

CPU and processes

The application server and worker each default to 4 CPUs / 512 PIDs. The webhook server defaults to 2 CPUs / 256 PIDs. Override them with the corresponding APPLICATION_SERVER_*, APPLICATION_WORKER_* or WEBHOOK_SERVER_* limit variables. Keep the worker's PID limit: it is the role with Docker access.

What you can scale

The self-hosted stack runs one container for each runtime role: one server, one worker, and one webhook server. You scale the work of the worker, not the number of containers:

• Practice reviews. SANDBOX_MAX_CONCURRENT sets how many review sandboxes the worker runs at once. On Docker, each concurrent review sandbox may use up to SANDBOX_MEMORY_BYTES (default 3 GiB). The self-hosted Compose file allows 1 concurrent sandbox. Raise SANDBOX_MAX_CONCURRENT only with sufficient RAM. Worker pod env vars explains how it caps the review capacity of the worker.
• conversation turns. HEPHAESTUS_WORKER_MENTOR_MAX sets how many conversation turns the worker runs at once. In the Helm chart, set worker.conversationTurns. The model provider usually sets the real limit. Conversation turn capacity explains how to size it.

These are deployment limits, not a claim that all workloads fit. Before raising webhook volume, Conversation turn capacity (HEPHAESTUS_WORKER_MENTOR_MAX), or SANDBOX_MAX_CONCURRENT, reproduce the release's Compose capacity qualification on your host shape. A release baseline is valid only with the raw k6 and container-resource evidence described there. Without that evidence, keep the conservative defaults above.

Database connections

Preparing code evidence can hold two database connections. The worker also needs connections for its control traffic and to record results. Thus, the worker's pool must hold two connections for each review that can prepare code evidence at the same time, plus room for the rest.

The scheduled GitHub and GitLab syncs together run at most four workspace syncs at the same time on each server. A workspace sync can hold two connections, so the scheduled syncs use at most eight connections of the server's pool. The rest stays free for requests.

The worker profile gives the worker a pool of 32 connections, in Compose and on any other deployment. The webhook server has no database and no pool. The server keeps the application default of 20. HIKARI_MAXIMUM_POOL_SIZE overrides the pool size of each JVM that receives it. The Compose stacks pass it to the worker only.

Each JVM replica can open up to its own pool size. Budget all clients against max_connections:

Clients	Connections
The pools of one server, one worker and one webhook server	20 + 32 + 2 = 54
The Hephaestus listener of the server and of the worker, outside pools	2
A database migration or a backup	up to 10
Operator sessions and monitoring	a few
The superuser reserve of PostgreSQL	3

Compose stops the old container of a service before it starts the new one, so the roles of two releases never run together. Then about 70 connections are in use at most. A rolling update, as on Kubernetes, runs the old and the new roles at the same time. It needs both sets of pools and listeners, 112 connections, and about 130 with the other clients.

At the default limit of 4 GiB, PostgreSQL allows 256 connections. Below 1.6 GiB, it allows 100, which holds the roles of one release only. For more replicas or larger pools, raise POSTGRES_MEM_LIMIT, or lower HIKARI_MAXIMUM_POOL_SIZE.

Memory headroom

When the JVM starts, the buildpack's memory calculator sets the Java memory from the container's memory limit. BPL_JVM_HEAD_ROOM is an optional percentage of that limit. The calculator keeps this percentage out of the Java memory that it calculates. The heap must also leave room for the worker's file cache and for other memory that the container uses outside the JVM.

The default headroom is 0 %. Thus, if you increase the container memory limit, the heap can also increase at the next start. Select the headroom for the actual workload of your worker.

The split Compose stack passes BPL_JVM_HEAD_ROOM to the worker only. If you do not set it, the buildpack's default behavior applies. The value applies only when the JVM starts. To apply a change, drain and restart the worker, as Graceful drain describes. For the calculation, see the Paketo memory calculator.

Disk

Docker's data directory holds the images, the containers and the volumes. On a host with pull-based deployment, plan for these parts:

Part	Size	What bounds it
The images of one release	About 1 GB to download for linux/amd64. The application server is about half of it	The reconciler keeps the running and the previous release. During an apply, it pulls a third set
PostgreSQL data (app_postgresql-data)	Grows with your workspaces and their history	Your data. Back it up as Backup and restore describes
Git repositories (app_git-repos, app_worker-git-repos)	Grows with the repositories that reviews read	Not bounded yet
NATS JetStream (core_nats-data)	Grows with buffered provider events	Not bounded yet
A restore rehearsal	A full copy of the database volume	Remove the clone's containers and volumes when the rehearsal ends
Free space before a pull	10 GiB, set by HEPHAESTUS_MIN_FREE_GIB	The reconciler refuses an apply below it

On a small disk, lower HEPHAESTUS_MIN_FREE_GIB. docker system df -v shows what each image, container and volume uses on disk.