Connect an AI Provider

Hephaestus ships with no provider, model, or key. Provider configuration does not use environment variables. Connections, models, prices, and grants are runtime catalog data you enter in the UI. Practice reviews and conversation answers stay inactive until a model is registered, priced, and assigned to a purpose.

The catalog you use determines whose provider account pays:

	Instance catalog	Workspace's own provider (BYO)
Who sets it up	Instance admin, under Instance admin → AI models	Workspace admin, under the workspace's Administration → AI models
Whose provider account is billed	Yours, because you registered the key	Theirs
Who may use a model	Every workspace, or only those you grant access	Only that workspace
Monthly cap	The shared-model budget you set per workspace	The workspace's own provider cap

The two caps are never added together. Neither pauses the other's work. See Workspace budgets. A workspace admin can add a provider only while Let workspaces add providers and models is on in instance settings. If you turn it off, new connections and models return 403. Existing ones remain fully manageable.

1. The staged flow, and why it is staged

A new connection and model are inactive. The server refuses model activation until the model has a cost declaration.

1. Create the connection. It saves disabled: the create form always sends enabled: false.
2. Select Test and fetch models for it.
3. Add a model under that connection, with its upstream model ID.
4. Declare its price or mark it No metered API cost.
5. Enable the connection.
6. Enable the model.
7. Share it publicly with all workspaces or grant it to a list.
8. As a workspace admin, assign it to a purpose on your workspace's AI models page. Examples are Practice reviews and Conversations.

Nothing runs until the last step. A ready Conversations row on AI models offers Ask Hephaestus to every workspace member. If you turn off every Conversations row, Ask Hephaestus cannot answer. There is no separate workspace switch or per-account grant before you invite a cohort. Each member's AI choice determines whether Hephaestus answers them. An instance admin without workspace membership can administer it but cannot use Ask Hephaestus there.

An attempt to skip a stage returns HTTP 400 with a specific message, not a generic validation error:

Attempt	What you get back
Create a model with enabled: true	“Create the model disabled, set its price, then activate it.”
Enable a model with an inactive connection or UNPRICED price	“Activate the connection and set a price before you activate the model.”
Reprice an enabled model to UNPRICED	“Disable the model before you change its price to UNPRICED.”

An enabled model without a price would spend money that nobody can account for. It would make the month's budget verdict UNVERIFIABLE rather than uncapped. The staged flow prevents this.

2. Connection fields

Both catalogs use the same fields. Only displayName, baseUrl, and apiProtocol are required.

Field	Required	What it is
Display name	yes	Free text, ≤ 128 characters. Editable after creation. Display name, service declaration, API key, and enabled state can change. Routing identity and slug cannot.
Base URL	yes	The provider's API root, ≤ 2048 characters. Examples: https://api.openai.com/v1, or https://<resource>.openai.azure.com/openai/v1 for Azure. Requires HTTPS. Must have no credentials, query string, or fragment. Immutable after create.
API	yes	The API that the endpoint serves. Under Chat models: Responses API (openai-responses), the default, Chat Completions API (openai-completions), and Messages API (Anthropic) (anthropic-messages). Under For practice scripts: Decisions API, Embeddings API, and Rerank API (Cohere-compatible). Models for practice scripts describes these three. Prefer Responses API for reviews and conversations, which call tools across many turns. It can preserve reasoning between tool calls for supported models and providers. OpenAI, Vercel AI Gateway, OpenRouter, and vLLM offer it. Support depends on the model and deployment. For Chat Completions-only endpoints, choose Chat Completions API. Immutable after create. The edit form shows the API as text.
Authentication	no	BEARER → Authorization: Bearer <key>, or API_KEY → api-key: <key>. Defaults to BEARER. Azure needs API_KEY. Immutable after create.
Service receiving requests	no	Search for the service that receives requests from the configured endpoint, such as Logos, Azure, or Vercel AI Gateway. If absent or uncertain, leave Not listed or unknown. This display declaration does not prove where inference runs or who operates the model. You can change or clear it later.
API key	no	Write-only and encrypted at rest. Never returned. A saved connection reports only hasApiKey and the last four characters. On edit, leave blank to keep the stored key. To delete it, select Remove stored API key. Optional: a self-hosted vLLM or Ollama gateway may need no credential.
Slug	no	Internal identifier, ≤ 63 characters. Derived from the display name if omitted. Immutable after create.

Three fields are immutable on purpose. Base URL, API, and authentication mode form the connection's routing identity. Every job preserves that identity in its provenance snapshot. An in-place edit would silently point historical runs at an endpoint they never used. The form's exact explanation is:

Endpoint, API and authentication cannot change. Add a connection instead.

To move providers:

1. Create a replacement connection.
2. Test it.
3. Point the assignments to it.
4. Disable the old connection.

A slug you type is taken literally. A derived slug gets a -2, -3 suffix if it conflicts. A supplied slug does not. A conflict returns 409 llm-connection-slug-conflict rather than silently changing the slug.

The presets

The create form offers these presets:

• OpenAI (https://api.openai.com/v1, Bearer).
• Azure OpenAI v1 (https://RESOURCE.openai.azure.com/openai/v1, API key).
• Other endpoint.

They only prefill the two fields. Nothing about the preset is stored. For an Azure mark beside a workspace model, use the separate Service receiving requests declaration.

Neither OpenAI nor Azure OpenAI serves a rerank API. Thus, when you choose Rerank API (Cohere-compatible) with one of these presets, the form changes the preset to Other endpoint. A base URL that you typed stays.

For a gateway, name the gateway that receives the request, not an unknown downstream service. If you cannot confirm the connection, leave it blank. The URL does not determine the platform. The platform does not set the model's data-handling tier.

What the URL guard rejects, and when

The server validates the base URL only on create. An update cannot change it, so no repeat check is necessary. The table lists four rejections. The HTTP 400 rejections have title Invalid request:

Message	Cause
“Provider URLs must not contain credentials or query parameters.”	The URL has userinfo, a query string, or a fragment. Gateways with URL keys (https://gw/v1?api-key=…) are refused. That value would enter snapshots, DTOs, and logs.
“Provider host must be a public HTTPS URL”	Non-HTTPS, or a private, link-local, loopback, or CGNAT host address that the operator did not allow (Model endpoints on your own network). It checks again at connect time to prevent DNS rebinds between save and connection.
“Provider host <host> is not in the allowed list”	Instance settings define an egress allowlist without this host. A blank allowlist allows any public host.
— (HTTP 403) “This server does not allow you to connect your own AI provider.”	A workspace admin tried to add a connection or model with Let workspaces add providers and models off.

CAUTION: Keep the loopback exception out of production

HEPHAESTUS_LLM_EGRESS_ALLOW_LOOPBACK=true permits http://localhost base URLs for local development. It lets a workspace admin direct a "provider" to host-local services. Never set it in production. It does not relax the private-address check.

Model endpoints on your own network

By default, a provider base URL must resolve to a public address. An admin who configures a provider must not be able to send requests into your private network. To keep model calls inside your network, for example to vLLM or Ollama on your own GPUs, allow the endpoint explicitly.

Set HEPHAESTUS_LLM_EGRESS_PRIVATE_HOSTS on the server and on the worker to a comma-separated list. On Kubernetes, set config.llmPrivateHosts (A model endpoint on a private network). Each entry is one of these:

Entry	Example	Allows
A host name	vllm.lab.example	This exact name, at any private address that it resolves to. Use only a name whose DNS you control. Wildcards are not supported. A host name is not checked against the addresses of this deployment, so never list the name of a Hephaestus service.
An IP address	192.168.5.9	Any host name that resolves to this address.
A CIDR range	10.20.0.0/16, fd00:1::/48	Any host name that resolves into this range. It can be no wider than /16 for IPv4 or /48 for IPv6.

These rules apply:

• Only private addresses qualify: 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 100.64.0.0/10, and fc00::/7. Loopback, link-local, and public addresses stay refused, even when a host name entry resolves to them. Every cloud metadata service at a 169.254.x.x address is link-local, so it stays refused.

• These metadata and credential ranges inside the private ranges also stay refused:

  Range	Provider
  100.100.100.200	Alibaba Cloud
  fd00:ec2::/32	AWS: instance metadata, EKS Pod Identity, DNS, and time
  fd20:ce::/48	Google Cloud
  fd00:a9fe::/32	Akamai (Linode)
  fd00:42::/32	Scaleway
  fd00:c1::/32	Oracle Cloud

  No such list is complete, so a range must also be narrow, as the next rules say.

• An entry stops startup if it is not a host name, an IP address, or a CIDR range. An IPv4 address needs all four parts: 10.1 is refused.

• A range stops startup if it is not inside one of the private ranges, for example 0.0.0.0/0 or 172.0.0.0/8.

• A range stops startup if it is wider than /16 for IPv4 or /48 for IPv6, for example 10.0.0.0/8 or fd00::/8. List the network that the model server is in.

• A range stops startup if it is inside a metadata range of the previous table, because it would never connect.

• A range stops startup if it covers an address of the server or worker itself. Such a range admits the services of this deployment, such as PostgreSQL, NATS, and the server. On Compose, this check covers the Compose network. On Kubernetes, it covers the address of the Pod, but not the whole Pod range or the Service range. Thus, never list a range that covers the Pod or Service range of the cluster.

• List single hosts or narrow ranges. A range lets admins send requests to every port of every host in it. These admins include the workspace admins of every workspace while Let workspaces add providers and models is on. A typical vLLM or Ollama server needs no key. Any workspace can then use it as its own provider. Its use does not count against a shared model's budget. Turn that setting off to keep the list to the connections of instance admins.

• For a host on this list, a failed Test and fetch models shows no HTTP status and no error. The server log has them.

• For a local development setup, use HEPHAESTUS_LLM_EGRESS_ALLOW_LOOPBACK.

• The check happens again at connect time, on the address that the connection uses. A host name that is not listed and that changes its DNS answer to a private address is still refused.

• Plain http:// stays refused. To permit it for listed hosts, set HEPHAESTUS_LLM_EGRESS_PRIVATE_HOSTS_ALLOW_HTTP=true. A host is listed if its name is an entry, or if the list covers all its addresses. The API key then crosses your network unencrypted.

• The instance allowlist in the admin UI still applies on top of this list.

• No admin UI shows or changes this list. Only the operator of the deployment sets it.

3. What "Test and fetch models" does

The probe sends one credentialled GET {baseUrl}/models. It needs no separate authentication by you. It uses Accept: application/json and your key in the header selected by the authentication mode. It sends no request body or query parameters and calls no other endpoint.

It reads data[].id from the response. It returns up to 1 000 ids and skips ids longer than 256 characters. It discards everything else the provider returns.

The probe has these limits:

• It stores nothing. It stores no discovered ids, last-tested timestamp, or health flag. The ids supply suggestions in your browser's model form and disappear on reload. You still create each model explicitly.
• It does not follow redirects. A 3xx fails. Only your entered URL passed the egress check. A redirect could send your API key to the Location host.
• It does not validate the API. It always calls /models, whatever API the connection declares. A green probe confirms the endpoint exists and accepts your key, not that a completion works.
• It does not hold a database connection. It runs outside a transaction. A stalled provider cannot occupy the pool while admins probe it.

Connect and read timeouts are each 5 seconds. The probe refuses a response body over 1 MiB and reads no further.

The instance form can probe an unsaved connection. For that draft probe:

1. Enter the base URL.
2. Enter the key.
3. Select Test and fetch models.

The probe uses the credential for that call only and never persists it. Workspace admins have no draft probe. They can test only a connection already saved.

On edit, the exact button label is Test saved connection until you enter a new key. Then it is Test changes. The first uses the stored credential, and the second uses the key on screen. A change to the base URL, authentication mode, or key discards a result in progress. A name change does not.

Reading a probe result

A probe always returns HTTP 200. An upstream failure is data, not an error response. A red banner therefore does not mean Hephaestus is broken. The table preserves the exact result text:

What you see	What it means	What to do
“Reachable. Found N models.”	2xx, with data[].id parsed. N can be 0. A 2xx without a data array is still reachable.	Nothing.
“The provider answered with HTTP 401 / 403.”	The endpoint answered. Your key is wrong, expired, or lacks permission.	Re-enter the key. On Azure, check for API key, not Bearer.
“The provider answered with HTTP 404.”	The host is correct but the path is not. Usually, the base URL lacks /v1 or has an extra path segment.	Correct the base URL. Because it is immutable, create the connection again.
“Hephaestus could not reach the provider. The server log has the reason.”	The provider is on the operator's private host list (Model endpoints on your own network), and the test failed. The test does not show the cause for these hosts.	Ask the operator to read the server log.
“The provider answered with HTTP 3xx.”	The probe refuses a redirect.	Enter the URL the provider serves from.
“Hephaestus could not reach the provider (<ExceptionName>).”	Nothing answered. Causes include DNS failure, refused connection, TLS failure, a 5 s timeout, or the connect-time address guard. The guard rejects the resolved address. The exception message is withheld because it can contain host details. Only the class name appears.	Check the host from the server, not your laptop. The server log has the same class name at INFO.
“The provider’s response was too large.”	/models response over 1 MiB.	Almost certainly not an OpenAI-compatible /models endpoint.
“We could not fetch the model list.” (any failure)	The banner headline for all these failures.	Save the connection if needed. Enter the model ID manually. A failed probe never blocks anything.

A workspace admin sees a narrower result:

Connected. N models available.

Failures show the same message, but provider model IDs never appear.

Many endpoints for practice scripts do not serve /models. The probe of such a connection can get HTTP 404, 405 or 501. Then the result shows this text instead of a failure:

No model list at this address. Add the model by its ID. If its calls fail too, check the base URL.

A wrong path in the base URL also gives HTTP 404. Thus, the text tells you to check the base URL when the calls of the model also fail.

Any other failure of such a probe, such as HTTP 401 or no answer, shows as a failure.

Reasoning effort

Reasoning effort sets how hard a reasoning model thinks on every review or conversation call. It uses OpenAI's scale:

• None.
• Minimal.
• Low.
• Medium.
• High.
• Extra high.
• Max.

Chat completions receive reasoning_effort. The responses API receives reasoning.effort. Provider default sends no effort value, so the model's provider default applies. Defaults and supported levels differ by model. Check the provider's model documentation.

None asks a reasoning model not to reason. For a non-reasoning model, use Provider default, because the provider may reject the parameter. Not every model accepts every level. A refused level fails the call rather than changing to a supported level. The same field exists on workspace-owned models.

Higher efforts use more output tokens. Each job's usage records the reasoning share and prices it as output, never twice. A model marked as supporting reasoning before this field existed keeps its previous value: Medium.

Models that think by default

Some model servers make a model think unless the request turns thinking off with a field of their own. An example is Qwen3 on vLLM, which reads chat_template_kwargs.enable_thinking. Hephaestus sends only the standard reasoning effort: reasoning_effort on Chat Completions and reasoning.effort on the Responses API. Such a chat template ignores that field. Thus, Provider default and None do not turn the model's thinking off, and Hephaestus cannot turn it off for one request. This also applies to the summary that Hephaestus writes when a conversation becomes long.

A model that thinks on every call can spend its whole output limit on thinking, and the turn then fails. To turn thinking off, set the default on the model server. For vLLM or SGLang, set the server's default chat template arguments so that enable_thinking is false.

Gateways that buffer answers

Symptom: Hephaestus shows no text for a long time and then the whole answer at once. Or a Hephaestus answer fails after 2 minutes with no text, but the provider's log shows a complete answer.

The worker's LLM proxy fails a conversation model call whose stream sends no byte for 2 minutes. It also fails a call whose provider sends no response headers for 5 minutes (Runtime roles). A gateway that buffers the whole answer sends nothing until the answer is complete. Configure every proxy between Hephaestus and the model to stream:

• For nginx, set proxy_buffering off on the completions path.
• Or let the upstream send the response header X-Accel-Buffering: no, which turns buffering off for that response in nginx.
• For other gateways, turn off response buffering for text/event-stream, or send SSE keep-alive comments while the model works.

A provider whose calls stall or fail repeatedly is paused for 60 seconds (Provider breaker).

The smallest conversation model

A conversation model needs a context window of at least 32,768 tokens and an output limit of at least 4,096 tokens. Its fixed instructions, tools, and situation take about 9,000 tokens. A conversation summary is one reply of the model.

Hephaestus refuses to use a model whose stated limits are smaller, with a message that names the limit. A model that states no limit is not checked. If a bound model's limits change below these values, each conversation turn fails at its start. The developer then sees that their conversation model is too small.

4. Pricing a model

Hephaestus never infers a price. OpenAI-compatible responses report token counts, not your account's dollar charge. Rates differ by contract, region, and commitment. Every model therefore needs a cost declaration.

Each model has one of these states:

• Price per 1M tokens (PRICED): you supply the rates below.
• No metered API cost (NO_CHARGE): declares a real $0, counts as confirmed $0 spend, and satisfies a cap. Its usage is a confirmed $0 also when the provider reports no tokens. It requires a note with the reason, such as an internal endpoint with infrastructure billed separately. Infrastructure can still cost money. This declaration covers only per-token API charges.
• No price set (UNPRICED): makes no cost declaration. Usage is recorded but excluded from monthly totals. A capped purse with unpriced monthly usage reads UNVERIFIABLE and pauses exactly like an exhausted purse. You cannot enable a model in this state.

The rate fields

Four rates, all US dollars per 1,000,000 tokens, stored as NUMERIC(18,8):

Field	Required for PRICED	Notes
Input (USD)	yes	Prompt tokens that are neither cache reads nor cache writes.
Output (USD)	yes	Completion tokens. Reasoning tokens are already inside this count and are not priced a second time.
Cache read (USD)	no	Prompt-cache hits, where the provider reports them.
Cache write (USD)	no	Prompt-cache writes, where the provider reports them.

The server enforces these rules with the exact messages below:

A price needs at least an input rate and an output rate (per 1M tokens).

A price needs at least one rate greater than zero. For a model without usage charges, choose No metered API cost.

Rates must be zero or greater.

Rates must also be below 10000000 per 1M tokens. This is not the column's limit. Above it, a rate cannot pass through the browser's binary64 representation and return unchanged.

You can set rates only when the model has a price. Clear the rates, or set a price first.

Repricing replaces a price, never edits it. The instance catalog keeps price history. A new price closes the current row's effective_to and creates another row. Past runs retain their actual billed rates. A price change never rewrites history.

A workspace's own model stores its price inline, without history. The workspace owns both model and provider account. There is no cross-party record to preserve.

An omitted rate can make the whole call unpriced. If a run reports tokens in a bucket with a null rate, the entire event is unpriced. For example, this happens with cache-read tokens if you priced only input and output. The ledger does not charge only the priced buckets. A partial charge would falsely present a confirmed total below the real amount. If your provider reports cache tokens, price them.

Missing prices and unverified measurements

A usage record needs both an accepted measurement and usable rates to have a confirmed cost. Measurement verification records whether source selection accepted one whole token vector. It does not certify that every provider call was observed. When runner and proxy totals are incomparable, the selected vector is retained, but its cost stays unavailable. This applies even to a model declared to have no metered API cost.

Adding missing rates can repair only verified measurements with positive billable tokens. It cannot repair an unverified measurement.

Historical records without a recorded verification decision remain unknown. Their existing tokens, rates, and costs are retained. Historical unpriced records with an unknown decision stay unpriced after rates are added. Do not edit the database to assign a verification decision or clear a budget pause. Unpriced records continue to affect capped purses as described in Integration settings.

Upgrade all runtime roles together using the stopped-writer procedure. An older repricer can bypass measurement verification during a mixed-version rollout.

Worked example

The example model has these rates per 1M tokens:

• Input: $2.50.
• Output: $10.00.
• Cache read: $0.25.
• Cache write: unpriced.

One completed run reports 12 345 input, 3 210 output, 40 000 cache-read, and 0 cache-write tokens. Each bucket uses tokens ÷ 1 000 000 × rate:

Bucket	Tokens	Rate / 1M	Amount
Input	12 345	$2.50	0.012345 × 2.50 = $0.0308625
Output	3 210	$10.00	0.003210 × 10.00 = $0.0321000
Cache read	40 000	$0.25	0.040000 × 0.25 = $0.0100000
Cache write	0	—	0 tokens, so $0 — an unpriced bucket with no tokens is not a missing rate
		raw	$0.0729625

The ledger column is NUMERIC(18,6). It rounds the total to six decimals with banker's rounding. The seventh digit is exactly 5, and the sixth (2) is already even. The stored amount is therefore $0.072962, not $0.072963. Half-to-even rather than half-up rounding prevents a long sequence of amounts from drifting upward against you.

Two further limits apply:

• A tiny paid call never stores $0. One input token at $0.05 / 1M costs $0.00000005. Six-decimal rounding would make it zero. The ledger stores $0.000001 instead to distinguish a paid call from a free call. The excess charge is less than a millionth of a dollar per call.
• The UI rounds differently from the ledger. Published rates show up to four decimals ($0.075 / 1M) for price-list checks. Spend appears as cents, with <$0.01 for a non-zero amount below one cent. Each cell of an average column, such as Avg per run, has the same decimals. Thus, the decimal points line up. The column shows whole cents when no average in it is below one cent. If an average is below one cent, the column shows three decimals, such as $0.004. If an average is below a tenth of a cent, it shows four, such as $0.0004. Thus, a small unit cost stays visible. An average above zero but below $0.00005 shows <$0.0001. No display is the stored number.

Set HEPHAESTUS_LLM_DISPLAY_CURRENCY=EUR to show a converted amount alongside USD. This is display-only, never an input to a price, budget, or ledger. EUR is the only accepted value. Any other value fails startup with an explanation of accepted values. An unsupported currency cannot silently appear as a missing feature.

5. Sharing an instance model

A model is granted by default. Only named workspaces can use it. To share it with every workspace, switch it to public. This also clears the grant list.

A revoked grant or disabled connection stops every workspace that has an assignment to that model. The assignment stays in place and appears correct. This is a common cause of silently stopped workspace reviews. If that happens, use the triage list.

A workspace admin who chooses a model sees its display name, connection name, reasoning effort, and price framing. They never see an upstream model ID, base URL, or credential.

6. Sharing provider capacity

Put models in a call pool when they share one provider quota boundary. The pool enforces capacity and rate accounting across all workers and workspaces. A shared host name does not prove that credentials or deployments share a quota. A model without a pool has no endpoint-wide limit.

Use the instance admin API under /api/admin/llm/call-pools:

1. Create a pool with its slug, display name, maximum concurrent calls, and interactive reserve.
2. Set requestsPerMinute and tokensPerMinute when the provider imposes these rates.
3. Set tokensPerCall to a conservative maximum input-plus-output charge when you configure a token rate.
4. List models with GET /api/admin/llm/call-pools/models.
5. Assign models with PUT /api/admin/llm/call-pools/{id}/models.

Use provider documentation to establish the token bound for every assigned model. Include reasoning and server-side tool tokens when the provider bills them. An understated bound cannot protect the provider's token quota. The pool reserves the full bound for each physical call, including cancelled calls and retries. This conservative policy can admit fewer requests than an exact tokenizer would permit.

Use 32 concurrent calls and an eight-call reserve only when the provider supports that capacity. The reserve must leave nonzero background capacity. Reviews, composition, and direct script calls never borrow the reserve. The rate limits also keep a proportional interactive share. The runtime guide explains fairness and call closure.

Configure membership while the affected models are idle. Pool settings and membership cannot change while their pool has open requests, recent rate debits, or a shared cooldown. Wait at least one minute after the last call closes. The slug cannot change. A workspace admin can read their model's membership but cannot change it. Deleting an idle pool leaves its models without endpoint-wide limits.

Only an instance-controlled credential can pause a shared pool after a provider rate limit. A workspace-controlled credential's rate limit reaches that caller without pausing other tenants.

7. Errors you can hit

In addition to the HTTP 400 responses above, the catalog returns these RFC 7807 problem types. The type slug is an API contract. The UI shows the title.

Status	type	When
409	llm-connection-in-use	A connection still has models. Delete the models before the connection.
409	llm-model-in-use	A model still has an assignment. Assign another model in that workspace before deletion.
409	llm-connection-slug-conflict	Your supplied slug is taken.
409	llm-call-pool-in-use	The call pool still has open requests, recent rate debits, or a shared cooldown. Wait for these to end before changing settings or membership, or deleting the pool.
409	llm-call-pool-slug-conflict	The pool slug is taken.
409	llm-model-slug-conflict	The model slug is taken on this connection.
409	llm-model-upstream-id-conflict	Two catalog entries would use the same upstream id on one connection. This is refused because price selection would be nondeterministic. A NO_CHARGE copy could silently replace a PRICED model for billing.
409	agent-binding-slot-mismatch	A model's declared tier does not match the assignment row. The body's declaredTier names the model tier, which the UI uses to explain the refusal. An undeclared model has no declaredTier, only the detail.
404	Resource not found	Unknown connection or model id.

Next

• Budgets, caps, and what a cap pauses: Workspace budgets and alerting.
• The environment variables that are deploy-time facts: Instance-wide LLM settings.
• Nothing is being reviewed: When a workspace goes quiet.
• Models for practice scripts: Models for practice scripts.

Data handling and members' AI choices

Owners configure Administration → Member onboarding after the workspace's existing mechanism manages membership. Members can see display names of models ready for their answer in that workspace. An optional model Model maker names who made it. An optional connection Service receiving requests names the service that receives requests. Their marks appear beside the actual model only in this workspace. Leave either blank if you cannot confirm it.

Neither field sets the data-handling tier. Operated by does. An OpenAI model served by Azure can show both marks. A private endpoint alone does not mean your organization operates the model. Microsoft-managed Foundry inference is provider-operated. A model your organization runs on its own Azure machines can be declared in-house.

Logos is the AI service in Edutelligence, with its own webapp and mark. It is not an Edutelligence brand choice. Your organization or a provider can run it. Select the Logos mark only if this connection receives requests through Logos. Declare Operated by from your deployment and agreement, not that mark. Hephaestus does not infer these facts from a model ID or connection URL.

The setup page is hidden by default. Before you enable it:

1. Choose display names suitable for developers to read.
2. Declare models.
3. Before tier assignments, wait until every runtime role has been upgraded.
4. Assign models.

For each model, configure its declaration:

1. In either model console, open Data handling.
2. Declare Operated by as Your organization or A provider under terms your organization accepted.
3. Before you enable the model, check provider retention and training terms with your provider.
4. Verify the endpoint, gateway routing, subprocessors, and data-handling agreement.
5. Declare the agreement's facts, not assumptions from the hostname.
6. In the admin-only note, record the region, agreement, renewal date, and retention details.

The declaration does not verify provider retention or training terms. From Operated by, Hephaestus derives In-house or Cloud. It shows that tier as a badge wherever the model appears. Token limits and the reasoning switch are under Advanced.

In workspace AI models, each purpose has three rows:

• In-house.
• Cloud.
• Members who have not chosen.

A row holds only a model declared as that tier. A model from another tier is refused with this exact text:

This model is declared as \{tier\}. Assign it under \{tier\}.

An undeclared model is refused with:

This model’s data handling is not declared yet. Declare it first, or assign it to Members who have not chosen.

If a model's declaration becomes less restrictive, it no longer matches its row. It reads Not ready and stops serving until you reassign it. Nothing is substituted. Grants, prices, funding sources, the practice-review switch, and budgets still apply. Max concurrent runs applies independently to each row.

For workspace member setup:

1. Assign a matching model to each required purpose and tier row.
2. Optionally require Slack or Outline account links under Member onboarding.
3. Turn on Ask members to set up on their first visit.

A required link needs an active integration and an enabled account-linking provider for the same server. Slack also needs the configured team. Outline needs the configured instance identity. Once the page is enabled, personal AI activity requires an explicit member choice. It asks only members who have not answered anywhere on this instance.

A member who chose in another workspace sees it only for a required account link they have not connected. Unknown or unlinked developers are not assumed to have agreed.

A member's choice is a ceiling: the least restrictive tier they accept. It belongs to the account, has one answer, and applies in every workspace they belong to on this instance. Members can change it from Your AI choice in any workspace sidebar or User settings. The In-house row serves In-house choices. The Cloud or In-house row serves Cloud choices.

In each workspace, Hephaestus selects the least restrictive ready row within the ceiling, never a less restrictive one. If no ready row fits the ceiling, that member gets no practice reviews or conversation answers there. Their choice page states this in one sentence below the answers. Members may choose any answer, including one no row covers yet.

Model and provider additions, removals, or new declarations never ask members again. The ceiling is checked on every request. A model with a less restrictive declaration stops serving until reassignment. Members see setup again only after you change Member onboarding itself. Only members who skipped setup or still have a newly required link see it again.

Members who have not chosen preserves existing assignments in workspaces that never required member choices. It serves only members without a choice where choice is optional. It is never a fallback for someone who chose. Existing models upgrade as Not declared, with no declaration. Existing assignments go into that row. Hephaestus does not infer data handling from names or endpoints.

A reviewed developer without a resolved account counts as not having answered. This includes developers who never signed in or have an unlinked identity. The optional-choice row serves them where choice is optional. Only a workspace with setup enabled, which requires a choice there, keeps AI off their work. Reconnect the developer's identity to restore their own choice.

This does not change another identified member's choice. Members may choose No AI even if model configuration or required account links are unavailable. It stops new practice reviews about them and Hephaestus answers to them in every workspace. It does not stop membership or access to existing feedback. A review checks the developer whose work it reviews, not the requesting admin.

A changed choice is checked again at execution and before each model request. It is also checked before acceptance or delivery of new feedback. An outbound request already in progress cannot be recalled. The choice does not remove shared source content from other members' reviews.

Changing or pausing setup

To make a required account link optional, clear its requirement. A disconnected required integration stays visible as unavailable until repair or removal of its requirement. It is not silently waived. A disabled sign-in provider does not erase a verified matching link.

A hidden setup page does not clear saved choices or restore unrestricted AI for members without a choice. The page shows a reminder while that is the case. Members can still open Your AI choice in the sidebar. A disabled model or changed data handling stops incompatible new requests. It never substitutes a less restrictive tier. An assignment switched off and on does not change a member's choice.

The schema upgrade is additive except for an assignment uniqueness key widened to include its tier. Rollback is supported before onboarding settings, member choices, declarations, or tier-specific assignments have been saved. After use, rollback is deliberately refused. Loss of a saved refusal could restore unrestricted processing.

CAUTION: Preserve saved AI refusals

A lost saved refusal could restore unrestricted processing. Recover through a verified pre-upgrade backup. Do not delete member choices to bypass the safeguard.

Replace an unavailable model

When a provider withdraws a model:

1. Register its replacement disabled.
2. Set its price.
3. Set its data-handling declaration.
4. Test the connection.
5. Enable the replacement.
6. Update each affected practice-review and conversation model assignment, including separate AI-choice tiers.
7. Disable the old model so new requests cannot select it.

Do not rename the old model's provider ID to the replacement. Historical usage must retain its model identity. An assignment to a disabled model stays visible but cannot run. It never silently selects another handling tier.

Before you delete a model, remove or replace its assignments. The server refuses deletion of an assigned model. Provider unavailability does not remove recorded feedback or earlier conversations.

Connect Anthropic

Choose Anthropic as the endpoint preset. The preset selects Messages API (Anthropic) and https://api.anthropic.com. Enter an Anthropic API key. The server sends the key in x-api-key and sets anthropic-version to 2023-06-01. The connection test reads the native /models endpoint.

The Messages API serves reviews, practice script chat models, feedback composition, and conversations. It permits custom tools that run in the sandbox. It refuses provider-hosted tools and remote MCP servers. Bedrock and Google Cloud authentication are not supported by this preset.

Anthropic reports uncached input, cache reads, and cache writes as separate counts. Set each token price in the model's price fields. The cache buckets are not subtracted from the uncached input. The server retains cumulative usage if a stream stops before its final event.

The runner caches the stable system, change, and core layers for five minutes. The session marker moves as the review continues. ADR 0069 owns this layout.

Native feedback composition uses the provider's defaults for thinking and sampling. It does not apply the review runner's effort or temperature overrides. This supports opaque model aliases without a second model-capability catalog. Pi uses its provider metadata for native model IDs that it recognizes. An unknown native alias keeps provider defaults for those options.

This protocol alone does not prove a cache gain.

Test model tool choice

Edit a saved model in either admin console. Select Test tool choice. The test sends two short synthetic requests to that model. The provider can charge for them. The result is saved at once, without Save changes. No reviewed work is sent.

The first request uses automatic tool choice. The second requests one named tool. A returned call to that tool confirms the forced_tool_choice capability. A refusal of the second request records no support only if the first request succeeds. A failed or inconclusive test keeps the previous capability.

Unknown capabilities use automatic tool choice. Model names and aliases do not determine capabilities.

Anthropic review sessions always use automatic tool choice, including repair turns. The admin form does not offer this test for native Anthropic models. This keeps their tool choice stable across cache reads. Other chat protocols can force the submission tool after a successful test. Repeat the test after the provider changes the model behind an alias.

The endpoints are PUT /admin/llm/models/{id}/capabilities and PUT /workspaces/{slug}/llm/models/{id}/capabilities. Each returns the observed forcedToolChoice value. An inconclusive test returns no value and changes no stored capability.

Models for practice scripts

A practice's script is the review program of that practice. It can prepare the places that the review must check, or decide parts of the practice itself. A script can ask models to rate or find those places. It never writes feedback, and the server checks every result before it records it. Write a practice script tells how to write one.

A script can use four kinds of model. The chat model is always the practice review's own model. You assign the three other kinds on AI models, under Models for practice scripts:

Purpose	API in the connection form	apiProtocol	Endpoint the proxy calls	Example providers
Decision model	Decisions API	openai-decisions	{baseUrl}/decisions	The OpenAI Decisions API
Decision model	Chat Completions API	openai-completions	{baseUrl}/chat/completions	A chat model that returns top_logprobs, such as a vLLM deployment
Embedding model	Embeddings API	openai-embeddings	{baseUrl}/embeddings	OpenAI, Azure OpenAI, vLLM
Reranking model	Rerank API (Cohere-compatible)	cohere-rerank	{baseUrl}/rerank	Cohere, Jina, vLLM

The Decision model accepts a model on a Decisions API or a Chat Completions API connection. With a chat completions model, Hephaestus reads each answer from the token log probabilities. That model must return top_logprobs. If the model has a reasoning effort, Hephaestus sends reasoning_effort: none for these calls, and the model must accept it. The model must give its answer without reasoning first. A model that always reasons, such as gpt-oss, cannot serve this purpose, and its calls stay not rated.

These purposes are optional. If you assign none of them, nothing breaks:

• Practice reviews run without them.
• A script that requires a missing model does not run. Its practice has no result in that review.
• A script that can work without the model continues without it, and tells the review what it could not rate.

The bundled practice scripts call none of these models. Assign one when a customized practice script uses it.

A host that loads models on demand can take minutes to load a model after a pause. A call from a practice script that waits longer than its script's deadline is not rated. On such a host, keep the models for practice scripts loaded, or expect calls that are not rated after a pause.

Assign a model for practice scripts

1. Create the connection. An instance admin uses Instance admin → AI models. A workspace admin uses Your providers at the bottom of the workspace's AI models page. Under each provider's name, an icon for each kind of model that its API serves comes before the API's name.
2. In API, choose the API from the table above. The three APIs for practice scripts are under For practice scripts.
3. Do steps 2 to 6 of the staged flow. These steps test the connection, add and price the model, and enable both. For an instance model, also do step 7 and share it. If the probe cannot list the models, enter the model ID yourself.
4. Open the workspace's AI models page. Under Models for practice scripts, open the row of the purpose, such as Embedding model.
5. Choose a model for each tier that must have one, and select Save assignment.

Each purpose offers only the models that can serve it. A model that you add under Your providers appears in the list at once. When no model can serve the purpose, the model list tells you what to do:

• If the workspace can add its own providers, it sends you to Your providers. An example is No embedding model yet. Connect a provider with the Embeddings API under Your providers, then add its model. The link Your providers scrolls to that section.
• If not, it tells you to ask an instance admin to share a model.

The list also gives two warnings:

• For a decision model with a reasoning effort: a decision needs a model that answers at once with token probabilities.
• For a reranking model without No metered API cost: some providers bill search units or report no tokens. Their calls have no price. For a self-hosted reranker, set No metered API cost on the model. The workspace sets it on its own model. On a shared model, only an instance admin can set it, so the warning tells you to ask one. See the caution in Usage by practice.

A Cloud model of a purpose for practice scripts is not used while Practice reviews has no ready Cloud model. Then its form shows this note:

Cloud members’ reviews run on an In-house model, so this model is not used. Assign a Cloud model for practice reviews to use it.

Privacy gives the rule.

Through the API, the connection field is apiProtocol, with the values in the table. The server refuses a model of the wrong kind for a purpose with HTTP 400. For example:

This is a chat model. Choose an embedding model here.

Which practices use a model for practice scripts

A script declares its models in its code, as required or optional. Hephaestus reads that declaration only when a review runs the script. Thus, a practice's needs appear only after a review in which the script declared its models. A script that fails before it declares them says nothing about them. After a change to the script, the needs appear again after the next review.

Each purpose row under Models for practice scripts shows these facts:

• The collapsed row: one column for each tier, In-house and Cloud. While the member choice is optional, a third column, Not chosen, shows the model for members who have not chosen. A cell shows the model that serves that tier, or "—" when none does. A house icon after a model marks an In-house model that serves a less strict tier. Not used after a muted model in the Cloud cell means that Cloud members do not get it. Their reviews run on an In-house model.

• Below the purpose name: Used by 2 practices or No practice uses it.

• The status badge, only when the row needs a decision:

  • Not set for some members: some members get no model for this purpose. The tier columns show which.
  • Not set: no member gets a model for this purpose.
  • Needs attention: an assigned model cannot run, or a practice requires a model that some members do not get.
  • Off: practice reviews are off, so no model for practice scripts is used.

  While practice reviews are on, a purpose that serves every member shows no badge. A purpose for practice scripts that no practice uses also shows no badge then, whatever it assigns.

• Used by, when you open the row: each practice that uses the model. A practice that requires the model shows Required. A practice that uses the model only when it is available shows no badge. The practices whose required model some members do not get come first. Below each of them, a line such as Not set for In-house members names the members without a model. The note As of each practice’s newest review. tells you where the list comes from. The list shows five practices, and Show all shows the rest.

A row with Needs attention opens when the page loads. Practices can require the model while some members do not get one. Then the row shows a warning with the count of those practices and the tier, for example:

2 practices need a decision model for In-house members. Until you assign one, their practice scripts do not run, and those practices have no result.

Used by names the practices. Until you assign a model for that tier, the script does not run for those members' work.

Add ?purpose= to the address of AI models to open one row and scroll to it, such as ?purpose=PRACTICE_EMBEDDING. The values are PRACTICE_REVIEW, MENTOR, PRACTICE_DECISION, PRACTICE_EMBEDDING and PRACTICE_RERANKING. Links such as Assign a decision model and Check the decision model use this address.

The same needs appear for one practice under Practice setup:

• An alert at the top says when some members do not get a model that the script requires. An example is The practice script does not run for In-house members’ work: no decision model is assigned for them. Assign a decision model opens the purpose row.
• The item Practice script shows the needs above the script's code. Each need is one row with the kind of model, such as Decision model. The review's own chat model has no row, because it always runs the review. A model that the script requires shows Required. A line with tier icons names the members without a model, such as Not set for In-house members. A link beside the line opens the purpose row.
• It uses no model. when the script declares no model.
• It uses only the review model. when the script declares only the chat model.
• Its models show after its next review. when no review has run the script, or the script failed each time before it declared its models.
• The script changed. Its models show after its next review. when the newest review that reported the needs ran an earlier version of the script.

Below the needs, As of the review on, with a date such as 3 Oct, opens the review that reported them. Practice review operations tells you how to read what a script did in one review.

Privacy

A model for practice scripts follows the same data-handling rules as the review model. For each review, Hephaestus chooses each model for practice scripts through the reviewed developer's AI choice, like the review model. A model for practice scripts is at least as strict as the review's own model. If a developer's review runs on an In-house model, the scripts of that review use only In-house models. The description of Models for practice scripts on AI models states this rule.

If no row of a purpose fits, the slot stays empty, and the review runs without it. The proxy checks the developer's AI choice again before each call.

A script sends parts of the reviewed work to the model, such as added code and comments. Declare Operated by for a model for practice scripts with the same care as for a review model.

Cost and limits

Price a model for practice scripts like any other model. Its calls count against the shared-model budget or the provider cap of the purse that pays for it.

Model	How Hephaestus bills it
Chat	On the review's own usage row, with the review's price.
Decision, embedding, reranking	One usage row for each kind and review attempt, with the price frozen when the attempt was admitted. The source types are PRACTICE_SCRIPT_DECISION, PRACTICE_SCRIPT_EMBEDDING and PRACTICE_SCRIPT_RERANKING.

A cached prompt token costs the cache rate of its model, as on the review's own usage row.

One review attempt has two limits for the direct model calls of all its practice scripts together. A direct model call is a call that a script makes through one of its model slots. The review sessions that a script opens with ctx.review() are not direct calls. They count against the work budget of the review, as Practice review operations tells.

Limit	What it counts	Default	Environment variable and property
Direct model token limit	Input tokens, cached ones included, and output tokens of all direct calls, chat calls included	200 000	PRACTICE_REVIEW_DIRECT_MODEL_TOKENS_PER_ATTEMPT, hephaestus.practice-review.direct-model-tokens-per-attempt
Direct model search unit limit	The search units that direct reranking calls billed	1 000	PRACTICE_REVIEW_DIRECT_MODEL_SEARCH_UNITS_PER_ATTEMPT, hephaestus.practice-review.direct-model-search-units-per-attempt

Only an operator can change them. Some reranking providers, such as Cohere, bill a call in search units and report no tokens. A search unit is not converted to tokens, because one search unit can hold a few tokens or many thousands. Thus, search units have a limit of their own.

The shared-model budget and the provider cap show on AI usage. The two direct model limits also bound spend on a workspace's own provider that has no provider cap.

The practice program host checks the direct model token limit before each call. It cannot know the search units of a call before the provider answers, so only the proxy counts them. The proxy enforces both limits. When a direct model limit, the shared-model budget or the provider cap is spent, the proxy refuses further direct calls with HTTP 402. The scripts then report the rest as not rated with budget.

The proxy counts only calls that have returned. Thus, each call that is in flight when a limit is reached can pass. The attempt can then end above the limit by the usage of those calls. The same applies to the shared-model budget and the provider cap. The host sends at most 4 direct calls at the same time.

The proxy also limits the model calls in flight for each attempt (Runtime roles). Thus, after a limit is spent, at most that many calls can still pass: 12 for a review at the default practice concurrency.

Each call from a practice script names the practice whose script makes it. The proxy refuses a call without a practice with HTTP 400. The workspace ABI gives the header, and Practice review operations gives the metric that counts these refusals.

Usage by practice

The workspace's AI usage page has four sections: Spend this month, By run type, Models for practice scripts, by practice, and By day. For an earlier month, the first section is Spend in and the month, such as Spend in September 2026. The first section shows one tile for each purse: Shared models and Own provider.

The Shared models tile shows the shared-model budget, and the Own provider tile shows the provider cap. A tile with a budget or cap shows it, such as of $25. It also shows a state when it applies: Paused, Near the budget or Near the cap, or No price set. A tile without one shows No budget set or No cap set. When the workspace does not use its own provider in the month, the Own provider tile is muted and shows $0. It tells you that work on a provider that you connect in AI models shows there.

In the current month, Set provider cap or Change provider cap in the heading of the section sets the cap of the workspace's own provider. The button appears only when the workspace uses its own provider in that month.

A workspace uses its own provider in a month when one of these conditions is true:

• The month has at least one call on a provider of the workspace. A call without a price or a call that costs $0 also counts.
• The month is the current month, and the workspace has an enabled model on an enabled connection of its own.

Thus, an admin can set the provider cap before the first call. A provider cap alone does not make a workspace use its own provider. The tables show Own provider columns only when the workspace uses its own provider in the month. On Instance admin → AI usage, the Own provider columns appear when at least one workspace uses its own provider in the month.

While the workspace does not use its own provider, its provider cap pauses nothing. Thus, no page shows a state for that cap, and Instance admin → AI usage shows no provider cap for the workspace. This is also true for a $0 cap.

Each table has its own money columns for Shared models and for Own provider. The two purses are never added together. Two tables show an average next to the spend of each purse:

• By run type shows Spend and Avg per run.
• Models for practice scripts, by practice shows Spend and Avg per review.

By day shows only the spend of each purse. The table of workspaces on Instance admin → AI usage shows the spend of each purse, the shared-model budget, and the provider cap. Usage describes its columns. On a narrow screen, the token and call columns are hidden.

No price set appears only when a row has usage without a price. In By run type and By day, it counts the runs that have usage without a price. A run is one review attempt or one conversation turn, with all of its usage rows. In Models for practice scripts, by practice, it counts reviews.

Models for practice scripts, by practice appears only in a month with decision, embedding, or reranking usage. On Instance admin → AI usage, the same table appears when you select Details for a workspace. The instance console has no table of practices across all workspaces. To see the practices of one workspace, open its details.

The table has one row for each practice:

Column	What it shows
Practice	The practice. A practice that the workspace no longer has shows its identifier and Not a current practice.
Models	The models for practice scripts that its script called: decision, embedding, or reranking. Each kind has its own icon.
Shared models: Spend, Avg per review	Its share of the cost on the instance catalog's models, and that cost divided by its reviews.
Own provider: Spend, Avg per review	Its share of the cost on the workspace's own provider, and that cost divided by its reviews.
Reviews	The reviews whose scripts called its models.
No price set	The reviews with a usage row without a price that it had a share in.
Input tokens, Output tokens, Calls	Its own counts from the proxy.

The rows show the highest spend first. Select the heading of any column except Models to sort by that column. The worked example tells how an amount and an average round.

With two rows or more, the table ends with a Total row. The total comes from the usage rows themselves, so the costs of the practices and Not attributed add up to it. A review can count for several practices. Thus, the Reviews and No price set of the rows can add up to more than the total.

A Not attributed row shows the cost that no practice can carry. An example is the usage of a review whose job was deleted. The row stays last, above Total, whatever the sort.

Hephaestus bills one usage row for each model kind and review attempt. It splits the cost of that row between the practices whose scripts made its calls:

1. Each practice gets a weight: its input tokens × the input rate, plus its output tokens × the output rate. The rates are those that the usage row applied.
2. Each practice gets the cost × its weight ÷ the sum of the weights, truncated to six decimals.
3. The practice with the largest weight also gets the remainder. Thus, the shares of one usage row add up to its cost exactly.
4. If no practice has a weight, each share is $0, and the table still shows the tokens.

A usage row without a price adds no cost to any practice. Its review counts one time in No price set for each practice that had a share in it. A model with No metered API cost adds a confirmed $0.

The table shows the usage of finished reviews, to date in the current month. These costs are already part of the run types above it. Chat calls from a script are not in this table, because they are part of each review's own cost. On By run type and By day, Runs counts review attempts and conversation turns only. A usage row of a model for practice scripts adds its calls and cost to its review, but it is not a run.

CAUTION: A priced reranking model that bills search units or reports no tokens

Some reranking providers, such as Cohere, bill search units and report no token usage. Hephaestus records the search units on the usage row. A model price has rates for tokens only, so Hephaestus records such a row without a price. A call that reports neither tokens nor search units is also recorded without a price.

In a purse with a budget or a cap, unpriced usage makes the month UNVERIFIABLE, and reviews pause. In such a purse, choose a reranking provider that reports token usage. For a self-hosted reranker, declare No metered API cost.