Configure product feedback

Configure product feedback to send reports about Hephaestus to the repositories where your maintainers work. GitHub filing is off by default. Until you name a repository, reports stay on the instance for instance admins to read.

Before you start

You need instance-admin access, a recent sign-in for destination changes, and permission to install the instance’s GitHub App. The Product feedback page is in the instance admin console. Workspace admins cannot configure it. GitHub is the only issue destination. GitLab, Slack, and Outline are not destinations for product feedback.

Complete the operator steps for product feedback before you name a repository. They cover lawful bases, the privacy notice, GitHub, the model provider, contracts, transfers, and the choice about publication. Redaction reduces risk but can miss personal data. A private report also leaves the instance when GitHub filing is enabled.

Configure destinations

1. Install the instance’s GitHub App on both destination repositories.
2. Give the app Issues: read and write permission.
3. Make the private repository private.
4. Use two distinct repositories with the same owner.
5. Open Product feedback in the instance admin console.
6. In Destinations, enter the private repository in Private repository.
7. Check that its installation is available and its visibility is private.
8. Choose an instance-level chat model in Model.
9. Set Daily draft limit for the instance.
10. After the private smoke test below, enter the public repository in Public repository.
11. Check its installation and visibility.

The default daily draft limit is 50. Each account also has a limit of five drafts per day. Failed model calls and retries count toward the draft budget. Choose a model with an available pool and provider connection. Use the routing evaluation to assess the model before public use.

The app acts as itself. Reports need no workspace connection or reporter GitHub token. The same owner permits Make private to move a public issue to the private repository. Before each private write, the server checks that the repository is still private. If that check fails or gives no answer, the write stops.

The card shows destination health, the draft count, the last filing time, and the seven-day route split by reason. A destination change is audited. It never redirects a report already queued for the previous repository. Those reports stay saved in the console, with the admin email if it is set up, and carry DESTINATION_CHANGED.

Hephaestus refuses to remove the private repository while a public issue can still be made private. It also refuses while erasure cleanup still has to move a public issue there. You can replace it with another private repository.

Inspect draft usage

The instance usage screen groups model spend by workspace. Product feedback drafts have no workspace, so that screen excludes them. A person's data export includes the calls for their reports.

For instance accounting, run this read-only query against PostgreSQL. It shows measured cost in USD and the number of calls whose cost remains unknown. An unknown call is not a free call.

SELECT date_trunc('month', occurred_at AT TIME ZONE 'UTC') AS month,
       count(*) AS calls,
       sum(input_tokens + cache_read_tokens + cache_write_tokens) AS input_tokens,
       sum(output_tokens) AS output_tokens,
       sum(cost_usd) FILTER (WHERE measurement_verification = 'VERIFIED'
                            AND pricing_state <> 'UNPRICED') AS measured_cost_usd,
       count(*) FILTER (WHERE call_state <> 'ENDED'
                        OR measurement_verification <> 'VERIFIED'
                        OR pricing_state = 'UNPRICED') AS unknown_calls
FROM llm_usage_event
WHERE workspace_id IS NULL AND source_type = 'PRODUCT_FEEDBACK_DRAFT'
GROUP BY 1
ORDER BY 1 DESC;

Route reports

Code selects the route after the model drafts the issue:

Route	Conditions
Public issue	The reporter allows publication. The report has a valid actionable draft for the outgoing revision. All privacy, security, safety, and confidence checks pass.
Private issue	The public gate does not pass and a private destination is set.
Saved report in the console, and the admin email if it is set up	No suitable destination is set, Pause is on, or filing stops.

Every public write checks the reporter’s current permission to publish and the frozen destination again. The first public filing waits five minutes after Send. Edit restarts that hold and repeats the checks. Keep private changes a held report to the private route.

Each private issue starts with a reason:

Reason	Meaning
opted out	The reporter did not allow a public issue.
security	The report can describe a security problem.
redaction	Code found a secret, credential, email address, or known person’s name.
about a person	The report concerns a named person.
low confidence	The report is unclear or is not an actionable issue.
abuse	The safety check found abuse.
no model draft finished	The model was unavailable, the limit was reached, the model response stayed invalid, or no draft finished within 10 minutes.

Security, blocking redaction, or a report about a named person sets a persistent risk latch. An edit or retry cannot clear it. Without a draft, the private issue uses the original report after code redaction. The public route never uses this fallback text.

The model is instructed to classify praise and support questions as non-issues. A misclassified report can still become public if the reporter allowed it and the other checks pass. The selected report kind alone does not keep it private.

The app uses Bug or Feature issue types where available. It adds no labels. A possible duplicate becomes a new issue with a reference to the possible match. The app does not comment on or close another issue automatically.

Recover reports that need action

Reports filed on GitHub are handled there. Needs action lists stopped reports with their reasons and Retry. It also lists erased reports whose GitHub issues need a manual step. See Handle erasure.

The console also retains reports that are not on GitHub, including instances without a destination. Instance admins can read their text and resolve or reopen them. Resolution in that list does not reply to the reporter.

1. Read the reason in Needs action.
2. Correct the destination, app access, or provider problem.
3. Select Retry for the report.
4. Check that one issue exists in the intended repository.

Automatic retries use backoff, with up to five attempts before manual action. After an uncertain GitHub response, the next attempt first looks for the issue by its marker. It writes again only when that issue is not there. Do not create another report or clear its state to bypass that check.

Every report that is not on GitHub stays saved in the console: no destination, or filing stopped. The main sidebar shows instance admins how many reports wait, and leads to the list.

Once a private repository is set, select File on GitHub on a report to send it there as a private issue.

Once a repository is connected, the list offers to move everything that waits, with Move to GitHub. A move that stops can run again: it continues with what is left, and no report gets a second issue. A report sent before this version told people about the AI model moves as written, and no model reads it. Filing keeps its pace, so a large move takes a while. The action needs a recent sign-in, and the audit log records who filed it. An admin can never publish a report: only the reporter can allow that.

The admin email announces it only if email delivery is set up. It also lists the erased reports' issues that need a maintainer, in one email per cleanup run. It carries the kind and a link, never the report text or attached context. Active instance admins need a verified address and the relevant email subscription to receive it. Without email delivery, the saved report remains available in the console. See Email setup for relay setup.

Publish a private issue

The reporter can select Allow public in Your reports. This permits publication but does not publish the report.

1. Confirm that the reporter allows public and that the report has a valid draft.
2. With write or admin permission on both repositories, comment /publish on the private issue.
3. Check for the app’s response.

The server checks the commenter’s current permission on both repositories at dispatch. Organization membership or a read-only collaborator role is insufficient. An unknown permission denies publication. The command can override confidence checks only. It cannot override the reporter’s choice, security, the risk latch, or abuse.

GitHub cannot transfer a private issue to a public repository. The app creates a public issue from the redacted draft, then closes the private issue with the public link. If GitHub's answer to the new issue is lost, the app looks for that issue in the public repository before it creates one again. The public issue contains no link back to the private one. The server checks commands every five minutes. During Pause, commands wait until you resume.

Handle erasure

The user guide describes reporter choices, emails, and withdrawal. Turning off the account default keeps held reports private but does not move issues already public. Make private moves each public issue to the private repository. Copies made while an issue was public can remain.

Account erasure and workspace purge remove the applicable reports and start external cleanup. Cleanup moves public issues to private, replaces their title and body with “Erased at the reporter’s request”, and closes them. It first reconciles any uncertain create, including a create with no known issue number. If no private repository is available, cleanup scrubs and closes the public issue where it is. The personal-data map owns the store and export inventory.

The app cannot delete an issue or its edit history. So each issue that cleanup scrubs becomes a manual step: its edit history still shows the erased text. Cleanup that has not finished after 30 days also becomes a manual step, and the app stops its own attempts. For each manual step, Hephaestus lists the step in Needs action with the issue link. It also sends the admin email if email delivery is set up.

If the app never learned the issue number, the step shows a marker instead. Search the repository for an issue whose body contains that marker.

1. Open the issue with the repository access needed for removal.
2. Delete the issue, or delete each earlier revision that still shows erased text.
3. If cleanup did not finish, also make the issue private, replace its text, and close it.
4. In Needs action, select Confirm done for the step, and confirm.

Hephaestus keeps the issue locator until you confirm the step. After that, it no longer knows where the issue is. The audit log records each confirmation. Do not remove app access or destinations before cleanup finishes.

Product improvement does not authorize research reuse of identifiable reports or quotations. Agree any research purpose, lawful basis, notice, retention, and safeguards separately with your institution.

Run it

Start with the private route before you enable the public route.

1. Configure the private destination, model, and daily limit.
2. Send a synthetic report through Send feedback with public permission off.
3. Check for one private issue with the report number and model disclosure.
4. Check that product_feedback_steps_total records step=file with outcome=ok.
5. Close the test issue.
6. Configure the public destination after the private test passes.
7. Monitor the first real public report, its hold, route, and alerts.

Do not send test issues to a production public repository. Use a separate test repository with synthetic data to test public behavior.

Check that it worked

The destinations card shows available app installations and a usable model. The private smoke test produces one issue and no public issue. Needs action explains stopped reports. If email delivery is set up, the admin email reaches its subscribed recipients. The first real public report waits for its hold and contains no detected personal details.

Undo

1. Turn on Pause in Destinations.
2. Check that new and unfiled reports appear in Reports that wait for you.

Keep the repositories configured. Follow-up, Make private, and erasure cleanup continue during Pause. Writes already in progress finish or reconcile. Pending /publish commands wait. To resume new filing, turn off Pause after you correct the cause.

Troubleshooting

Symptom	Check and cause	Action
Installation missing or suspended	The app cannot reach the named repository.	Restore its installation and Issues: read and write permission. Select Retry.
Private destination unavailable	The repository is public, inaccessible, or has an unknown visibility.	Turn on Pause. Restore private visibility and app access before Retry.
Model unavailable	The model, pool, credentials, or provider connection is unavailable.	Restore the instance model and its pool. Reports without drafts stay private.
Many no model draft finished routes	The daily limit is reached, model output stays invalid, or the draft calls fail.	Check the count and model health. If product_feedback_steps_total{step="draft",outcome="expired"} rises, check the server's outbox backlog. For outcome="not_configured", check the daily instance and account draft limits, then the model and its provider connection. For outcome="attempts_exhausted", check provider failures. Reassess the model with the routing evaluation.
Many private routes	Read the route split by reason. The gate can be strict or reports can contain detected personal data.	Inspect synthetic evaluation results before you change the model. Keep privacy checks in place.
Public report waits longer than five minutes	The hold is a minimum. Drafts, filing pace, rate limits, or reconciliation can delay it.	Check server health, model availability, and Needs action.
GitHub write stopped	DESTINATION_UNAVAILABLE or GITHUB_REFUSED explains the stop.	Turn on Pause if privacy is in doubt. Correct app access or the refusal, then select Retry.
/publish does nothing	Pause, missing reporter permission, insufficient repository access, or a safety condition blocks it.	Resume only when safe. Check the app’s response and both repository permissions.
No admin email	No verified subscribed admin or no usable relay exists.	Read the saved report in the console. Restore email delivery and subscriptions.
No reporter email	The account lacks a verified address, delivery is unavailable, or updates are off.	Use Your reports for status. Private comments do not generate reporter emails.
Cleanup needs action	The app cannot remove edit history, or cleanup remained incomplete for 30 days.	Complete the manual erasure steps above before you confirm them.