# Source contract Hephaestus may review work against a practice only from evidence with explicit semantics. Contract `3.0.0` defines the logical source catalog, its capture states, and how well each source must be captured. A folder index records what was supplied to one run. It does not prove that a practice is empirically observable. > If any required evidence clause is unsatisfied, refuse the review and create no result. A source failure is never `DOES_NOT_APPLY`, a strength, or evidence that desired behavior is absent. ## One frozen job folder The worker renders permitted database records in one read-only repeatable-read transaction. Integration-owned projections and history visibility checks remain the authorization boundary. The folder index records capture facts separately from practice readiness and typed source refusals. Another repository's records cannot make a missing primary source ready. The [workspace ABI](/contributor/agent/agent-workspace-abi.md) owns paths, quote coordinates, transfer budget and retention. ## Contract boundary The contract separates three identities: 1. **Source catalog:** logical source kinds and their authority, identity anchoring, completeness, required capture quality, privacy, and absence-state semantics. 2. **Practice definition:** the events a review selects and the evidence that review *requires* before it may run at all. A source declares the item kinds it applies to, so the catalog constrains which sources a practice may name. 3. **Folder index:** the state, provenance facts, and captured files for one invocation. A practice's `reads.sources` decide **readiness**, not staging. Every source permitted for the workspace and purpose is rendered for every review, whether or not a practice named it. Practice criteria and the reviewing model decide relevance. Relevance is not a reason to withhold data from the model. A required source that did not arrive withholds the selected practice. A COMPLETE need bounds what a result can assert is absent. Neither cuts the sandbox down. Paths are materializations, not source identities. Prompts, practice files, task envelopes, runner code, model settings, and executables are invocation inputs and require separate evaluation provenance. Runtime components must reference the exact contract version and catalog digest they implement. Inputs outside the catalog cannot satisfy a practice requirement. This contract governs automated-review evidence, not every file used by the separate Mentor conversation runtime. Mentor inputs remain subject to Mentor consent and integration controls and cannot support an automated practice review. Reusing one for review requires registering it as a source kind first. ## Source vocabulary ### Authority and identity | Authority | Meaning | | -------------------------- | --------------------------------------------------------------------------------- | | `UPSTREAM_SNAPSHOT` | Content captured from an upstream object. | | `SYNCHRONIZED_MIRROR` | A local projection whose currentness depends on upstream synchronization. | | `DETERMINISTIC_DERIVATION` | A deterministic transform of declared dependencies. | | `LOSSY_DERIVATION` | A sample, excerpt, aggregation, heuristic, or other information-losing transform. | A source's authority constrains what it may claim. Only a capture anchored to an immutable upstream identity can declare `PINNED_IDENTITY`. A `SYNCHRONIZED_MIRROR` reflects state that moves independently of the capture. A `LOSSY_DERIVATION` can never claim completeness. Both rules are enforced in `SourceContract`'s constructor, so a catalog that breaks either one fails at startup. A database row is not automatically authoritative for the current state of GitHub, GitLab, Slack, or Outline. ### Required capture quality Required capture quality is a fact about the source, recorded once in the catalog as `requiredQuality`. Each practice that names the source does not restate it. | Value | Meaning | | ------------------------ | ----------------------------------------------------------------------------------------------------- | | `ANY_CAPTURE` | The capture must have succeeded. Whatever it holds, including nothing, is reviewable. | | `COMPLETE` | The capture must cover the whole selected scope. A fragment cannot ground a judgment about the whole. | | `COMPLETE_AND_NON_EMPTY` | Complete, and containing something. | The three values form a ladder, not two independent booleans. The fourth cell of the cross-product, non-empty but possibly partial, asserts something no source can support. A partial capture that contains something cannot rule out that the omitted part mattered. A source that cannot report `COMPLETE` may not demand it, and a source that can never be validly empty may not demand non-emptiness. Both are rejected when the catalog is read, so a requirement that would refuse every review permanently cannot be stated. ### Privacy classes Privacy class describes risk. It does not authorize collection, disclosure, or model egress. | Class | Scope | Minimum treatment | | -------------------- | --------------------------------------------------------------------------- | ------------------------------------------------------------------ | | `INTERNAL` | Non-public project or workspace content without person-level review results | Workspace authorization and purpose-bound retention | | `PERSONAL` | User-linked project content, authorship, reviews, comments, or activity | Personal-data access, export, and erasure coverage | | `SENSITIVE_PERSONAL` | Private conversations or derived person-level review results | Restricted audience and stricter disclosure and retention controls | Each catalog source references separate source-use decisions whose purposes are machine-enforced. Capture requires `AUTOMATED_PRACTICE_REVIEW`. Result persistence, delivery, and contributor evidence APIs require `PRACTICE_FEEDBACK_DELIVERY`. Mentor reuse requires `CONVERSATIONAL_MENTORING`. And workspace-admin evidence review requires `OPERATOR_EVIDENCE_REVIEW`. Processor, region, transfer, and model-egress controls remain separate deployment obligations. A source allowed in an internal mirror is not thereby allowed in an external model request. ## Source catalog The machine-readable [catalog](https://github.com/hephaestus-build/Hephaestus/blob/main/server/application/src/main/resources/contracts/source-use/3.0.0/catalog.json), its adjacent JSON Schemas, and the checks in [`gate:contracts`](https://github.com/hephaestus-build/Hephaestus/blob/main/scripts/validate-source-use-contracts.ts) jointly define conformance. Schema validation alone is insufficient. Standard JSON Schema `uniqueItems` compares whole values, not a selected identity property. The [validation checks](#validation) separate resource validation and digest checks from the server's catalog and governance checks. Prose does not maintain a second field-by-field catalog. Authoring tools resolve source kinds through the catalog and reject unknown kinds. A source declares the item kinds it applies to, so a practice on one kind of work can only name sources that suit it. The constraints with direct authoring consequences are: * Only sources anchored to an immutable upstream identity — the pull-request diff and the repository tree — declare `PINNED_IDENTITY`. Every mirrored source, including the pull-request and issue records themselves, has identity policy `NOT_APPLICABLE`. A source's `authority` decides whether it can anchor at all. A `SYNCHRONIZED_MIRROR` reflects upstream state that moves independently of the capture. Calling it pinned would describe a copy that has drifted as demonstrably current. * A `LOSSY_DERIVATION` cannot satisfy `COMPLETE`. It is a bounded summary of its subject, not the subject itself. It cannot answer "is all of it here?". An author who asserts an absence asks exactly this question. Linked-work number extraction cannot establish that it found every relationship. Outline renders every permitted mirrored document and can report `COMPLETE` for that mirror scope, not upstream currentness. * Three sources cannot be logically empty — the pull-request, issue, and document core records. Every other source may be validly empty. * The checks source (`scm.pull-request.checks`) carries `head_checks`, `head_check_runs`, and `merge_state_status` in `checks.json`. The core record carries none of these fields. It carries `head_check_runs` only when a read listed them for that head. An absent field means that the runs are unknown, not that the head has no runs. A practice that needs the runs and cannot read them returns Could not determine, never Not met. A run with `finished: false` has no result. A claim that depends on its outcome is Could not determine, and admission refuses a citation of it (`unfinished_run_cited`). A run list never proves that a run is absent, because a provider can add runs after the read. `complete: true` says only that the read listed every run the provider knew at `observed_at`. `limitation: AWAITING_APPROVAL` says that the CI waits for a maintainer to approve it, so it has not run. Then `head_checks` is `PENDING`, whatever the provider's rollup says. A result can cite only the listed runs, with the state that they show at `observed_at`. GitHub supplies check runs and commit statuses. GitLab supplies none yet, because its pipeline hooks cannot show which runs finished. Slack threads and Outline documents have no check runs. The [sync lifecycle](/contributor/sync-lifecycle.md#check-runs-of-the-head) owns how the runs are read. * The review discussion source (`scm.review-threads`) carries `review_decision` in `review_threads.json`. The core record carries no aggregate review decision. * Two cannot be partial: the pull-request diff and the document core record. An available diff is also pinned. Its `requiredQuality` is `COMPLETE`: a verified empty range is available evidence. Each practice still needs its own review selection and supported result. Capture readiness assigns neither applicability nor an outcome. Missing, failed or incomplete required captures remain unready. `ENGINEERING_BASELINE` with `ENGINEERING_APPROVED` records maintainer approval of the shipped, minimized product scope. Contract `1.3.0` records explicit maintainer engineering approval of the wider permitted workspace scope in [#1732](https://github.com/hephaestus-build/Hephaestus/issues/1732). It is not controller or DPO approval. A later expansion of source, purpose, audience, processor, retention policy or collection scope needs its own decision. It is also the only basis and outcome the contract can express. A use nobody approved carries no decision, not a recorded refusal. Thus, no entry here can read as a controller sign-off. The controller's own record remains separate. Approval is the gate that runtime configuration cannot waive: a source whose decision is missing or expired is never read, whatever the deployment sets. Within that boundary, a connected workspace integration and an enabled practice authorize the everyday code sources that the product reads. Thus, a default deployment reviews pull requests, issues, and conversations without further configuration. Privacy class does not gate collection — it records what treatment a source needs. Changing a source's meaning, selection scope, limit, mapping, identity policy, required capture quality, dependency, or normalization requires a new contract version. Contract `1.2.0` introduced a checkout with Git history reachable from the reviewed commit, plus mirrored branches and tags. The source's governance and retention policies apply. The reviewed commit and tree identify the checkout. The folder index also digests the `repos/reviewed/.git/HEAD` and captured-refs witnesses, which pin the exported refs — those can change between two captures of the same commit. ## Capture state Source absence is separate from quality so the folder index does not collapse unavailable, partial, and empty evidence into one state. ### Availability * `AVAILABLE`: capture facts and logical content state are present. * `NOT_COLLECTED`: no unexpired use decision permits the source for this purpose, or collection is switched off for it in this deployment. Never "no practice wanted it". Every source that applies to the item kind is staged for every review. Relevance is not a reason a source can be missing. * `UNAVAILABLE`: the source could not be retrieved for a non-error reason. * `REDACTED`: the entire source was withheld for privacy or security. * `COLLECTION_ERROR`: an attempted collection failed. A non-available source contains no captured files or source identifiers, and carries a typed `SourceAbsenceReason` saying *why* — `GOVERNANCE_NOT_EFFECTIVE`, `NO_PROVIDER`, `NO_WORKING_COPY`, `CONSENT_NOT_ACTIVE`, and the rest. Safe, typed reason codes may be role-filtered. Free-form exception text must not enter the folder index. ### Available-source dimensions Only `AVAILABLE` has: * logical content: `EMPTY` or `NON_EMPTY`. * completeness: `COMPLETE`, `PARTIAL`, or `UNKNOWN`. * capture facts: `capturedAt` — the only required one — plus an optional `observedAt`, an optional `sourceEffectiveAt` watermark, and an optional pinned identity. * captured files. Capture facts hold only what could not be known before the capture ran. The folder index pins the catalog by digest. Everything derivable from that catalog is read from it, never copied into the capture. This includes selection scope, limits, required capture quality, and states a source can report. Logical content is independent of serialized bytes: a JSON file containing `[]` represents an `EMPTY` source. Field-level redaction remains an available, usually partial source. Whole-source redaction uses `REDACTED`. Source age alone cannot prove currentness. An age threshold needs a watermark recording when a copy was last *verified* against upstream. The mirror records only when a row was last *written*. An age derived from it measures time since the last upstream change. It reports a correctly mirrored quiet record as stale and a record modified moments ago as current. Adding one therefore has a prerequisite: a reconciliation pass that advances a verified-as-of watermark after confirming a record is unchanged. Until that watermark exists, the system cannot substantiate a currentness guarantee on a mirrored source. `IdentityPolicy` records only whether a capture can be anchored to something immutable at all. A readiness report names the contract version and the catalog digest used to interpret it. It names the item kind, the folder index's capture time, and one decision per practice. A folder index captured under a contract version the runtime no longer ships is *unreplayable*, not invalid. The recorded result stands. Only the ability to re-derive it is unavailable. ## Practice definition and source requirements A practice version owns one authored definition. Reads contains the source contract and declared source needs. Reviewed by controls automated, human or guidance-only behavior and retains its reason and limits. Review on selects automatic events and state. Applies when checks every review origin. The [review pipeline](/contributor/practice-review-pipeline.md#select-a-review) owns applicability checks and capture refusal behavior. SCM capture retains the pinned base, head, changed paths and unified diff in the captured change record of the job context. Changed paths include both names of a rename. Applicability reads these captured facts without access to the repository mirror. The practice's item kind is explicit and need not have an event. | Need | Meaning | | -------- | ------------------------------------------------------------------------------------------- | | REQUIRED | The review needs this source at its declared capture quality. Failure withholds the result. | | COMPLETE | The claim needs complete source capture. A fragment cannot establish bounded absence. | | OPTIONAL | The review uses the source when available and records its absence as a limitation. | Scope and consent determine whether the material can be collected. The selected About role must resolve to one eligible account, unless About is explicitly null. Feedback mode Off stops delivery at every destination, not result recording. A request bypasses event selection but never authorization or readiness. The [pipeline](/contributor/practice-review-pipeline.md) owns those boundaries. ## Conservative examples ### Required evidence is insufficient `scope-one-reviewable-change` is bound to `scm.pull_request.opened` and `scm.pull_request.ready`, and requires `scm.pull-request.core` and `scm.pull-request.diff`, with `scm.repository.tree` as an optional source. The diff's contract demands a complete capture, including a verified empty range. If the diff cannot be read, or the pinned commit is gone from the mirror, readiness refuses the practice and no result is created. The runtime makes no claim that the change is focused, sprawling, or `DOES_NOT_APPLY`. No source need can demand currentness that the pull-request record cannot show. Nothing available could substantiate one. ### Empty evidence is valid A complete pull-request comment query may materialize as `[]`. Its state is `AVAILABLE + EMPTY + COMPLETE`, and `scm.pull-request.comments` sits at `ANY_CAPTURE`, so that is valid evidence rather than `UNAVAILABLE`. A redacted or failed query cannot support the same practice claim, and a practice with a COMPLETE need is withheld on a partial capture. ## Runtime behavior The runtime resolves exact practice versions before capture. It then stages **every governed source the purpose permits in the workspace**, on every review. There is no per-run source selection. Practice review configuration decides which practices can run, never which sources are captured. A source listed as anything other than `AVAILABLE` always gives a typed reason code. That reason is never that no practice asked for it. Readiness is checked for each selected practice. An OPTIONAL source failure does not withhold an otherwise eligible practice. A REQUIRED or COMPLETE source failure does. If no practice is eligible, no model request is made. One invocation can contain the governed union needed by several practices. A script can use only its declared sources to support a result. A scan also restricts reads and searches to its assigned file and declared context. Evaluations that require process isolation must use separate evidence views. Citations must name a declared, available source and an exact quote from captured text. Core checks the captured bytes and their digest. Domain verifiers check source-specific side, version and claim-basis meaning. A failed check withholds the candidate at admission. Feedback composition cannot admit it as a result. This proves traceable grounding, not model causality. The [workspace ABI](/contributor/agent/agent-workspace-abi.md#evidence-and-read-receipts) owns manifest fields, read receipts and coordinate verification. The [script runner](/contributor/practice-script-runner.md) owns the candidate citation shape. Shell output alone is not an authorized source. Readiness decisions remain server-side. The folder index carries safe capture reason codes to the sandbox. Do not put raw exceptions or private policy detail in a model-visible reason. ## Change and staleness Historical records keep their original contract and practice version. Currentness is derived: * source resources are immutable within a published contract version. Any semantic change creates a new version. The item record schemas under `contracts/items/` are not source resources. An optional field added to a record keeps the contract version. * an [explicit practice upgrade](#cut-a-contract-version) controls when a practice moves to another source-contract version. * changing a practice's review configuration or automated-review policy stales that practice's claims. * adding a source cannot silently alter existing requirements because each practice names its sources explicitly. * governance withdrawal disables its exact future use without rewriting historical semantics. This includes collection, result persistence and delivery, mentoring reuse, or evidence disclosure. * erasure or expiry may make a claim unreplayable without making the historical result false. Semantic currentness, automated-review-validation status, governance eligibility, and replayability remain separate. Result and review APIs expose `currentness` as `CURRENT`, `STALE`, or `UNVERIFIABLE`. They compare the review digest of the evaluated practice version with the review digest of the current practice version. A result with an end time is `STALE`. Presentation-only edits retain semantic currentness. Rows without comparable versioned fingerprints are `UNVERIFIABLE`. Neither value says whether a later review ran, and pull request work is never compared. Whether a result still concerns the stored work is a separate fact. A result records the item version that its review read in `item_version`. A pull request version identifies the base, the head, and a digest of the title and description. `review_job.capture` keeps the folder index manifest with its capture time. Two server paths end a result with `WORK_CHANGED`: * For an issue or a pull request, `ResultCurrentness` compares the synchronized source's version with the result's `item_version`. It ends the result when they differ. It checks when an SCM event for the item is recorded. It also checks when the item's source is observed or a result is recorded. Thus, a new result can end when it is recorded. * A recorded `scm.pull_request_review.*` event ends current results of its item without a version comparison. It ends only results whose source reviews were ordered no later than the event. A result of a later-ordered review for the same practice, item and developer ends the earlier one with `REPLACED`. The mentor's result history shows `endedAt` and `endReason` of an ended result, beside its recorded outcome. It does not name a replacing result, and its bounded sample can omit one. The history states two more facts of a result whose words it may copy: * `capturedAt` is the capture time in the retained manifest of the job that produced the result. Only the assignment of the result's own review, practice and item that records this result names that job. The custody's `sourceJobIds` also list copied dependencies, so they do not name it. When that job is missing, belongs to another review, or has no valid manifest, the capture time is unknown. `recordedAt`, the job's submission time and the provider's edit times are not capture times. * `storedWorkVersion` compares the exact recorded `item_version` with the version of the work that Hephaestus stores now. `ProductItemQuery.currentWorkVersion` reads that version, for an issue or a pull request only. `ResultCurrentness` reads the same version. `SAME` means that the two stored versions are equal. It is not a live provider read. The version does not cover every field of the work. A matching version does not show that a review is current or that a problem was repaired. `DIFFERENT` includes an edit of only the title or the description. `UNKNOWN` applies to other kinds and to a source that is unavailable or withheld. It also applies to a result whose words the history may not copy. A history read ends no result. The results of a removed item stay out of the history. A result without an end time is not proof that it concerns the current work. The version comparison ends no result when its source is unavailable. An end reason records why the claim stopped. It does not say which fields changed, when they changed, or what a later review found. ## Adding or changing a source Complete the [source governance decision](/admin/dsms/item-source-governance.md) before enabling a new or expanded source use. Accuracy or product benefit alone does not authorize collection. ### Cut a contract version The source-contract version is separate from the application release version. Use this procedure when source semantics change, after the governance decision above. Run commands from the repository root. 1. Copy the current directory under `server/application/src/main/resources/contracts/source-use/` to an unused version directory. Keep its catalog, source-use decisions, and all four schemas together. Leave every published version unchanged. 2. Set the new `catalog.json` version and `source-use-decisions.json` contract version. Update every schema version constant and each schema's `$id` to the new version. Use the schema identifier format of the current contract resources. Update version-specific references to other schemas if the change introduces them. 3. Apply the intended source and governance changes only in the new directory. Finalize `catalog.json`, including formatting, before calculating its SHA-256 digest. Replace `NEXT_VERSION` below with the new version: ``` node --input-type=module -e ' import { createHash } from "node:crypto"; import { readFileSync } from "node:fs"; console.log(createHash("sha256").update(readFileSync(process.argv[1])).digest("hex")); ' server/application/src/main/resources/contracts/source-use/NEXT_VERSION/catalog.json ``` Set `automated-review-readiness-report.schema.json` → `properties.catalogDigest.const` to that digest. Hash the exact file bytes, not parsed or normalized JSON. Recalculate the digest after any subsequent catalog edit. 4. Set `ClasspathSourceCatalogRegistry.CURRENT_VERSION_VALUE` to the new version. The registry derives current resource paths and the runtime digest from this value. It accepts only the current contract. Keep published resources unchanged for verification. 5. Update current-version documentation and fixtures without changing historical contract resources. Cover the new catalog, its pinned digest, and rejection of unsupported versions in the registry tests. Cover an explicit practice upgrade in the practice-version and review-digest tests. 6. Run `vp run gate:contracts` and `vp run test:server:unit`. Check both the resource-validation results and the runtime validation below. Follow the [local verification procedure](/contributor/local-verification.md) for the remaining checks. A practice definition pins its source contract in `reads.contract`. A version cut does not change existing practice versions. A definition update creates a practice version with a new review digest. The stopped 1.0 upgrade creates current versions for practices that pin 2.0.0, including custom practices. It also updates their catalog comparison bases. Historical versions stay unchanged. The [currentness rules](#change-and-staleness) apply after the update. The [catalog update path](/contributor/practice-catalog.md#workspace-adoption-lifecycle) owns workspace proposal and acceptance behavior. ### Validation The rules are enforced mechanically, in two places: * `gate:contracts` validates every contract resource and the bundled practice catalog against their schemas. It holds the readiness schema's pinned catalog digest to the catalog's actual hash. It exercises the policy schema against deliberately broken fixtures. These include a retired evidence profile and evidence requirements written onto a policy instead of the definition. A guard whose failure path never runs is indistinguishable from one that cannot fail. The gate also checks published-resource immutability against the branch's merge base and tests that guard. Changed or deleted published files fail that check. * When it reads the catalog at startup, the server rejects duplicate source kinds. It rejects a mirror that pins an identity. It rejects a lossy derivation that claims completeness. It rejects required capture quality that the source can never report. It rejects a source whose use decisions do not cover every product purpose. It rejects a decision whose retention or erasure policy disagrees with its source. ## Prior art * [Gebru et al., "Datasheets for Datasets"](https://doi.org/10.1145/3458723) — the source catalog is a datasheet. It records composition, selection scope, uses, and limitations alongside the data. * [GDPR Article 5](https://eur-lex.europa.eu/eli/reg/2016/679/art_5/oj) — purpose limitation, minimization, accuracy, and storage limitation, which the source-use decisions record per source and purpose.