# Delivery Metrics The **Repo / Delivery metrics** workflow reports the [DORA](https://dora.dev/guides/dora-metrics/) delivery metrics each Monday. It reads only GitHub records that the delivery flow already writes. It uses no external service and gates nothing. Each run reports the last 7 days and the last 28 days in its job summary. It keeps the measurements as the `delivery-metrics` artifact for 90 days. ## Sources | Record | Written by | Gives | | ----------------------------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | | Merged pull requests into `main` | the merge queue | first commit time, opening time, merge time, merge commit, merge-queue entries, and the CI runs and statuses of the merged head | | Published releases | `release.yml` | publication time and the release commit | | Deployments of `Staging` and `Production` | `promote.yml` through `scripts/record-deployment.ts` | promoted commit, outcome, outcome time | | First-parent history of `main` | Git | the order of merge commits | A deployment carries a change when it deploys the change's merge commit or a later commit on the first-parent history of `main`. A release carries a change in the same way. The outcome of a deployment is its last `success`, `failure` or `error` status. The `success` time is when the public webapp reported the promoted version. [Release Management ยง Deployment history](/contributor/release-management.md#deployment-history) defines these states. ## Metrics [DORA](https://dora.dev/guides/dora-metrics/) defines five metrics: change lead time, deployment frequency, failed deployment recovery time, change fail rate and deployment rework rate. Production deployments are the DORA measure. Staging shows the flow before it. Where this report differs from DORA's definition, the report says **proxy**. | Metric | Definition here | DORA's definition | | --------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | | Lead time for changes | Median and p90 per change: first commit to merged, to running on staging, and to running in production; merged to running on staging, to a published release, and to running in production; and pull request opened to running in production. A deployment that was rolled back does not count as the change running. | Commit to running in production. Same. | | Push to merge | Three stages per merged change: from the last push (CI/CD started on the merged head) to its `All CI Passed` status, from there to its last merge-queue entry, and from that entry to the merge. Also how many changes the queue dropped at least once. | Not a DORA metric. It splits the part of lead time before the merge. | | Deployment frequency | Successful deployments per environment, per week, and the number of days with a deployment. | Same. | | Change fail rate (proxy) | Deployments that ended in `failure` or `error`, or that the next deployment moved back, divided by decided deployments. | Deployments that cause a failure in production that needs remediation. Here, `failure` also covers a version check that a slow host failed. | | Failed deployment recovery time (proxy) | From the first of consecutive failed deployments to the next deployment that stayed. The unit is an incident, not a deployment. | Time to recover from a failed deployment. DORA formerly called it time to restore service. | | Deployment rework rate (proxy) | Deployments that ended an incident, divided by decided deployments. A release number cannot tell rework apart: before 1.0, every fix is a patch release. | Deployments that were unplanned and made to fix a production incident. | A **rollback** is a successful deployment of a commit that is older on `main`'s first-parent history than the one it replaced. The request's rollback permission does not decide it. A **repeat** is a successful or undecided deployment of what the environment already runs, such as a re-promotion of the running build. It moves nothing, so it is neither a deployment nor a failure. The report counts repeats separately. A failed deployment counts as a failure, whatever it deployed. After a failure, what runs is unknown, so the next deployment of the last good build is a recovery, not a repeat. The Version PR is not a change. It consumes the changes it releases. A change that has not arrived yet is counted as waiting, not as a zero duration. Reverted pull requests are reported beside the fail rate. ## Limits * A deployment record states what the promotion workflow observed: the public webapp version. It does not prove that every service is healthy. Host metrics are the record of what a host runs. * A failure that no deployment record shows, such as an incident that a forward fix resolved, is not counted. * GitHub's search lists at most 1,000 pull requests. The report fails instead of measuring fewer than it found. * The report reads deployments from 28 days back. The first of them has no predecessor, so it cannot be a rollback or a repeat. * A deployment without a final status, such as one still in progress, counts in no rate. The report gives the number. * Days with a deployment are calendar days in UTC. ## Run it locally You need an authenticated GitHub CLI and a checkout with the history of `main`: ``` git fetch origin main GITHUB_REPOSITORY=hephaestus-build/Hephaestus vp run report:delivery ``` The report goes to standard output. The measurements go to the same JSON file that the workflow keeps as its artifact, under the ignored temporary directory.