The release gate
The release gate answers one question about one commit: can this project ship? It reads what CI, your runners and your uptime checks proved, and says Shippable, Not yet or Unproven, always with its reason. The gate reports; it doesn't stop a deploy.
Everyone who sees a project sees its gate. The project's editors, admins and the owner change what it asks for.
Open it
- Open the project and its Health tab (see Read a project's dashboard).
- Under Surfaces, press the verdict in a surface's Release column.
On a project's page you can also open the command palette (⌘K / Ctrl K) and choose Release gate.
Read the verdict
The top of the page names the verdict and the project:
- Shippable: "smart-home is shippable at 4f2a1c." Every requirement was proved on that commit.
- Not yet: "smart-home isn't shippable yet." Something required failed. The first reason follows, such as "3 of 88 failed in the last web-e2e run", naming the check whose tests failed. That count can differ from the Health tab's Tests cell ("2 failed · CI", with "Latest failing run" under it), which counts the latest run of a workflow that is failing, wherever it ran.
- Unproven: "smart-home is unproven." Nothing failed, but proof is missing or too old, and the reason says which: "Not checked on this commit", "Out of date since Nov 12".
- "Nothing has checked smart-home yet." when there's no proof at all.
Under it, what the verdict doesn't cover: "2 newer commits not checked", "Proof runs out on Nov 19". The last line says which commit it's about ("About 4f2a1c, the newest commit on main with proof.") and "The gate reports what's proved. It doesn't stop a deploy."
Then Shippable when, on the same commit: lists each requirement with its state: Proved, Failed or Unproven, and where the proof came from ("Proved · Studio Mac mini", "Proved · CI"). Press the state to open the result that decided it. A requirement that isn't proved says why.
What counts
A requirement is proved on the commit the gate is about, and each kind of proof has its own clock:
- Tests, a minimum number of tests and an image boot: on that commit, and under 14 days old. The project can change the number of days.
- CVE scans: within 7 days, against a scanner database under 5 days old.
- Signing and notarization: no age limit. They're tied to the file's hash, and fail when the certificate expires or the signature is revoked.
- Uptime and TLS: healthy now.
Carried proof. A result from an older commit still counts when the check's declared inputs haven't changed since: "Carried from 4f2a1c, inputs unchanged · Nov 12". It ages from its own date. A check that declares no inputs, and CI, can't carry.
What never counts. A run on a tree with changed files, on a commit that isn't pushed, or on a branch other than the release branch doesn't count, and neither does a check that proved nothing (Ran with no tests read, Couldn't run, Skipped, Cancelled, Interrupted). They show as notes: "Not counted: Files were changed, so its results don't count". The run board warns about changed files before a run starts.
With no gate set up, the page says "No gate set up yet: from CI and uptime, under 14 days old".
Check an earlier commit
- Under Check an earlier commit, choose a commit ("91ce02 · Nov 12").
- The verdict is now about that commit: "About 91ce02, the commit you chose." The page's address names it, so you can share it.
Choose The newest with proof to go back.
Run what isn't proved
A requirement that isn't proved offers Run on Studio Mac mini for each machine you can start runs on that has an approved check for it.
- Press Run on Studio Mac mini.
- The run opens on Runs, Queued until the machine checks in (about every 30 seconds), then running.
- When it ends, the gate reads the new proof.
"No machine you can start runs on has a check for this." says when none can. Starting runs on someone else's machine needs them to let editors start runs: see Pair a runner. Running checks is in Run checks.
Change "Shippable when…"
- Under Shippable when, on the same commit:, press Edit (or Set up this project's gate when there's none yet). Viewers don't see it.
- The editor says "smart-home is shippable when, on the same commit:". For each requirement, choose:
- What it asks: Tests pass, Enough tests run, The image boots, No CVEs over a severity, It's signed, It's notarized or Uptime and TLS are healthy.
- Of: one of the project's surfaces, or The whole project.
- From (for tests): Anywhere, CI or A runner.
- At least this many tests or Allow CVEs up to, where it applies.
- Add a requirement adds one; Remove requirement 2 takes one away. Start from what its surfaces suggest fills it in from the project's surfaces.
- Set Release branch (a label for results; it never chooses what a machine runs), Tests count for this many days and Scans count for this many days.
- Press Save the gate. The verdict is worked out again at once. Cancel leaves it as it was.
A gate that asks for nothing says "It asks for nothing yet, so nothing can make it Shippable."
Findings
When a guide walk or a person has recorded findings for the project, Findings lists them under the requirements, by severity (S1 Blocker to S4 Cosmetic), each quoted as recorded. Findings aren't coverage: they don't change the verdict.
- Open a GitHub issue opens GitHub's new-issue page with the finding filled in. You submit it on GitHub. For a public repository the page warns first; press Open the issue to go on.
- Dismiss, for the project's editors, takes a finding off the list.
See Walk the guides.
From the command line
vitals gate prints the project's verdict with each requirement, and exits 0 only when it's Shippable. vitals gate --wait waits until the verdict is about this folder's commit, for release scripts. See The vitals command line.
When the verdict changes
A project going from Shippable to Not yet raises an alert under the Release not shippable rule. Proof running out moves a project to Unproven and raises nothing, because nothing broke. See Alerts.