# Run checks

A check is one thing that can pass or fail on a machine: a test command, a Docker smoke, a signing verification. A run is one or more checks started together. This guide covers running checks in the desktop app, reading what they say, starting runs on a machine from the web, and letting them run by themselves.

It assumes the app is installed and this machine is paired: see [Install the desktop app](install-the-desktop-app.md) and [Pair a runner](pair-a-runner.md). With nobody signed in, the app runs checks too, and keeps their results on the machine: see [Start without an account](install-the-desktop-app.md#start-without-an-account). For `vitals` in a terminal, see [The vitals command line](vitals-cli.md).

## Without a runner: browser checks

To check that a page loads and says what it should, you don't need a machine of your own. On a project's **Health** tab, **Browser checks** → **Add browser check** has ShipVitals' cloud open the page in a real browser on a schedule, with a screenshot of each run that loads. It runs no code of yours: your own Playwright or other tests run on a paired machine (this guide) or in CI (see [Send test results and your own metrics](ci-and-push-metrics.md)). See [Browser checks](browser-checks.md).

## This Mac

**This Mac** sits at the foot of the sidebar with one word: **Ready**, **Running**, **Paused** or **Needs setup**. **Needs setup** means a linked check waits for review, or a tool a check needs is missing (**Doctor** says which); the menu bar panel says "This Mac needs setup" too. Press it, or press `g` then `m`. On Windows it's **This PC**.

Its header says the machine's role and how many checks are linked ("Laptop · 12 checks"). Its tabs are **Doctor**, **Links**, **Privacy**, **Relays**, **Background** and **Storage**. Links, Privacy and Relays belong to one workspace; when the machine runs checks for more than one, a **Workspace** switcher appears above them.

Anything that needs you shows at the top with its next step, such as "Can't find ~/code/120." with **Choose folder**.

### Doctor

**Doctor** says what this machine can run: "This Mac can run 11 of 12 checks." Each line names a tool the linked checks need and where it was found ("Node 24.22 · found through your shell"), or what's missing:

- "Android SDK · not found. 1 check needs it." with the command to install it, **Copy** and **Show me how**.
- "Docker · not running. 2 checks need it." with **Start Docker**.
- An optional scanner such as Trivy, with **Why we pin this**: ShipVitals offers one pinned version and never installs a scanner by itself.

After fixing something, press **Check again**.

### Background

**Background** holds the machine's role (**Laptop** or **Always on**), **Checks at once**, **Pause**, **Open at login**, **Notifications** and **Global shortcut**. A laptop waits for power for runs it didn't start by hand.

To pause: press **Pause** and choose **Pause for 1 hour**, **Pause until tomorrow** or **Pause until I resume**. "A check that's running finishes." **Resume** starts it again. The menu bar panel has the same **Pause**.

### Storage

**Storage** shows how much the runs kept here use ("1.2 GB of 5 GB used"). Logs, reports and screenshots stay on this machine; when they reach the limit the oldest runs go first, and every check's latest result is kept.

1. Choose a limit under **Keep up to**, or press **Clean up now**.
2. Read **Remove older runs?** and press **Remove older runs**. The page says how many went. Results already in the cloud stay.

## Approve checks

Nothing runs until a person at the machine has read and approved its exact command. An approval covers that command, on that branch, on this machine: the code on the branch runs with it.

1. Open **This Mac → Links**. Each linked folder shows its project, "Approved branch: main" and "5 of 6 checks approved".
2. A project with checks to read says "1 check waits for someone here to review it." Press **Review**.
3. Read each command, where it was found and what it needs. Untick what you don't want; **Edit** changes a command.
4. Press **Approve 1 check**, then **Approve** in the system dialog, which lists each command again.

**Link a folder** links more projects (it opens **Where's your code?**).

### When a check changes

If a check's command changes (in the repo's `Makefile` or `.shipvitals/checks.yml`, say), it stops on this machine. **Run all**'s sheet counts it among the checks that won't run, and on the board it's **Couldn't run**: "The check changed. It won't run until someone reviews it", which is no proof for that run. **Links** shows it waiting. Press **Review**: the system dialog shows "Changed since you approved it:" with the old and new lines. Approve it again, or leave it.

On the web, the same check shows "Waiting for approval on MacBook Pro". It can only be approved at the machine.

**Forget approvals**, at the foot of **Links**, stops every check for the workspace on this machine until someone reviews it again. The links and their results stay.

## Run all

1. Press `⌘R` (or **Run → Run All**, **Run all** on **Runs**, or **Run all** in the menu bar panel). This opens a sheet; nothing starts yet.
2. The sheet says what would run: "3 projects · 12 checks · about 38 min" ("first time, no estimate" before there's history), then each project with its commit, checks and time. It warns before you start:
   - "2 files changed, won't count for the gate"
   - "on feature/x, not the approved branch"
   - "1 check can't run here: Android needs the Android SDK" with **Fix**, which opens Doctor
   - "On battery (64%). Plug in, or runs may be slow."
3. Tick **Only what changed** to skip checks whose files didn't change since their last pass, **Choose checks** to pick them, or **Stop on first failure**.
4. Press **Start** (or `Enter`).

The run board opens: a row per check, grouped by project. Its header reads "Running 6 of 12 · about 14 min left · 4 passed · 1 failed". Failed rows rise to the top, and a finished project's passed rows fold into "All 5 passed" (**Show passed** opens them). A running row shows its step ("Running tests · 120 of 214") and a clock; a waiting one says why ("Waiting for the simulator"). Select a row, or press `j` and `k`, to see its log beside the list.

Closing the window doesn't stop the run. The menu bar panel shows its progress.

To stop it, press **Stop** (`⌘.`). Checks still running come back **Cancelled**.

## The verdicts

When the run ends, the board becomes **Verdicts**, one line per project:

- "smart-home isn't shippable. Android: 3 of 88 failed."
- "agentgarm is unproven. Files were changed, so its results don't count."
- "120 is shippable at a7d311."

Under them: the counts, the time and **All results sent** (or **Results stay on this machine**).

- When something failed, press **Re-run failed** (`⇧⌘R`) to run just those checks again, on the machine that ran them.
- Otherwise press **Done**.

A project's first results wait until you've seen what's sent: **Before web's first upload**. See [What leaves your machine](what-leaves-your-machine.md).

## Read a result

Every check ends in one of these, always with its word and a reason:

| Word | Means | Counts for the gate |
|---|---|---|
| **Passed** | It ran and its results were read: "Passed · 214 tests" | Yes |
| **Ran** | It exited 0 with nothing to count | Only if the check says so |
| **Failed** | The product did something wrong: "3 of 214" | Blocks |
| **Flaky** | It failed, then passed on retry on the same commit | Passes, and is noted |
| **Couldn't run** | The machine stopped it, not the product: "Docker isn't running" | No proof, never a failure |
| **Skipped** | Deliberately not run: "needs Xcode", "nothing it covers changed" | No proof |
| **Cancelled** | A person stopped it | No proof |
| **Interrupted** | The machine stopped mid-run: "The machine went to sleep" | No proof |
| **Out of date** | True, but about an older commit | No proof |

Press a check on the board to open its result.

**A failed check** leads with what broke: the check, "3 of 214 failed", the commit, the time and the machine, then the first error with its file and line. Below:

- **Open in editor** (`e`) opens that file and line in the editor chosen beside it.
- **Re-run** runs this check again.
- **Copy for your agent** (`⇧⌘C`) copies the check, its command, the first errors and the commit as a prompt to paste into your coding agent. "Copied. Paste it into your agent."
- **Save to the repo** writes the project's checks to `.shipvitals/checks.yml`, uncommitted, so another machine can share them. **Edit** opens the check's review.
- "Passed yesterday on b81d0e. 3 commits since, by 1 person." with **Compare**, then the other failures grouped by their first error, the evidence and the log.

**Couldn't run** leads with the fix instead: "Couldn't run: Docker isn't running." and "Nothing was tested, so this says nothing about agentgarm." Press **Start Docker** (or **Open System Settings**), then **Check again**, then **Re-run**.

On the web, a result shows the first error that was sent: "On the web only the first error that was sent is shown. The full log stays on the machine that ran it."

## Start a run from the web

Runs appears in the sidebar once the workspace has a paired machine (`g` then `r`). From a browser or a phone you can start runs on a machine you paired, or on one that lets editors start runs (see [Pair a runner](pair-a-runner.md#let-editors-start-runs)). Only checks already approved on that machine run.

From **Runs**:

1. Press **Run all**.
2. Under **Run on**, choose the machine: "Studio Mac mini · ready", "MacBook Pro · asleep, runs when it wakes". The sheet says when it will start: "Studio Mac mini checks in about every 30 seconds, so it usually starts within 30 seconds."
3. Press **Start**. The run opens as **Queued**, then **Running**, then its verdicts.

From a project's **Health** tab, an empty **Tests** or **Signing** cell offers **Run on Studio Mac mini** (or **Run on…** and a list of machines). It says "Queued. Studio Mac mini checks in about every 30 seconds." When you can't start runs there, it says which machine can ("Studio Mac mini can run this.") or offers **Set up a runner**. The release gate page has the same buttons: see [The release gate](release-gate.md#run-what-isnt-proved).

**Runs** lists the current run under **Now** and the rest under **Earlier**, each with its machine and how it started (**By hand**, **From the web**, **On a schedule**, **On a commit**). A run you start from the web on a machine you paired counts as **By hand**; **From the web** is someone else's run on it.

## Run on a schedule or on each commit

Both are off until someone turns them on. They run approved checks only and never change your working tree.

### A nightly run

1. Open the project, then **Health**, and find **Runs** (it appears once the workspace has a paired machine).
2. Under **Schedules**, tick **Every night at 02:00, or at next wake**, and set the **Time of day**. Missed runs collapse into one.
3. **Run again before proof runs out** re-runs checks before the release gate's proof expires, "So a quiet project stays Shippable."

Viewers read these; the project's editors change them.

### Commit runs

Commit runs are turned on at the machine, because they let anyone who can push run code on it.

1. Open **This Mac → Links** and find the project.
2. Tick **Commit runs**.
3. Read **Run checks on every push to main?** "Anyone who can push to main will have code run on this Mac, as you. Turn this on only for repositories whose contributors you trust." Press **Turn on commit runs**, or **Not now**.

A check runs only when the folder is on that commit; otherwise it says "This folder is on feature/x, 3 commits away". Tick **Clean checkout** to test every commit in a separate copy in ShipVitals' own folder, leaving your working tree alone.

A clean checkout runs nothing the repository names, Git LFS's filter included, so its Git LFS files are only pointers there. A check that covers any of them (the `paths` it names, else its working folder) doesn't run against them: it's **Couldn't run**, "Its Git LFS files are only pointers in the clean checkout. Turn Clean checkout off to run it in your folder". Untick **Clean checkout** for that link, or give the check `paths` that leave the Git LFS files out.

The project's **Schedules** then shows **On every push to main**, "Turned on at MacBook Pro." Editors can untick it there to switch it off for the project.

## Offline

Checks keep running without a connection. The line above every page says "**Offline.** Showing figures from 09:14. Changes are off until you're back online." and, when results are waiting, "3 results waiting to send". They send, in order, when the connection returns. The menu bar panel says "Offline. Runs still work, and results send when you're back."

A part of a page that wasn't kept says **Not available offline**.
