What leaves your machine
When the desktop app or vitals runs your checks, the results go to ShipVitals and most of the evidence stays on your machine. This page says what goes, what stays, and how to change it for each project. The app shows the same table in This Mac → Privacy (This PC on Windows and Linux). With nobody signed in, nothing on this page goes anywhere: the app and vitals keep every result on the machine.
What goes, what stays, what never leaves
- Goes to the cloud by default
- Check names, status, counts, durations
- Stays here by default
- Full logs
- Never leaves
- Folder paths and file contents
- Goes to the cloud by default
- Commit, branch, whether the tree was clean
- Stays here by default
- Screenshots, videos, traces
- Never leaves
- Environment values and secrets
- Goes to the cloud by default
- Failing test names and a short first error, scrubbed
- Stays here by default
- Reports and traces
- Never leaves
- Your LLM key
- Goes to the cloud by default
- Tool names and versions, OS, machine name
- Stays here by default
- Guide-walk evidence
- Never leaves
- Keychain items, signing keys
- Goes to the cloud by default
- Signing facts, hashes, CVE counts
- Stays here by default
- Never leaves
| Goes to the cloud by default | Stays here by default | Never leaves |
|---|---|---|
| Check names, status, counts, durations | Full logs | Folder paths and file contents |
| Commit, branch, whether the tree was clean | Screenshots, videos, traces | Environment values and secrets |
| Failing test names and a short first error, scrubbed | Reports and traces | Your LLM key |
| Tool names and versions, OS, machine name | Guide-walk evidence | Keychain items, signing keys |
| Signing facts, hashes, CVE counts |
In more detail:
- The first error of a failed check is at most 2 KB. Expected and received values are taken out first, wherever the test runner prints them (
Expected: 3,Received: 4, a- Expected/+ Receiveddiff), because they can be your customers' data. - A file in that text appears only as its name and line, such as
checkout.spec.ts:12, never with its folders. - Scrubbing: everything that goes passes one scrubber first. It hides the check's own environment values, key-shaped values,
Authorizationand cookie headers, home-folder paths and email addresses, and leaves[hidden]where it did. If text can't be scrubbed with confidence, it isn't sent: the check shows "Its output couldn't be cleaned, so it wasn't sent". The scrubber can't recognise customer data, and the app says so: "We scrub what we can recognise. Keep screenshots off for projects with real customer data." - Folder paths, environment values and the contents of your files don't go, apart from the evidence files you choose to send below.
See it on your machine
- In the desktop app, choose This Mac at the foot of the sidebar (or press
gthenm). - Choose Privacy. The page says "What leaves this machine for …", with the table above.
- If this machine runs checks for more than one workspace, choose one under Workspace. Privacy is that workspace's.
Before a project's first upload
The first results of each project wait on your machine until you've seen them.
- After the project's first run, the app shows Before …'s first upload: "This is what goes to ShipVitals for this run. It asks once for each project, and never again."
- Read the rows: Result, Commit, Counts, Test names, First error, and Files or Stays here when there's evidence. Under them: "Error messages can contain what your app showed."
- Press Show raw JSON to see exactly what's sent, request by request. Hide raw JSON folds it away.
- Press Upload to send exactly what's shown. Later uploads of this project go without asking.
Don't upload these drops these results: nothing is sent, and the next upload asks again. Closing the sheet leaves them waiting.
With vitals run in a terminal, the same rows print after the run, then "Upload these results? It won't ask again for this project. [y/N]". See The vitals command line.
Choose what each project sends
Under Each project in This Mac → Privacy, each linked project has three switches. They apply to uploads from now on.
- Failure details (on): "Test names and a first error of at most 2 KB, with expected and received values removed. Off sends counts only." Off, a guide walk's findings go without their text too.
- Screenshots (off): adds the screenshots a check keeps, so a teammate on the web can see them. A screenshot shows whatever the app showed, including any folder paths on screen.
- Reports and traces (off): adds reports, a check's saved output and traces. Traces can hold your app's network requests and responses and your test files. Files nothing can look inside, like videos, stay here.
- Tick or untick a switch. The page says "Saved."
- With Screenshots or Reports and traces on, the project says who can see them: "Everyone in … who can see … can see what's uploaded."
Text files and reports are scrubbed and have their paths cut to file names before they go; a trace's archive is opened and every entry looked at. A file that holds a known secret or a folder path stays on your machine.
How long uploads are kept
Screenshots, reports and traces are kept by the workspace's plan: 7 days on Free, 30 days on Indie and Beta, 90 days on Studio and Agency. The Privacy page says it for your workspace, for example "Screenshots, reports and traces are kept 30 days on Beta."
Share one failure
With the switches off, evidence of a failure stays on the machine that ran it. On the web, the check says where, for example "The screenshot is on MacBook Pro".
To send one failure's evidence without turning anything on:
- In the desktop app, open the failed check in a run this machine ran for ShipVitals.
- Press Share this failure. "This sends this one failure's evidence to ShipVitals, so a teammate on the web can see it. Nothing is turned on for the project."
- Read the preview, then press Share. The check says "Shared. It's on the web with this run." If it can't be sent yet, it says "Ready to share. It goes up when it can be sent, and retries until then.", and the rest of your results keep going up meanwhile.
If the failure kept nothing more, the sheet says "This failure kept nothing more to share."
Keep a run on your machine
vitals run --no-uploadruns the checks and sends nothing.- Without signing in,
vitalskeeps every result on this machine.vitals statussays "Not signed in. Results stay on this machine." On a machine whose runner is paired (by the desktop app, orvitals login --token), it says "vitals isn't signed in. This machine's runner still sends its results." - Offline, results wait and send, in order, when ShipVitals can be reached.
vitals upload is different: it sends the test reports you name from CI or a laptop, test names and failure messages included, and any --artifact files as they are. The project switches above don't apply to it. Before they go, test names and failure messages have key-shaped values, home folders and email addresses replaced with [hidden]; their file paths stay as the report wrote them. See Send test results and your own metrics.
Walking the guides
Walk the guides uses a model, on your own OpenRouter key.
- The key is kept in this machine's keychain, read only when the walker calls the model, and never uploaded.
- What the model sees: the guide's text, the test persona's name and role, the address of the site it walks, and the page the walker's browser shows (its structure and a screenshot), sent to OpenRouter. Every call asks for zero data retention, and OpenRouter routes it only to an endpoint that keeps no prompt. A test persona's password is typed by ShipVitals, never shown to the model.
- What goes to ShipVitals is the walk's Confirmed findings, under Failure details: on, each one's text, scrubbed, with paths cut to a file's name and line; off, only its id, severity, category, guide, step and its screenshots' file names. A walk's screenshots stay on your machine.
Relays
A self-hosted service on your network (such as Umami) can be read by an always-on machine there, which sends ShipVitals what it reads: the service's sites (their names and ids) and their figures, how far it has read, and, when a sync fails, a scrubbed reason. Its address, user and password are entered on that machine only (This Mac → Relays, or vitals relay add), tested from there and kept in its keychain. They never leave it. Each sync is checked for the password before it goes: a password of 8 characters or more found in the data stops that sync ("the last sync stopped: the password appears in its data."), and a shorter one is replaced with "…".