Walk the guides
The desktop app and vitals walk can walk your project's how-to guides step by step in a browser on your machine, as a careful reader would, and report what doesn't match. They use a model on your own OpenRouter key. Findings aren't coverage: a walk finds some defects and proves nothing about the rest.
To walk your staging site from ShipVitals' cloud instead, paid with your workspace's AI credits and with no key or machine of your own, see Walk the guides in the cloud. This guide covers walking on your machine: in the desktop app, then with vitals walk.
Before you start
- A linked project. Link the project's folder on this machine first (see Run checks). Its guides are read from that folder.
- Guides the walker can read: Markdown files with numbered steps, in
docs/how-toordocs/guides. - An OpenRouter key. You pay OpenRouter for the model calls; ShipVitals doesn't.
- Optional: a
.shipvitals/walk.ymlin the project, which names the default target, test personas and how to reset fixture data:
target: http://localhost:5173
personas:
- { name: Test Owner, role: owner, email: owner@example.test, password: … }
reset: { path: /__e2e/reset }A persona's email must be on a test domain (such as .test or example.com), so a real account is never offered. reset is posted to on the target before each guide.
Set up the walk
- In the desktop app, open Runs and press Walk the guides. The page says Walk the guides.
- If several projects are linked here, choose one under Project.
- Under Your OpenRouter key, copy your key, then press Paste your key. The page says "Your OpenRouter key is saved on this Mac." It's kept in this machine's keychain and never uploaded. Forget the key removes it.
- Under The walker's browser, press Download the browser the first time (about 150 MB, into ShipVitals' folder) and agree in the dialog. The page then says "The walker's browser is on this Mac."
- Check Target: "This Mac (localhost) or a host you name as staging." A staging host's name has to say so, such as
staging.,dev.,test.,qa.orpreview.. - Choose a Test persona from the project's walk file, or Signed out.
- Under Guides, untick any guide you don't want walked. Each shows its number of steps.
- Choose a Model. Each says what a step costs, as "… · about $0.004 a step". Only models with a zero-data-retention endpoint are offered.
- Set the Spending limit (USD): $2 to start with, at most $50. "The walk stops cleanly at this, and reports what it covered." If your key has its own limit, the page says how much is left under it.
- If the project's walk file has no reset, the page warns that what one guide changes is still there for the next. Tick Walk without a reset to go on.
Start it
- Read the summary above the button, for example:
- "Walk 8 guides on web · as Test Owner (owner) · against localhost:5173"
- "Uses your OpenRouter key · … · about $0.30 · limit $2.00"
- "The walker can only open localhost:5173. It can't run shell commands."
- Press Start walk.
Cancel goes back to Runs.
The bounds
- Origins: the walker's browser opens only the target. Anything else it asks for is stopped, and the board says how many: "The walk's guard stopped 3 requests outside localhost:5173."
- No shell: it has browser tools only.
- Spending: every model call is reserved against the limit before it's made; one that wouldn't fit isn't made.
- Steps: each guide gets a limited number of model turns. A guide that runs out says "The walk's step limit ended this guide first."
- Data: the fixture reset runs before each guide when the walk file names one, and
DELETErequests are refused.
Watch it
The page says Walking …, with what's spent of the limit ("$0.0812 of $2.00") and Stop.
Each guide shows as a checklist. Each step is Waiting, then Pass, Fail, Warn, Blocked or Unwalked, with the walker's note under it. The current step shows the screenshot of what the walker sees.
It ends with one of:
- "The walk finished."
- "The walk stopped at its spending limit. What it didn't reach is Unwalked."
- "You stopped the walk. What it didn't reach is Unwalked."
Read the findings
When the walk ends, the page says Walked … and lists Findings, or "Nothing reproduced as a finding."
Each finding shows its id, its severity (S1 Blocker, S2 Major, S3 Minor or S4 Cosmetic), the guide and step, then Expected and What happened. Each one was tried again from a fresh start: Confirmed means it happened again, Observed once means it didn't. Confirmed ones are listed first.
The text is quoted as the walker wrote it: a page or a guide can say anything.
Act on a finding
- Open a GitHub issue opens a new issue on the project's GitHub repository, filled in. If the repository is public, or ShipVitals can't tell, it says so first; press Open the issue to go on. A finding too long for the link is put on your clipboard to paste. Without one GitHub repository bound to the project, the page says "Bind one GitHub repository to this project to file issues from here."
- Dismiss takes the finding off the list. The page says "Dismissed."
Confirmed findings also go to ShipVitals, under the project's Failure details switch, and show on the project's release gate (see The release gate). A finding you dismiss afterwards is sent as dismissed. See What leaves your machine.
Save or hand off the result
- Save report writes
QA_REPORT_<YYYYMMDD-HHMM>.mdat the root of the linked folder, with the findings' screenshots inqa/shots/. The page says "Saved … in the project's folder." The report is in the format theqa-repairskill reads. - Copy for your agent puts the findings on your clipboard, fenced as quoted evidence. The page says "Copied. The findings are fenced as quoted evidence for your agent."
- Done goes back to Runs.
With vitals walk
vitals walk walks the guides from a terminal, with the same engine, key, browser and bounds as the desktop app. Run it in a linked folder (see The vitals command line); if several projects are linked here, name one: vitals walk <project>.
- The first time, run
vitals walk --set-key --install-browser.--set-keyasks for OpenRouter key: without showing what you type (or pipe it in), then says "Your OpenRouter key is saved in this machine's keychain. It's never uploaded."--install-browserdownloads the walker's browser (about 150 MB, into ShipVitals' folder): "Downloading the walker's browser…". The walk then starts as in step 2. - After that, run
vitals walk. It walks every guide, on the walk file's target (elsehttp://localhost:5173), as its first test persona, with a limit of $2. It says, for example, "Walking 8 guides on http://localhost:5173 as Test Owner, with your OpenRouter key, limit $2.00." and "The walker can only open http://localhost:5173. It can't run shell commands." - It prints each step as it's marked, such as
✓ sign-in · 1 Open the sign-in page(✓pass,✗fail,!warn,⊘blocked,◌unwalked). - When it ends it prints how it ended, "1 finding confirmed · $0.0812 spent of $2.00", and "Saved … for qa-repair.":
QA_REPORT_<YYYYMMDD-HHMM>.mdin the linked folder, as Save report writes it.
--guide <slug>walks only that guide; give it more than once for several.--persona <name>,--target <address>,--limit <dollars>and--model <id>set what the desktop app's page sets.--without-resetwalks when the walk file names no reset.--no-savedoesn't save the report.- Ctrl-C stops the walk before its next turn, and it still reports what it covered.
--cloudwalks in ShipVitals' cloud instead, with the workspace's AI credits: see Walk the guides in the cloud.
It exits 0 when the walk finished with nothing found, 1 when it found something, 2 when it stopped short, and 3 when the walk didn't start (a refusal or a wrong option). Without a key it says "Walking on this machine uses your own OpenRouter key: vitals walk --set-key reads it from stdin and keeps it in this machine's keychain. Or walk in ShipVitals' cloud with --cloud."; without the browser, "The walker's browser isn't downloaded yet: vitals walk --install-browser downloads it (about 150 MB, into ShipVitals' folder)."
When it doesn't work
- The page says
- "Link a project on this Mac first: its guides are read from its folder."
- What to do
- Link the project's folder, then come back.
- The page says
- "This project has no guides the walker can read: …"
- What to do
- Add Markdown guides with numbered steps in
docs/how-toordocs/guides.
- The page says
- "The clipboard doesn't hold an OpenRouter key. Copy it, then try again."
- What to do
- Copy the key, then press Paste your key.
- The page says
- "Downloading the browser needs Node 24 or newer on your PATH, …"
- What to do
- Install Node 24, then press Download the browser again.
- The page says
- "The browser didn't download. Check your connection and try again."
- What to do
- Press Download the browser again.
- The page says
- "OpenRouter didn't answer. Check the key and try again."
- What to do
- Check the key on OpenRouter, then paste it again.
- The page says
- "No model on OpenRouter meets the walker's needs with zero data retention."
- What to do
- Try again later.
- The page says
- "The walker opens only this Mac (localhost) or a host whose name says it's staging …"
- What to do
- Change Target.
- The page says
- "The target can't have a user name or password in it."
- What to do
- Take them out of Target; use a test persona.
- The page says
- "That staging host didn't resolve to an address off this Mac."
- What to do
- Check the host name.
- The page says
- "Your OpenRouter key has nothing left to spend."
- What to do
- Add credit or raise the key's limit on OpenRouter.
- The page says
- "The limit must be more than $0 and at most $50."
- What to do
- Change Spending limit (USD).
- The page says
- "A walk is already running on this Mac."
- What to do
- Wait for it to end, or stop it.
- The page says
- "Walking guides needs the ShipVitals app's own engine. …"
- What to do
vitals agentruns checks on this machine, and it doesn't walk. Stop it withvitals agent stop, then quit and reopen the app.
- The page says
- "The report wasn't saved: …"
- What to do
- Fix what it names, then press Save report again.
| The page says | What to do |
|---|---|
| "Link a project on this Mac first: its guides are read from its folder." | Link the project's folder, then come back. |
| "This project has no guides the walker can read: …" | Add Markdown guides with numbered steps in docs/how-to or docs/guides. |
| "The clipboard doesn't hold an OpenRouter key. Copy it, then try again." | Copy the key, then press Paste your key. |
| "Downloading the browser needs Node 24 or newer on your PATH, …" | Install Node 24, then press Download the browser again. |
| "The browser didn't download. Check your connection and try again." | Press Download the browser again. |
| "OpenRouter didn't answer. Check the key and try again." | Check the key on OpenRouter, then paste it again. |
| "No model on OpenRouter meets the walker's needs with zero data retention." | Try again later. |
| "The walker opens only this Mac (localhost) or a host whose name says it's staging …" | Change Target. |
| "The target can't have a user name or password in it." | Take them out of Target; use a test persona. |
| "That staging host didn't resolve to an address off this Mac." | Check the host name. |
| "Your OpenRouter key has nothing left to spend." | Add credit or raise the key's limit on OpenRouter. |
| "The limit must be more than $0 and at most $50." | Change Spending limit (USD). |
| "A walk is already running on this Mac." | Wait for it to end, or stop it. |
| "Walking guides needs the ShipVitals app's own engine. …" | vitals agent runs checks on this machine, and it doesn't walk. Stop it with vitals agent stop, then quit and reopen the app. |
| "The report wasn't saved: …" | Fix what it names, then press Save report again. |