Send test results and your own metrics
Two kinds of data come from you rather than from a service: test results from your CI, which show in a project's health, and metrics your own servers push, such as active users. Both are set up per project, in Connections.
Only the owner and admins can set them up (see Connect a service).
Test results from CI
Open it
- In Connections → All services, under Entered by hand, or sent by you, press Test results from CI. Under Choose a project, press the project.
- Or press Set up CI on a project's Overview ("No test results yet from …"; see Projects) or on a repo's row in the binding review (Review bindings).
The page says Test results for …: "Each test run is uploaded after the tests, so the project's health shows them."
With GitHub Actions
- Under GitHub Actions, copy the step under Add this step after your tests (press Copy snippet).
- Add it to your workflow after the step that runs your tests, and push.
"No secret needed. The action proves which repo it's running in, and ShipVitals checks it against your GitHub install."
While no run has arrived, the page says "Waiting for the first run…" and checks again every few seconds. When the repo's first workflow run lands, the line turns into "First run · 318 passed · 2 failed · 4m 12s".
Press Done to go back to Connections.
If the project has no GitHub repo bound: "… has no GitHub repository bound to it. Connect GitHub and bind one, or upload with a token below." If the GitHub App doesn't cover the repo: "The GitHub App doesn't cover this repository yet, so its runs would be refused. Add it to the install, or upload with a token below."
From your machine or another CI
An upload token belongs to one project.
- Under Local or other CI, press Create an upload token.
- Type a Token name that says where it's used (like "release script" or "CircleCI") and press Create an upload token.
- The token appears once: "You won't see this token again. Store it as
SHIPVITALS_TOKEN." Press Copy the token and store it in your CI's secrets or your environment asSHIPVITALS_TOKEN. - Press I've stored it. Hide it.
- Copy the command under Upload the results (a
vitals uploadcommand for your test runner) and run it after your tests.
The project's tokens are listed under Upload tokens, each with "last used …" or "not used yet".
Name the surface the tests cover
Without it, an upload's results are the whole project's. To have them fill one surface's row under Surfaces on the project's Health tab ("Test results on the main branch fill this, from CI or a vitals upload that names this surface."), name the surface:
- Add
--surface <platform>to thevitals uploadcommand, for example--surface web. The platform is one ofweb,api,mcp,ios,android,tvos,cast,desktop-mac,desktop-win,desktop-linux,cli,docker,browser-extension,bot,ha-addonorlibrary. - If the project has more than one surface of that platform, add its address or id after a colon:
--surface web:app.example.com. An address matches without itshttps://,www.or case. - With GitHub Actions, give the step
surface: webunderwith:instead.
When no surface of the project matches, or several do and no address chooses one, the results stay the project's. An unknown platform, or an unknown --format, is a warning ("vitals: unknown surface …"): vitals upload and the GitHub Action upload the results without it (a format is then told from each file). With --strict (strict: true in the Action) it stops before anything is sent: vitals upload exits 3, and the Action's step fails.
Revoke an upload token
- Press Revoke … next to the token.
- The row asks "Revoke it? Uploads with it stop at once." Press Revoke (or Cancel).
Push metrics
For a project with no analytics service: "Report your own active users. Send aggregates only; ShipVitals never sees who your users are."
Create a push key
- In Connections → All services, press Push metrics, then choose the project. The page says Push metrics for …: "No push key yet. Create one to get the snippets."
- Press Create a push key, type a Key name (like "api server") and press Create a push key.
- The key appears once: "You won't see this key again. Store it as
SHIPVITALS_PUSH_KEY." Press Copy the key and store it on your server asSHIPVITALS_PUSH_KEY. - Press I've stored it. Hide it.
A project has two active keys at most, so you can rotate one. With two, the page says "A project has 2 active keys at most. Revoke one to make another."
Send a metric
Under each key, choose curl, TypeScript or Python and copy the snippet. It signs each request with the key, which it reads from SHIPVITALS_PUSH_KEY; the snippet itself names only the key's id. Add it where your server computes the figure.
To check the key works, press Send a test metric. The page says "Sending a test metric…", then "It arrived: test.ping = 1 on …."
Revoke a push key
- Press Revoke … next to the key.
- The row asks "Revoke it? Pushes signed with it are refused at once." Press Revoke (or Cancel).
When it doesn't work
- The page says
- "There's no active project yet. Create one in Projects first."
- What to do
- Make a project (see Projects), then come back.
- The page says
- "The test metric wasn't sent. Try again in a moment."
- What to do
- Wait a moment and press Send a test metric again.
- The page says
- "The test metric was sent, but it wasn't stored. Try again."
- What to do
- Press Send a test metric again.
- The page says
- "Push metrics aren't available on this deployment."
- What to do
- This ShipVitals server doesn't take pushed metrics.
- The page says
- "This didn't refresh, so what's shown may be out of date."
- What to do
- Reload the page.
- The page says
- "That didn't work. Try again in a moment."
- What to do
- Wait a moment and try again.
| The page says | What to do |
|---|---|
| "There's no active project yet. Create one in Projects first." | Make a project (see Projects), then come back. |
| "The test metric wasn't sent. Try again in a moment." | Wait a moment and press Send a test metric again. |
| "The test metric was sent, but it wasn't stored. Try again." | Press Send a test metric again. |
| "Push metrics aren't available on this deployment." | This ShipVitals server doesn't take pushed metrics. |
| "This didn't refresh, so what's shown may be out of date." | Reload the page. |
| "That didn't work. Try again in a moment." | Wait a moment and try again. |