- Runs your CI starts. A push to your headline branch or a pull request runs the Peeps workflow, which reports each result to Peeps. You don’t need to do anything in Peeps.
- Runs you start from Peeps. Peeps starts the Peeps workflow in your CI with the tests you picked, and reports the results back.
api scope on GitLab. The Repository page shows whether it has them.
Run every test
On the project’s Repository settings page, click Run all in your CI. Peeps starts one batch with every test on the headline branch. When the button is disabled, its tooltip says why: usually a missing permission, or no tests on the headline branch yet. You can also comment/peeps run on an issue or pull request in the repository. That runs every test on the headline branch and replies with the batch number.
Run some tests
1
Pick the tests
On Test cases, select the tests you want and click Run or schedule. For one test, open it and click Run.
2
Pick the branch
The dialog says where the run goes: Runs in your CI on main — Peeps starts your workflow and reports the results here.When Peeps follows more than one branch, pick the Branch to run on. Only the tests on that branch run. See Tracked branches.
3
Start the run
Click Run now, or Run test for one test. The runs appear in Peeps straight away as pending, and fill in as your CI reports them.
- No environment. The workflow in your repository decides what the tests run against. Environments and variables in Peeps are not sent to your CI.
- No script version. The test file on the branch is what runs.
- No notification recipients. Peeps reports the result on the run, and in your organization’s Slack if it’s connected.
Where results show
Each test’s run appears on its test case page like any other run. The run page shows Ran in your CI, with the branch and commit, linking to the job in your CI.- Evidence. Peeps uploads your Playwright HTML report, with any traces, screenshots and videos your config records, so the run page shows them. For a pytest suite, it shows pytest-playwright’s output, plus a Test evidence card with the failing phase, the exception, the traceback, captured output, values recorded with
record_property, and files the test saved. - Failure analysis. A failed run is analyzed as any other is, from that evidence.
- Batches. Each job that runs the Peeps action is one batch, so a push or a pull request usually shows up as one batch of results.
- Branches. Runs keep the branch they ran on. A test case’s Branches card shows the latest result on each branch Peeps follows, with Show runs on this branch.
Limits
- 2,000 tests per run. A run from Peeps with more is cut to 2,000, and Peeps says how many didn’t fit. A run your CI reports with more than 2,000 tests under the project’s test root isn’t recorded, and the Repository page says A run in your CI was not recorded. Split the suite across several jobs, or give each Peeps project its own test root.
- Cancelling. Cancelling a run that Peeps started cancels its workflow run in your CI, which stops every other test in that batch too. Cancelling a run your own CI started stops it only in Peeps, so the rest of your pipeline carries on.
What can’t start a run yet
These start runs on a project whose tests Peeps hosts, but can’t reach your CI yet:- Schedules. Your CI decides when tests run. An existing schedule shows Not running: this project runs in your CI.
- Incoming webhooks and deployment triggers.
- Pull request checks set up in Peeps. Your own workflow’s
pull_requesttrigger reports the run on every pull request instead. - Retry. A run in your CI can’t be retried on Peeps’ runners. Start a fresh run instead.
schedule trigger to your workflow. Those runs report to Peeps like any other.