Skip to main content

​
The CI job never connected

Peeps started your workflow, but no job reported back. When Peeps is fixing or writing a test, it waits up to 10 minutes for the job, then stops and says The CI job never connected. When you started a run, its runs stay pending until Peeps closes the batch, about two hours later. Either way, open the workflow’s latest run in your repository’s Actions tab (on GitLab, Build > Pipelines) to see why. The usual causes:
  • The workflow isn’t on the branch. GitHub runs the workflow file as it is on the branch Peeps asked for. Commit it to your headline branch, and to any other branch you run on or generate onto.
  • The file name doesn’t match. Peeps starts the file named in Workflow file on the project’s Repository settings page, peeps.yml by default.
  • The workflow was changed. Peeps needs the workflow_dispatch trigger with its mode, sessionId and ref inputs, id-token: write, and the job’s name as the page gives it. Compare yours with the copy on the Repository page.
  • The job never started. A broken npm ci, invalid YAML, or a runner that’s busy or offline all stop the job before it reaches Peeps.
  • GitLab refused the pipeline. Peeps starts pipelines with variables, which new gitlab.com projects don’t allow. The Repository page says so and offers Allow Peeps to start pipelines.

​
Generation failed

The test case’s card says why, and links to the job with View the CI run. See when generation fails.

​
Peeps finds no tests, or not all of them

  • Check the test root. Peeps reads tests under the project’s Test root. If your suite lives in a subdirectory, set the test root to it in the Settings card on the Repository page, save, then click Run inventory now.
  • Check the file names. When Peeps reads the repository itself, it looks for *.spec.ts, *.test.ts and their JavaScript variants, and for pytest’s test_*.py and *_test.py. If your config uses other patterns, such as *.e2e.ts, let your CI report the tests: every run of the Peeps workflow on your default branch sends the list Playwright itself sees. If your headline branch is another branch, run the workflow in inventory mode on it instead.
  • “This is not all of your tests.” Peeps reads up to 400 test files each time it scans the repository. The notice on the Repository page says how many it read. Point the test root at your suite, split the repository across several projects, or let your CI report the tests, which reads up to 2,000 files.

​
A test shows as missing

The test isn’t on the headline branch any more. The test case says whether the file was deleted, or whether the file is still there and the test isn’t, which usually means it was renamed. A renamed test is a new test case. See When a test goes missing. If the test case says Not on main — on staging, the test is on another branch Peeps follows and hasn’t reached the headline branch yet.

​
A run in your CI was not recorded

Peeps records up to 2,000 tests from one run. A run with more under the project’s test root isn’t recorded at all, and the Repository page says so. Split the suite across several jobs, or give each Peeps project its own test root.

​
Run all in your CI is disabled

Its tooltip says why. Usually Peeps lacks the Actions: write permission on GitHub, or a token with the api scope on GitLab. Grant it, then click Re-check permissions. The button is also disabled while there are no tests on the headline branch.

​
Are Python tests supported?

Yes, for pytest-playwright suites. Peeps reads them from your repository, runs them in your CI with the Python workflow, and records their results, with pytest’s own evidence on each run. Fixing a failing test and generating a new one work with TypeScript Playwright tests today.

​
My repository has more than one suite

Connect each suite to its own Peeps project, with its own test root. Several projects can be connected to the same repository. A project runs one workflow, in one language. If one test root holds both TypeScript and Python tests, the project runs the TypeScript tests in your CI and its Python tests can’t run there yet. To run both, give each language its own project, test root and Workflow file.

​
Can I edit a test in Peeps?

Not its code. A test in your repository changes through your repository: edit it there, or ask Peeps to fix it, which opens a pull request. You can still rename a test case, write its steps, and archive it. Those change how Peeps organizes the tests, not the repository.

​
Which /peeps commands are there?

Comment on an issue or pull request in the connected repository: ABC-12 is the id shown next to the test case in Peeps. Peeps reacts with 👀 when it takes a command and replies in the same thread. On GitHub, only people with write, maintain or admin access to the repository can run commands. On GitLab, project members with the Developer role or higher can. Edited comments and comments from bots are ignored.

​
My workflow fails on pull requests from forks

GitHub doesn’t give a workflow triggered by a fork’s pull request an identity token, so the job can’t authenticate to Peeps. Skip the job for pull requests from forks. See Common setups.

​
Can I move an existing project’s tests into my repository?

Not yet. Only an empty project can be connected. Use Create a project for your repository on the existing project’s Repository page to start a new project with the same environments, and connect that. See When the project already has tests.

​
What does Peeps see of my code?

Your test files, your BASE_URL if the workflow sets one, and the Playwright report from each run, including anything your traces record, such as cookies and tokens your tests obtained for your app. While Peeps fixes or writes a test, it also sees the files it reads and the output of the tests it runs, and it can run code on your runner. The Peeps action’s security notes list everything in detail.
Still stuck? Get in touch.