Guide · Agent skill
How to use the Fix Flaky Playwright Tests agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The fix skill is a packaged set of instructions that tells an AI agent how to diagnose and repair a Playwright end-to-end test that fails or passes only some of the time. It is useful for developers and QA engineers who keep seeing a test go red in CI while it passes on their own machine. After this guide you will know what the skill does, how to install it from the ZIP or the Claude apps, how to ask for it in plain language, and what to check before letting it run commands on your project.
Fix Flaky Playwright Testsalirezarezvani/claude-skills · MIT Download the skill Quick start
- 1Download and unzip the skill. Download the ZIP from this page and unzip it so that SKILL.md and flaky-taxonomy.md sit in a fix folder.
- 2Place it in a skills folder. Move the folder into ~/.claude/skills/ for personal use, or into a project's .claude/skills/ folder. In the Claude apps, you can upload it under Settings → Capabilities → Skills instead.
- 3Confirm project requirements. Make sure Node.js with npx is available and the Playwright test runner is installed in the project that contains the failing test.
- 4Describe the failure. Ask the agent in plain words, for example "the checkout test fails in CI but passes locally", and include the test file path or test name if you have them.
- 5Let the agent reproduce and categorize. The agent reruns the test, captures a trace, and sorts the failure into timing, isolation, environment or infrastructure.
- 6Review the diff. Read the proposed fix before accepting it, since the agent edits your test code.
- 7Check the 10/10 result and prevention advice. Confirm the test passed all ten verification runs, then consider the suggested CI retry and trace settings.
What the fix skill makes the agent do
When the skill is active and you ask the agent to fix a broken or unreliable Playwright test, the agent follows a fixed routine rather than guessing. It first reruns the test to reproduce the failure, sometimes many times or with parallel workers. It then captures a trace, sorts the problem into one of four categories, applies a fix that matches that category, and reruns the test ten times to confirm it is stable.
The four categories are timing or async problems, test isolation problems, environment differences between CI and local runs, and browser infrastructure problems. The skill also suggests changes that stop the problem from coming back, such as enabling retries in CI or recording traces on the first retry.
Install the fix skill
The skill is distributed as a ZIP file from the skill's page in this directory. You can install it in one of two ways.
For Claude Code, unzip the download into your personal skills folder at ~/.claude/skills/, or into a project-specific .claude/skills/ folder inside the project you are working on. The skill's folder contains SKILL.md and flaky-taxonomy.md, and both files need to stay together.
In the Claude apps, open Settings, go to Capabilities, and upload the skill under Skills.
- Personal install: ~/.claude/skills/ (available in all your projects)
- Project install: .claude/skills/ inside the project (shared with anyone using that repo)
- Claude apps: Settings → Capabilities → Skills
Check that your project meets the requirements
The skill expects Node.js with npx available, the Playwright test runner installed in the project, and a Playwright test project that contains the failing test file. Without these, the commands the skill runs will not work, so confirm them before you start.
- Node.js with npx
- Playwright test runner installed in the project
- A Playwright test project containing the failing test file
Ask for the fix with a natural request
You do not need a special command. The skill is triggered by phrases such as "fix test", "flaky test", "test failing", "debug test", "test broken", "test passes sometimes", or "intermittent failure". Give the agent as much context as you can: the test file path, the test name, and a short description of the symptom.
Useful inputs include a file path such as e2e/login.spec.ts, a test name such as "should redirect after login", or a description such as "the checkout test fails in CI but passes locally".
- "Fix the flaky test in e2e/login.spec.ts"
- "The checkout test fails in CI but passes locally"
- "This test passes sometimes, can you debug it?"
- "Intermittent failure in should redirect after login"
Understand what the agent runs
The skill's steps are concrete Playwright commands. The agent first runs the test with a list reporter. If it passes, the agent repeats it ten times to look for flakiness, and if it still passes, it runs with four parallel workers. Next it runs the test with full tracing and no retries so that the failure is recorded.
The skill's categorization table maps each symptom to a check. For example, a timing problem reproduces with --repeat-each=20, and a test isolation problem is one that passes when run alone with --workers=1 and --grep. The final verification step runs the test ten times, and every run must pass.
npx playwright test <file> --reporter=list
npx playwright test <file> --repeat-each=10 --reporter=list
npx playwright test --fully-parallel --workers=4 --repeat-each=5
npx playwright test <file> --trace=on --retries=0
npx playwright test <file> --repeat-each=10 --reporter=listRead the output and apply the fix
The skill reports the root cause category and the specific issue, the fix applied with a diff, the verification result (10/10 passes), and a prevention recommendation. Review the diff before accepting it, since the agent edits your test code.
Typical fixes in each category include replacing waitForTimeout() with web-first assertions, creating test data per test, matching viewport sizes between local and CI, and increasing timeouts or reducing parallel workers on slow runners.
- Timing/Async: replace waitForTimeout() with web-first assertions, add missing await, wait for network responses
- Test isolation: remove shared mutable state, create per-test data, check for database state leaks
- Environment: match viewport sizes, handle font differences, check timezone-dependent assertions
- Infrastructure: increase timeouts for slow CI runners, reduce parallel workers if the browser runs out of memory
What people use it for
- Diagnosing a login test that passes on a laptop but fails intermittently in CI
- Stabilising a checkout test that breaks only when the suite runs in parallel
- Replacing hard-coded waitForTimeout() calls with web-first assertions across a flaky spec
- Tracking down a test that leaks cookies or localStorage into the next test
- Standardising how a QA team triages intermittent Playwright failures
Frequently asked questions
Is the fix skill free?
The skill is released under the MIT licence, which is stated on this page. Check the licence file in the source repository for the full terms before redistributing it.
What do I need to use the fix skill?
You need Node.js with npx, the Playwright test runner installed in your project, and a Playwright test project that contains the failing test file. The skill runs scripts, but it does not use the network.
Where do I install the fix skill?
Unzip the download into ~/.claude/skills/ for personal use or into a project's .claude/skills/ folder. In the Claude apps you can upload it under Settings → Capabilities → Skills.
Does the fix skill work with frameworks other than Playwright?
No. The commands and fixes are specific to Playwright, so it will not help with projects that use other test tools.
Can the fix skill hide a real bug?
Yes, if misused. The skill aims to make tests stable, so it could mask a genuine application defect where the test is correctly catching a problem. Check the failure before accepting a fix.
Is it safe to run on a production or shared environment?
Be careful. The routine repeatedly runs test commands, sometimes up to ten or more repetitions with four workers, which can add load and may modify test data. Use a local or dedicated test environment.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Choose the fix skill if you maintain a Playwright suite with intermittent failures, especially tests that pass locally but fail in CI, and you want a consistent triage checklist. Skip it if you do not use Playwright, if the test is catching a real application bug, or if you would have to run it against a shared or production environment.
Download the skill