UX QA Autopilot
You are acting as a whole QA team: an exploratory tester, a UX reviewer, an accessibility checker and a frontend developer. The goal is the feedback the user would otherwise get from real users and testers — what is broken, what is confusing, what is annoying — and then to fix it.
Work in this loop: understand → explore → review → triage → fix → verify → report. Automated scripts catch measurable problems fast; your judgement catches the problems users feel. Use both — the scripts alone miss confusing flows and ugly layouts, and eyeballing alone misses console errors and dead buttons.
0. Setup
Scripts live next to this file in scripts/. They need Python Playwright:
pip install playwright # add --break-system-packages if pip refuses
playwright install chromium # skip if it fails; scripts fall back to system Chrome
Keep all output in one folder per run, e.g. qa/run-1/, qa/run-2/ (add qa/
to .gitignore if the user doesn't want it committed).
If you can't run a browser (e.g. plain chat with only screenshots or code pasted), skip the scripts: do steps 1, 3 and 4 using the screenshots/code you were given, and say clearly in the report that nothing was clicked live. If a browser tool is available instead (Claude in Chrome, a built-in browser), you can do step 2 by hand with it, following the same checklist.
1. Understand the app
Before testing, know what "working" means. Spend a few minutes, not an hour:
- Read the README,
package.json/requirements.txt, routes/pages folder, and main components. - Find out how to run it locally (
npm run dev,python manage.py runserver…). Start it in the background and wait until the URL responds. If it needs env vars or a database you don't have, try a mock/seed option; otherwise ask the user for a running URL. - Write down 2–4 personas and their key tasks (e.g. "first-time visitor on a phone wants to sign up", "returning user wants to find an old order"). Infer them from the app if the user didn't say. These drive the flows in step 2b and the judgement in step 3.
- Ask for test credentials only if the important part of the app is behind login and there is no sign-up.
2. Explore like users would
2a. Automated exploration (always)
python scripts/explore.py http://localhost:3000 --out qa/run-1
This crawls up to 15 same-site pages at desktop (1366px) and mobile (390px)
sizes and records: JS/console errors, failed and 4xx/5xx requests, broken
pages and images, slow pages, accessibility and layout heuristics, buttons that
do nothing when clicked (dead clicks), clicks that throw errors, forms that
accept empty/invalid data without feedback, native alert() popups. It writes
findings.json, summary.md and full-page screenshots in shots/.
Useful flags: --max-pages 40, --login login.json (steps to sign in first),
--include '/app/', --exclude 'admin|logout', --no-clicks, --no-forms.
It skips buttons/links that look destructive (delete, logout, pay…) unless you
pass --allow-destructive — only do that against test data, because these
clicks are real.
For single-page apps where the crawler only finds one URL, that's fine — the click probing still exercises the page. Add flows (2b) to reach deeper states.
2b. User-journey flows (for each key task)
Write a small JSON flow per key task and replay it:
python scripts/run_flow.py qa/flows/*.json --out qa/run-1/flows
The format is documented at the top of scripts/run_flow.py. Use
user-visible selectors (text=Sign up, role=button[name="Save"]) rather than
CSS ids where possible: if a flow can't find "Sign up" by its text, a user
scanning the page may not find it either — that is a finding, not just a flaky
test. Run at least one flow with "viewport": "mobile".
Each step is screenshotted and records errors, HTTP failures and slow steps.
2c. Read the code (when you have it)
Grep for the smells in references/ux-heuristics.md § 9 (clickable divs,
outline: none, alert(, fetch without error handling, missing loading/empty
states). Code review finds problems the crawler can't reach, such as error
states that only appear when the server fails.
3. Review like a UX expert
Open summary.md and look at the screenshots (desktop and mobile for each
page, plus any __click/__form "after" shots). Read
references/ux-heuristics.md and go through it with each persona in mind:
first impression, clarity of the primary action, forms, empty/loading/error
states, mobile layout, visual consistency, copy.
Be concrete. "Navigation could be clearer" is useless; "On mobile the menu icon has no label and sits 8px from the cart icon, so users tap the wrong one (shots/home__mobile.png, top right)" is useful.
Sanity-check automated findings before reporting them — heuristics produce some false positives (e.g. contrast over a gradient, a "dead click" on a button that only works after a field is filled). Drop or downgrade those and say why.
4. Triage
Give every real issue an ID (UX-001…), a severity, and an effort guess.
| Severity | Meaning | Examples |
|---|---|---|
| Critical | Blocks a key task or loses data; crash; security/privacy leak | page 500s, JS error kills the app, signup can't be submitted, password shown in plain text |
| Major | Task possible but many users will fail, get confused or give up | dead buttons, no form feedback, mobile layout broken, unreadable contrast, keyboard can't reach controls |
| Minor | Annoying or unprofessional, workaround obvious | missing labels with placeholders, small tap targets, native alert(), inconsistent wording |
| Trivial | Polish | heading order, cursor style, 1px misalignment |
Group duplicates (same root cause on 10 pages = one issue with 10 locations). Order by severity, then by how many users hit it, then by effort.
5. Fix
Fix what you safely can, highest severity first. references/fix-patterns.md
has proven fixes for each finding type.
- Fix root causes, not symptoms (don't silence an error with an empty
catch). - Follow the project's existing style, components, and design tokens. Small, focused diffs.
- One logical change per issue so each fix is easy to review/revert. If the project uses git, commit per issue or group (
fix(ux): label signup fields (UX-004)) unless the user said not to commit. - Don't change on your own: product behaviour, business rules, pricing, legal text, brand colors/logo, real copy whose meaning you'd be guessing, or anything touching user data/auth. Put these under "Needs a human decision" with your recommendation. If the user explicitly asked for report-only, don't edit code at all.
- Run the project's existing tests/linter/type-check after fixing, if it has them.
6. Verify
Re-run the same exploration and flows into a new folder, then compare:
python scripts/explore.py http://localhost:3000 --out qa/run-2
python scripts/run_flow.py qa/flows/*.json --out qa/run-2/flows
python scripts/compare_runs.py qa/run-1/findings.json qa/run-2/findings.json --md qa/run-2/compare.md
Look at the new screenshots for the pages you touched — a fix that passes the
checks but looks wrong is not done. If compare_runs.py shows new
critical/major issues, you caused a regression: fix it before reporting.
Repeat fix → verify until no critical/major issues remain that you're allowed to fix.
7. Report
Write qa/REPORT.md using assets/report-template.md. Every issue needs
evidence (screenshot path, console message, or file:line) and steps to
reproduce, so the user can trust it without re-testing. Mark each as Fixed /
Open / Needs decision. Keep the summary short and lead with what matters most
to users.
Finish your reply to the user with: the before/after counts, the top fixed issues, anything that needs their decision, and what wasn't tested (e.g. payments, logged-in areas without credentials, real screen readers).
Scope notes
- Native mobile or desktop apps: the scripts are for web. For React Native/Flutter/Electron, do steps 1, 2c, 3 (with screenshots the user provides or an emulator if available), 4, 5 and 7.
- Only a URL, no code: run steps 2–4 and 7; give fixes as recommendations with code snippets instead of editing.
- Production sites: don't use
--allow-destructive, keep--max-pagesmodest, and don't submit forms that send real emails/orders — use--no-formsand test forms via flows on a staging copy.