ui-test
browserbase/skills
利用 browse CLI 在真实浏览器中运行对抗性 UI 测试,通过分析 git diff 仅测试发生变更的区域,或全面探索整个应用程序,以发现功能、无障碍性、响应式布局和用户体验方面的缺陷。
...展开全部UI 测试 — Agentic UI 测试技能
在真实浏览器中测试 UI 变更。你的任务是设法找出问题,而不是确认其正常运行。
三种工作流:
- 差异驱动 — 分析 Git 差异,仅测试发生变更的部分
- 探索式 — 浏览应用程序,找出开发者未曾预料到的缺陷
- 并行测试 — 将独立的测试组分布到多个 Browserbase 浏览器中
测试工作原理
主代理负责协调——它规划测试策略、向子代理分配任务并合并结果。子代理执行实际的浏览器测试。
规划:多角度规划,随后一次性执行
在启动任何子代理之前,您必须亲自完成全部三轮规划并输出结果。规划工作在您自己的响应中进行——绝不能委托给子代理。请勿跳过规划阶段直接进入执行。
第一轮 — 功能测试:核心用户流程是什么?哪些功能应正常工作?将每项测试按“操作 → 预期结果”的格式写出。
第二轮 — 对抗性测试:重新阅读第一轮内容。你遗漏了什么?考虑以下方面:不同的用户类型/角色、错误路径、空状态、竞争条件、边界输入(空值、超大值、特殊字符、快速点击)。
第三轮 — 覆盖缺口:重新阅读第一轮和第二轮内容。请关注:无障碍访问(axe-core、仅键盘操作)、移动端视口、控制台错误、与应用其他部分的视觉一致性?
去重:将前三轮测试合并为一个编号测试列表。删除重复项。将每个测试分配到一个组(例如:A组、B组)。
然后执行一次——每个组启动一个子代理。每个子代理仅接收其特定的测试列表,不会执行其他操作。子代理不进行探索或规划——它们仅执行分配的测试并报告结果。
在调用任何 Agent 工具之前,请在响应中输出这三轮测试、合并后的计划以及分组分配情况。
任务拆分原则
- 子代理执行分配的测试,而非进行开放式探索。主代理向每个子代理分发一份特定的编号测试列表。子代理不进行规划、探索或决定测试内容——它们仅执行列表中的测试并停止。
- 瓶颈在于最慢的代理——因此需拆分工作,确保没有单个代理承担不成比例的工作量。多个小型代理 > 少数大型代理。
- 根据变更规模调整工作量——单个组件的修复不需要太多代理或太多步骤,而整页的重新设计则需要。让差异范围决定计划。
- 遇到失败时不要过早终止——在分配的测试范围内尽可能多地发现缺陷。
为子代理设定步骤预算
主代理必须在每个子代理的提示中明确设定浏览步骤上限。子代理不会自行限制——除非另有指示,否则它们会一直运行直到完成。
作为粗略的经验法则:针对少量定向检查约需 25 个步骤,针对包含功能性、对抗性及无障碍性测试的整页约需 40 个步骤,针对多页或广泛类别约需 75 个步骤。请根据分配的测试实际需求进行调整——这些仅是起点,而非硬性规则。
作为粗略的经验法则:针对少量定向检查约需 25 步,针对包含功能性、对抗性及无障碍性测试的完整页面约需 40 步,针对多个页面或广泛类别约需 75 步。请根据分配测试的实际需求进行调整——这些仅为起点,而非硬性规定。
每个子代理的提示必须包含:
You have a budget of N browse steps (each `browse` command = 1 step). Count your steps as you go. When you reach N, stop immediately and report:
- STEP_PASS/STEP_FAIL for every test you completed
- STEP_SKIP||budget reached for every test you didn't get to
Do not retry or continue after hitting the budget.
Run only these tests: [numbered list from the merged plan]
Do not explore beyond the assigned tests.
Do NOT generate an HTML report or write any files. Return only step markers and your findings as text.
主代理不应运行 browse (除非用于验证开发服务器是否正常运行)。所有测试均在子代理中进行。
当子代理达到预算上限时,主代理应原样接受部分结果。切勿重新运行或重试子代理。请在最终报告中包含“已跳过”的测试项,以便开发者知晓未覆盖的内容。
报告
每个子代理应报告以下内容:
Tests: 8 | Passed: 5 | Failed: 2 | Skipped: 1 | Pages visited: 2
主代理将这些内容合并为最终报告,其中包含:
Tests: 20 | Passed: 14 | Failed: 4 | Skipped: 2 | Agents: 3 | Pass rate: 70%
请勿报告“已用步骤数”——浏览命令的计数属于实现层面的细节,对评审者而言并非有意义的指标。
测试理念
你是一位对抗性测试人员。你的目标是发现缺陷,而非证明正确性。
- 尝试破坏你测试的每一项功能。不要仅仅检查“按钮是否存在?”——快速点击两次按钮、提交空表单、粘贴 500 个字符、在流程中途按下 Esc 键。
- 测试开发者未曾考虑到的场景:空状态、错误恢复、纯键盘导航、移动端内容溢出。
- 每项断言都必须有据可依。对比测试前后的快照。通过引用检查特定元素。除非有来自无障碍树或确定性检查的具体证据,否则切勿报告“通过”。
- 报告失败时需提供足够细节以便复现。包括确切的操作步骤、预期结果、实际结果以及建议的修复方案。
断言规范
每个测试步骤必须生成结构化的断言。请勿撰写“这看起来不错”这类自由形式的描述。
步骤标记
每个测试步骤必须生成且仅生成一个标记:
STEP_PASS||
或
STEP_FAIL|| → |
step-id:简短标识符,例如homepage-cta,form-validation-error,modal-cancelevidence:证明该步骤通过的观察结果(元素引用、文本内容、URL、评估结果)expected → actual:预期结果与实际结果的对比screenshot-path:已保存截图的路径(仅限失败情况——参见下文“截图捕获”)
失败情况下的截图捕获
每个 STEP_FAIL 都必须附带一张截图,以便开发人员直观地查看问题所在。
当测试步骤失败时:
# 1. Take a screenshot immediately after observing the failure
browse screenshot --path .context/ui-test-screenshots/.png
# If --path is not supported, take the screenshot and save manually:
browse screenshot
# The browse CLI will output the screenshot path — move/copy it:
cp /tmp/browse-screenshot-*.png .context/ui-test-screenshots/.png
在任何测试运行开始时设置截图目录:
mkdir -p .context/ui-test-screenshots
规则:
- 文件名 = 步骤 ID(例如:
double-submit.png,axe-audit.png,modal-focus-trap.png) - 存储在
.context/ui-test-screenshots/— 该目录已被添加到 Git 忽略列表中,开发人员和其他代理均可访问 - 对于并行运行,请包含会话名称:
(例如,- .png signup-double-submit.png) - 请在故障发生瞬间截屏——捕捉故障状态,而非恢复后的状态
- 对于视觉/布局类错误,还需截取基线(正常运行状态)的屏幕截图以供对比:
-baseline.png
验证方法(按严格程度排序)
- 确定性检查(最严格)——
browse eval返回可供检查的结构化数据。示例:axe-core 违规计数、document.title、表单字段值、控制台错误数组、元素计数。 - 快照元素匹配 — 辅助功能树中存在具有特定角色和文本的特定元素。通过引用进行检查:
@0-12 button "Save"。元素要么存在于树中,要么不存在。 - 前后对比 — 操作前的快照、执行操作、操作后的快照。验证树结构是否按预期发生变化(元素出现、消失、文本更改)。
- 截图 + 视觉判断(可靠性最低)——仅适用于辅助功能树无法捕获的纯视觉属性(颜色、间距、布局)。必须始终说明具体评估的内容。
前后对比模式
这是核心验证循环。请对每次交互都采用此方法:
# 1. BEFORE: capture state
browse snapshot
# Record: what elements exist, their text, their refs
# 2. ACT: perform the interaction
browse click @0-12
# 3. AFTER: capture new state
browse snapshot
# Compare: what changed? What appeared? What disappeared?
# 4. ASSERT: emit marker based on comparison
# If dialog appeared: STEP_PASS|modal-open|dialog "Confirm" appeared at @0-20
# If nothing changed:
browse screenshot --path .context/ui-test-screenshots/modal-open.png
# STEP_FAIL|modal-open|expected dialog to appear → snapshot unchanged|.context/ui-test-screenshots/modal-open.png
准备工作
which browse || npm install -g browse
避免权限疲劳
该技能会执行大量 browse 命令(快照、点击、评估)。为避免逐一批准,请将 browse 允许的命令中:
将这两个模式都添加到 .claude/settings.json (项目级别)或 ~/.claude/settings.json (用户级别):
{
"permissions": {
"allow": [
"Bash(browse:*)",
"Bash(BROWSE_SESSION=*)"
]
}
}
第一个模式涵盖普通 browse 命令。第二个模式涵盖并行会话(BROWSE_SESSION=signup browse open ...)。两者均需启用以避免审批提示。
模式选择
| 目标 | 模式 | 命令 | 身份验证 |
|---|---|---|---|
localhost / 127.0.0.1 |
本地 | browse open |
无需(默认使用干净且隔离的本地浏览器) |
| 已部署/预发布站点 | 远程 | browse open |
Browserbase 凭据;在支持的情况下使用上下文 |
规则:如果目标 URL 包含 localhost 或 127.0.0.1,则在第一个 browse open 上传递 --local。
本地模式(localhost 的默认设置)
browse open http://localhost:3000 --local
browse open ... --local 默认使用一个干净且隔离的本地浏览器,这最适合可重现的本地主机 QA 运行。
仅在需要时使用本地模式变体:
browse open— 自动发现现有的可调试本地 Chrome 浏览器。仅当测试明确需要现有的本地登录信息/Cookie/状态时才使用此选项。--auto-connect browse open— 连接到特定的 CDP 目标(显式连接本地浏览器)。--cdp
远程模式(通过 Cookie 同步部署的网站)
# Step 1: Sync cookies from local Chrome to Browserbase
node .claude/skills/cookie-sync/scripts/cookie-sync.mjs --domains your-app.com
# Output: Context ID: ctx_abc123
# Step 2: Open in remote mode with the synced context
SESSION_JSON="$(browse cloud sessions create --context-id ctx_abc123 --persist --keep-alive)"
SESSION_ID="$(echo "$SESSION_JSON" | jq -r .id)"
CONNECT_URL="$(echo "$SESSION_JSON" | jq -r .connectUrl)"
browse open https://staging.your-app.com --cdp "$CONNECT_URL"
browse snapshot
# ... run tests ...
browse stop
browse cloud sessions update "$SESSION_ID" --status REQUEST_RELEASE
Cookie-sync 标志: --domains, --context, --verified, --proxy "City,ST,US"
工作流 A:差异驱动测试
第一阶段:分析差异
git diff --name-only HEAD~1 # or: git diff --name-only / git diff --name-only main...HEAD
git diff HEAD~1 -- # read actual changes
对已更改文件进行分类:
| 文件模式 | 对界面的影响 | 测试内容 |
|---|---|---|
*.tsx, *.jsx, *.vue, *.svelte |
组件 | 渲染、交互、状态、边界情况 |
pages/**, app/**, src/routes/** |
路由/页面 | 导航、页面加载、内容、404处理 |
*.css, *.scss, *.module.css |
样式 | 视觉呈现(截图)、响应式设计 |
*form*, *input*, *field* |
表单 | 验证、提交、空输入、长输入、特殊字符 |
*modal*, *dialog*, *dropdown* |
交互 | 打开/关闭、退出、焦点陷阱、取消与确认 |
*nav*, *menu*, *header* |
导航 | 链接、活动状态、路由、键盘导航 |
| 仅限非UI文件 | 无 | 跳过 — 报告“无需进行 UI 测试” |
第二阶段:将文件映射到 URL
检测框架: cat package.json | grep -E '"(next|react|vue|nuxt|svelte|@sveltejs|angular|vite)"'
| 框架 | 默认端口 | 文件 → URL 模式 |
|---|---|---|
| Next.js 应用路由器 | 3000 | app/dashboard/page.tsx → /dashboard |
| Next.js 页面路由器 | 3000 | pages/about.tsx → /about |
| Vite | 5173 | 检查路由配置 |
| Nuxt | 3000 | pages/index.vue → / |
| SvelteKit | 5173 | src/routes/+page.svelte → / |
| Angular | 4200 | 检查路由模块 |
第三阶段:确保运行的是正确的代码
测试前,请确认开发服务器正在提供 diff 中的代码——而不是过时的分支。
如果要测试拉取请求或特定分支:
# Check what branch is currently checked out
git branch --show-current
# If it's not the PR branch, switch to it
git fetch origin && git checkout
# Install deps — the lockfile may differ between branches
yarn install # or npm install / pnpm install
如果开发服务器此前已在其他分支上运行,请在检出后重启它。
查找正在运行的开发服务器:
for port in 3000 3001 5173 4200 8080 8000 5000; do
s=$(curl -s -o /dev/null -w "%{http_code}" "http://localhost:$port" 2>/dev/null)
if [ "$s" != "000" ]; then echo "Dev server on port $port (HTTP $s)"; fi
done
若未找到:请告知用户启动其开发服务器。
验证是否确实能正常渲染:
之后 browse open + browse snapshot,请检查无障碍树中是否包含真实的页面内容(导航、标题、交互元素)——而不仅仅是错误覆盖层或空的 body 区域。 Next.js 开发服务器在显示全屏构建错误对话框时仍可能返回 HTTP 200 状态码。如果快照为空或被错误对话框占据,则说明服务器出现故障——请在测试前修复构建问题。
第 4 阶段:制定测试计划
针对每个发生变更的区域,同时规划正常流程测试和对抗性测试:
Test Plan (based on git diff)
=============================
Changed: src/components/SignupForm.tsx (added email validation)
1. [happy] Valid email submits successfully
URL: http://localhost:3000/signup
Steps: fill valid email → submit → verify success message appears
2. [adversarial] Invalid email shows error
Steps: fill "not-an-email" → submit → verify error message appears
3. [adversarial] Empty form submission
Steps: click submit without filling anything → verify error, no crash
4. [adversarial] XSS in email field
Steps: fill "" → submit → verify sanitized/rejected
5. [adversarial] Rapid double-submit
Steps: click submit twice quickly → verify no duplicate submission
6. [adversarial] Keyboard-only flow
Steps: Tab to email → type → Tab to submit → Enter → verify success
第 5 阶段:执行测试
browse stop 2>/dev/null
mkdir -p .context/ui-test-screenshots
# localhost/default QA → clean, reproducible local run
browse open http://localhost:3000 --local
对于每项测试,请遵循“测试前/测试后”的模式:
# Navigate
browse open http://localhost:3000/path --local
browse wait load
# BEFORE snapshot
browse snapshot
# Note the current state: elements, refs, text
# ACT
browse click @0-ref
# or: browse fill "selector" "value"
# or: browse type "text"
# or: browse press Enter
# AFTER snapshot
browse snapshot
# Compare against BEFORE: what changed?
# ASSERT with marker
# STEP_PASS|step-id|evidence OR STEP_FAIL|step-id|expected → actual
第 6 阶段:报告结果
## UI Test Results
### STEP_PASS|valid-email-submit|status "Thanks!" appeared at @0-42 after submit
- URL: http://localhost:3000/signup
- Before: form with email input @0-3, submit button @0-7
- Action: filled "[email protected]", clicked @0-7
- After: form replaced by status element with "Thanks! We'll be in touch."
### STEP_FAIL|double-submit|expected single submission → form submitted twice|.context/ui-test-screenshots/double-submit.png
- URL: http://localhost:3000/signup
- Before: form with submit button @0-7
- Action: clicked @0-7 twice rapidly
- After: two success toasts appeared, suggesting duplicate submission
- Screenshot: .context/ui-test-screenshots/double-submit.png
- Suggestion: disable submit button after first click, or debounce the handler
---
**Summary: 4/6 passed, 2 failed**
Failed: double-submit, xss-sanitization
Screenshots saved to `.context/ui-test-screenshots/` — open any failed step's screenshot to see the broken state.
完成后 browse stop 完成后。
第 7 阶段:生成 HTML 报告
生成文本报告后,请生成一份独立的 HTML 报告,以便评审人员能在浏览器中直接打开。该报告将截图内联嵌入(base64 编码),因此可作为单个文件使用——无需任何外部依赖。
原因:文本报告虽适用于代理对话,但评审人员(产品经理、设计师、其他工程师)更希望获得可直接打开、浏览和分享的可视化文档。内联截图能让故障一目了然。
如何生成
- 阅读 references/report-template.html 中的 HTML 模板
- 通过将模板中的占位符替换为实际测试数据来生成报告:
| 占位符 | 值 |
|---|---|
{{TITLE}} |
报告标题为 |
---
name: ui-test
description: Runs adversarial UI tests in a real browser using the browse CLI, analyzing git diffs to test only changed areas or exploring the full app to find bugs in functionality, accessibility, responsive layout, and UX.
license: MIT
---
# UI Test — Agentic UI Testing Skill
Test UI changes in a real browser. Your job is to **try to break things**, not confirm they work.
Three workflows:
- **Diff-driven** — analyze a git diff, test only what changed
- **Exploratory** — navigate the app, find bugs the developer didn't think about
- **Parallel** — fan out independent test groups across multiple Browserbase browsers
## How Testing Works
The main agent **coordinates** — it plans test strategy, delegates to sub-agents, and merges results. Sub-agents do the actual browser testing.
### Planning: multiple angles, then execute once
**You MUST complete all three planning rounds yourself and output them before launching any sub-agents.** Planning happens in your own response — it is NOT delegated to sub-agents. Do not skip ahead to execution.
**Round 1 — Functional:** What are the core user flows? What should work? Write out each test as: action → expected result.
**Round 2 — Adversarial:** Re-read Round 1. What did you miss? Think about: different user types/roles, error paths, empty states, race conditions, edge inputs (empty, huge, special chars, rapid clicks).
**Round 3 — Coverage gaps:** Re-read Rounds 1–2. What about: accessibility (axe-core, keyboard-only), mobile viewports, console errors, visual consistency with the rest of the app?
**Deduplicate:** Merge all three rounds into one numbered list of tests. Remove overlaps. Assign each test to a group (e.g. Group A, Group B).
**Then execute once** — launch one sub-agent per group. Each sub-agent receives its specific list of tests to run, nothing more. Sub-agents do not explore or plan — they execute assigned tests and report results.
Output the three rounds, the merged plan, and the group assignments in your response before calling any Agent tool.
### Principles for splitting work
- **Sub-agents run assigned tests, not open exploration.** The main agent hands each sub-agent a specific numbered list of tests. Sub-agents do not plan, explore, or decide what to test — they execute the list and stop.
- **The bottleneck is the slowest agent** — split work so no single agent has a disproportionate share. Many small agents > few large ones.
- **Size the effort to the change** — a single component fix doesn't need many agents or many steps. A full-page redesign does. Let the scope of the diff drive the plan.
- **No early stopping on failures** — find as many bugs as possible within the assigned tests.
### Giving sub-agents a step budget
**The main agent MUST include an explicit browse step limit in every sub-agent prompt.** Sub-agents do not self-limit — they will run until done unless told otherwise.
As a rough heuristic: ~25 steps for a few targeted checks, ~40 for a full page with functional + adversarial + a11y, ~75 for multiple pages or a broad category. **Adjust based on what the assigned tests actually require** — these are starting points, not rules.
As a rough heuristic: ~25 steps for a few targeted checks, ~40 for a full page with functional + adversarial + a11y, ~75 for multiple pages or a broad category. **Adjust based on what the assigned tests actually require** — these are starting points, not rules.
Every sub-agent prompt must include:
```
You have a budget of N browse steps (each `browse` command = 1 step). Count your steps as you go. When you reach N, stop immediately and report:
- STEP_PASS/STEP_FAIL for every test you completed
- STEP_SKIP|<test-id>|budget reached for every test you didn't get to
Do not retry or continue after hitting the budget.
Run only these tests: [numbered list from the merged plan]
Do not explore beyond the assigned tests.
Do NOT generate an HTML report or write any files. Return only step markers and your findings as text.
```
The main agent should NOT run `browse` commands itself (except to verify the dev server is up). All testing happens in sub-agents.
**When a sub-agent hits its budget, the main agent accepts the partial results as-is.** Do not re-run or retry the sub-agent. Include SKIPPED tests in the final report so the developer knows what wasn't covered.
### Reporting
**Every sub-agent reports back with:**
```
Tests: 8 | Passed: 5 | Failed: 2 | Skipped: 1 | Pages visited: 2
```
**The main agent merges into a final report with:**
```
Tests: 20 | Passed: 14 | Failed: 4 | Skipped: 2 | Agents: 3 | Pass rate: 70%
```
Do not report "steps used" — browse command counts are implementation plumbing, not a meaningful metric for reviewers.
## Testing Philosophy
**You are an adversarial tester.** Your goal is to find bugs, not prove correctness.
- **Try to break every feature you test.** Don't just check "does the button exist?" — click it twice rapidly, submit empty forms, paste 500 characters, press Escape mid-flow.
- **Test what the developer didn't think about.** Empty states, error recovery, keyboard-only navigation, mobile overflow.
- **Every assertion must be evidence-based.** Compare before/after snapshots. Check specific elements by ref. Never report PASS without concrete evidence from the accessibility tree or a deterministic check.
- **Report failures with enough detail to reproduce.** Include the exact action, what you expected, what you got, and a suggested fix.
## Assertion Protocol
Every test step MUST produce a structured assertion. Do not write freeform "this looks good."
### Step markers
For each test step, emit exactly one marker:
```
STEP_PASS|<step-id>|<evidence>
```
or
```
STEP_FAIL|<step-id>|<expected> → <actual>|<screenshot-path>
```
- `step-id`: short identifier like `homepage-cta`, `form-validation-error`, `modal-cancel`
- `evidence`: what you observed that proves the step passed (element ref, text content, URL, eval result)
- `expected → actual`: what you expected vs what you got
- `screenshot-path`: path to the saved screenshot (failures only — see Screenshot Capture below)
### Screenshot Capture for Failures
**Every STEP_FAIL MUST have an accompanying screenshot** so the developer can see what went wrong visually.
When a test step fails:
```bash
# 1. Take a screenshot immediately after observing the failure
browse screenshot --path .context/ui-test-screenshots/<step-id>.png
# If --path is not supported, take the screenshot and save manually:
browse screenshot
# The browse CLI will output the screenshot path — move/copy it:
cp /tmp/browse-screenshot-*.png .context/ui-test-screenshots/<step-id>.png
```
Setup the screenshot directory at the start of any test run:
```bash
mkdir -p .context/ui-test-screenshots
```
**Rules:**
- File name = step-id (e.g., `double-submit.png`, `axe-audit.png`, `modal-focus-trap.png`)
- Store in `.context/ui-test-screenshots/` — this directory is gitignored and accessible to the developer and other agents
- For parallel runs, include the session name: `<session>-<step-id>.png` (e.g., `signup-double-submit.png`)
- Take the screenshot at the moment of failure — capture the broken state, not after recovery
- For visual/layout bugs, also screenshot the baseline (working state) for comparison: `<step-id>-baseline.png`
### How to verify (in order of rigor)
1. **Deterministic check** (strongest) — `browse eval` returns structured data you can inspect. Examples: axe-core violation count, `document.title`, form field value, console error array, element count.
2. **Snapshot element match** — a specific element with a specific role and text exists in the accessibility tree. Check by ref: `@0-12 button "Save"`. An element either exists in the tree or it doesn't.
3. **Before/after comparison** — snapshot before action, act, snapshot after. Verify the tree changed in the expected way (element appeared, disappeared, text changed).
4. **Screenshot + visual judgment** (weakest) — only for visual-only properties (color, spacing, layout) that the accessibility tree cannot capture. Always accompany with what specifically you're evaluating.
### Before/after comparison pattern
This is the core verification loop. Use it for every interaction:
```bash
# 1. BEFORE: capture state
browse snapshot
# Record: what elements exist, their text, their refs
# 2. ACT: perform the interaction
browse click @0-12
# 3. AFTER: capture new state
browse snapshot
# Compare: what changed? What appeared? What disappeared?
# 4. ASSERT: emit marker based on comparison
# If dialog appeared: STEP_PASS|modal-open|dialog "Confirm" appeared at @0-20
# If nothing changed:
browse screenshot --path .context/ui-test-screenshots/modal-open.png
# STEP_FAIL|modal-open|expected dialog to appear → snapshot unchanged|.context/ui-test-screenshots/modal-open.png
```
## Setup
```bash
which browse || npm install -g browse
```
### Avoid permission fatigue
This skill runs many `browse` commands (snapshots, clicks, evals). To avoid approving each one, add `browse` to your allowed commands:
Add both patterns to `.claude/settings.json` (project-level) or `~/.claude/settings.json` (user-level):
```json
{
"permissions": {
"allow": [
"Bash(browse:*)",
"Bash(BROWSE_SESSION=*)"
]
}
}
```
The first pattern covers plain `browse` commands. The second covers parallel sessions (`BROWSE_SESSION=signup browse open ...`). Both are needed to avoid approval prompts.
## Mode Selection
| Target | Mode | Command | Auth |
|--------|------|---------|------|
| `localhost` / `127.0.0.1` | Local | `browse open <url> --local` | None needed (clean isolated local browser by default) |
| Deployed/staging site | Remote | `browse open <url> --remote` | Browserbase credentials; use contexts where supported |
**Rule: If the target URL contains `localhost` or `127.0.0.1`, pass `--local` on the first `browse open`.**
### Local Mode (default for localhost)
```bash
browse open http://localhost:3000 --local
```
`browse open ... --local` uses a clean isolated local browser by default, which is best for reproducible localhost QA runs.
Use local-mode variants only when needed:
- `browse open <url> --auto-connect` — auto-discover an existing debuggable local Chrome. Use this only when the test explicitly needs existing local login/cookies/state.
- `browse open <url> --cdp <port|url>` — attach to a specific CDP target (explicit local browser attach).
### Remote Mode (deployed sites via cookie-sync)
```bash
# Step 1: Sync cookies from local Chrome to Browserbase
node .claude/skills/cookie-sync/scripts/cookie-sync.mjs --domains your-app.com
# Output: Context ID: ctx_abc123
# Step 2: Open in remote mode with the synced context
SESSION_JSON="$(browse cloud sessions create --context-id ctx_abc123 --persist --keep-alive)"
SESSION_ID="$(echo "$SESSION_JSON" | jq -r .id)"
CONNECT_URL="$(echo "$SESSION_JSON" | jq -r .connectUrl)"
browse open https://staging.your-app.com --cdp "$CONNECT_URL"
browse snapshot
# ... run tests ...
browse stop
browse cloud sessions update "$SESSION_ID" --status REQUEST_RELEASE
```
Cookie-sync flags: `--domains`, `--context`, `--verified`, `--proxy "City,ST,US"`
## Workflow A: Diff-Driven Testing
### Phase 1: Analyze the diff
```bash
git diff --name-only HEAD~1 # or: git diff --name-only / git diff --name-only main...HEAD
git diff HEAD~1 -- <file> # read actual changes
```
Categorize changed files:
| File pattern | UI impact | What to test |
|-------------|-----------|--------------|
| `*.tsx`, `*.jsx`, `*.vue`, `*.svelte` | Component | Render, interaction, state, edge cases |
| `pages/**`, `app/**`, `src/routes/**` | Route/page | Navigation, page load, content, 404 handling |
| `*.css`, `*.scss`, `*.module.css` | Style | Visual appearance (screenshot), responsive |
| `*form*`, `*input*`, `*field*` | Form | Validation, submission, empty input, long input, special chars |
| `*modal*`, `*dialog*`, `*dropdown*` | Interactive | Open/close, escape, focus trap, cancel vs confirm |
| `*nav*`, `*menu*`, `*header*` | Navigation | Links, active states, routing, keyboard nav |
| Non-UI files only | None | Skip — report "no UI tests needed" |
### Phase 2: Map files to URLs
Detect framework: `cat package.json | grep -E '"(next|react|vue|nuxt|svelte|@sveltejs|angular|vite)"'`
| Framework | Default port | File → URL pattern |
|-----------|-------------|-----|
| Next.js App Router | 3000 | `app/dashboard/page.tsx` → `/dashboard` |
| Next.js Pages Router | 3000 | `pages/about.tsx` → `/about` |
| Vite | 5173 | Check router config |
| Nuxt | 3000 | `pages/index.vue` → `/` |
| SvelteKit | 5173 | `src/routes/+page.svelte` → `/` |
| Angular | 4200 | Check routing module |
### Phase 3: Ensure the right code is running
Before testing, verify the dev server is serving the code from the diff — not a stale branch.
**If testing a PR or specific branch:**
```bash
# Check what branch is currently checked out
git branch --show-current
# If it's not the PR branch, switch to it
git fetch origin <branch> && git checkout <branch>
# Install deps — the lockfile may differ between branches
yarn install # or npm install / pnpm install
```
If the dev server was already running on a different branch, restart it after checkout.
**Find a running dev server:**
```bash
for port in 3000 3001 5173 4200 8080 8000 5000; do
s=$(curl -s -o /dev/null -w "%{http_code}" "http://localhost:$port" 2>/dev/null)
if [ "$s" != "000" ]; then echo "Dev server on port $port (HTTP $s)"; fi
done
```
If nothing found: tell the user to start their dev server.
**Verify it actually renders:**
After `browse open` + `browse snapshot`, check that the accessibility tree contains real page content (navigation, headings, interactive elements) — not just an error overlay or empty body. Next.js dev servers can return HTTP 200 while showing a full-screen build error dialog. If the snapshot is empty or dominated by an error dialog, the server is broken — fix the build before testing.
### Phase 4: Generate test plan
For each changed area, plan **both happy path AND adversarial tests**:
```
Test Plan (based on git diff)
=============================
Changed: src/components/SignupForm.tsx (added email validation)
1. [happy] Valid email submits successfully
URL: http://localhost:3000/signup
Steps: fill valid email → submit → verify success message appears
2. [adversarial] Invalid email shows error
Steps: fill "not-an-email" → submit → verify error message appears
3. [adversarial] Empty form submission
Steps: click submit without filling anything → verify error, no crash
4. [adversarial] XSS in email field
Steps: fill "<script>alert(1)</script>" → submit → verify sanitized/rejected
5. [adversarial] Rapid double-submit
Steps: click submit twice quickly → verify no duplicate submission
6. [adversarial] Keyboard-only flow
Steps: Tab to email → type → Tab to submit → Enter → verify success
```
### Phase 5: Execute tests
```bash
browse stop 2>/dev/null
mkdir -p .context/ui-test-screenshots
# localhost/default QA → clean, reproducible local run
browse open http://localhost:3000 --local
```
For each test, follow the **before/after pattern**:
```bash
# Navigate
browse open http://localhost:3000/path --local
browse wait load
# BEFORE snapshot
browse snapshot
# Note the current state: elements, refs, text
# ACT
browse click @0-ref
# or: browse fill "selector" "value"
# or: browse type "text"
# or: browse press Enter
# AFTER snapshot
browse snapshot
# Compare against BEFORE: what changed?
# ASSERT with marker
# STEP_PASS|step-id|evidence OR STEP_FAIL|step-id|expected → actual
```
### Phase 6: Report results
```
## UI Test Results
### STEP_PASS|valid-email-submit|status "Thanks!" appeared at @0-42 after submit
- URL: http://localhost:3000/signup
- Before: form with email input @0-3, submit button @0-7
- Action: filled "[email protected]", clicked @0-7
- After: form replaced by status element with "Thanks! We'll be in touch."
### STEP_FAIL|double-submit|expected single submission → form submitted twice|.context/ui-test-screenshots/double-submit.png
- URL: http://localhost:3000/signup
- Before: form with submit button @0-7
- Action: clicked @0-7 twice rapidly
- After: two success toasts appeared, suggesting duplicate submission
- Screenshot: .context/ui-test-screenshots/double-submit.png
- Suggestion: disable submit button after first click, or debounce the handler
---
**Summary: 4/6 passed, 2 failed**
Failed: double-submit, xss-sanitization
Screenshots saved to `.context/ui-test-screenshots/` — open any failed step's screenshot to see the broken state.
```
Always `browse stop` when done.
### Phase 7: Generate HTML report
After producing the text report, generate a standalone HTML report that a reviewer can open in a browser. The report embeds screenshots inline (base64) so it works as a single file — no external dependencies.
**Why:** Text reports are good for the agent conversation, but reviewers (PMs, designers, other engineers) want a visual artifact they can open, scan, and share. Screenshots inline make failures immediately obvious.
#### How to generate
1. Read the HTML template at [references/report-template.html](references/report-template.html)
2. Build the report by replacing the template placeholders with actual test data:
| Placeholder | Value |
|-------------|-------|
| `{{TITLE}}` | Report title for `<title>` tag (e.g., "UI Test: PR #1234 — OAuth Settings") |
| `{{TITLE_HTML}}` | Report title for the visible `<h1>`. If a PR URL is available, wrap the PR reference in an `<a>` tag so it's clickable (e.g., `UI Test: <a href="https://github.com/org/repo/pull/1234">PR #1234</a> — OAuth Settings`). If no URL, use plain text same as `{{TITLE}}`. |
| `{{META}}` | One-line context: date, app URL, user, branch |
| `{{TOTAL_TESTS}}` | Total STEP_PASS + STEP_FAIL count |
| `{{AGENT_COUNT}}` | Number of sub-agents that ran |
| `{{PASS_COUNT}}` | Number of STEP_PASS |
| `{{FAIL_COUNT}}` | Number of STEP_FAIL |
| `{{PASS_RATE}}` | Integer percentage (e.g., "92") |
| `{{RATE_CLASS}}` | `good` (≥90%), `warn` (70–89%), `bad` (<70%) |
| `{{FAILURES_SECTION}}` | HTML for failed test cards (see below) |
| `{{PASSES_SECTION}}` | HTML for passed test cards (see below) |
3. For each test result, generate a `<details>` card. Failed tests should be **open by default** so reviewers see them immediately:
```html
<!-- Failed test card (open by default) -->
<div class="section">
<h2>Failures <span class="count">{{FAIL_COUNT}}</span></h2>
<details class="test-card fail" open>
<summary>
<span class="badge fail">FAIL</span>
<span class="step-id">step-id-here</span>
<span class="evidence">expected → actual</span>
</summary>
<div class="body">
<dl>
<dt>URL</dt><dd>http://localhost:3000/path</dd>
<dt>Action</dt><dd>What was done</dd>
<dt>Expected</dt><dd>What should have happened</dd>
<dt>Actual</dt><dd>What happened instead</dd>
</dl>
<div class="suggestion">Fix: description of suggested fix</div>
<div class="screenshot">
<img src="data:image/png;base64,..." alt="Screenshot of failure">
<div class="caption">step-id.png — captured at moment of failure</div>
</div>
</div>
</details>
</div>
<!-- Passed test card (collapsed by default) -->
<div class="section">
<h2>Passed <span class="count">{{PASS_COUNT}}</span></h2>
<details class="test-card pass">
<summary>
<span class="badge pass">PASS</span>
<span class="step-id">step-id-here</span>
<span class="evidence">evidence summary</span>
</summary>
<div class="body">
<dl>
<dt>URL</dt><dd>http://localhost:3000/path</dd>
<dt>Evidence</dt><dd>What was observed</dd>
</dl>
</div>
</details>
</div>
```
4. **Embed screenshots as base64** so the HTML is fully self-contained:
```bash
# Convert screenshot to base64 data URI
base64 -i .context/ui-test-screenshots/step-id.png | tr -d '\n'
# Use as: src="data:image/png;base64,<output>"
```
Read each screenshot file referenced in STEP_FAIL markers, base64-encode it, and embed it as an `<img src="data:image/png;base64,...">` in the corresponding test card. For STEP_PASS, only embed a screenshot if one was explicitly taken (e.g., baseline screenshots).
5. Write the final HTML to `.context/ui-test-report.html`:
```bash
# Write the generated HTML
cat > .context/ui-test-report.html << 'REPORT_EOF'
<!DOCTYPE html>
...generated report...
REPORT_EOF
# Open it for the reviewer
open .context/ui-test-report.html # macOS
# xdg-open .context/ui-test-report.html # Linux
```
6. Tell the user: `Report saved to .context/ui-test-report.html` and offer to open it.
**Rules:**
- Failures section comes before passes — reviewers care about what's broken first
- Failed cards are `open` by default; passed cards are collapsed
- Every STEP_FAIL card MUST have an embedded screenshot — if the screenshot file is missing, note it in the card
- Include the suggestion/fix in each failure card if one was provided
- The report must work offline — no CDN links, no external assets
- Keep the HTML under 5MB — if screenshots push it over, reduce image quality or skip baseline screenshots for passes
## Adversarial Test Patterns
Apply these to every interactive element you test. Read [references/adversarial-patterns.md](references/adversarial-patterns.md) for the full pattern library (forms, modals, navigation, error states, keyboard accessibility).
## Deterministic Checks
These produce structured data, not judgment calls. Use them as the strongest form of assertion.
| Check | What it catches | Assertion |
|-------|----------------|-----------|
| axe-core | WCAG violations | `violations.length === 0` |
| Console errors | Runtime exceptions, failed requests | empty error array |
| Broken images | Missing/failed image loads | no images with `naturalWidth === 0` |
| Form labels | Inputs without accessible labels | every input has `hasLabel: true` |
For the exact `browse eval` recipes, read [references/browser-recipes.md](references/browser-recipes.md).
## Workflow B: Exploratory Testing
No diff, no plan — just open the app and try to break it. Use this when the user says "test my app", "find bugs", or "QA this site."
### Approach
1. **Discover the app** — read `package.json` to detect the framework, then open the root URL and snapshot to see what's there
2. **Navigate everything** — click through nav links, visit every reachable page, note what exists
3. **Test what you find** — for each page, apply the adversarial patterns below (forms, modals, navigation, keyboard, error states)
4. **Run deterministic checks** — axe-core, console errors, broken images, form labels on every page
5. **Report findings** — use STEP_PASS/STEP_FAIL markers, include reproduction steps for failures
Don't try to be systematic about coverage. Just explore like a user would, but with the intent to break things. The agent is good at this — let it roam.
### Tips for exploratory runs
- Start with the homepage, then follow the navigation naturally
- Try the 404 page (`/does-not-exist`) — is it custom or default?
- Look for empty states (pages with no data)
- Test forms with garbage input before valid input
- Check mobile viewport (375px) on every page — does it overflow?
- If the app has auth, use cookie-sync first
## Workflow C: Parallel Testing
Run independent test groups concurrently using named `browse` sessions (`BROWSE_SESSION=<name>`). Each session gets its own browser. Works with both local and remote mode.
Use when testing multiple pages or categories and you want faster wall clock time.
Read [references/parallel-testing.md](references/parallel-testing.md) for the full workflow: session setup, agent fan-out, cookie-sync for auth, and result merging.
## Design Consistency
Check whether changed UI matches the rest of the app visually. Read [references/design-consistency.md](references/design-consistency.md) when doing visual or design checks.
## Test Categories
| Category | How | Assertion type |
|----------|-----|---------------|
| Accessibility | axe-core + keyboard nav | Deterministic (violation count) |
| Visual Quality | Screenshot + heuristic evaluation | Visual judgment (weakest — note specifics) |
| Responsive | Viewport sweep + screenshots | Visual + deterministic (overflow check) |
| Console Health | Console capture eval | Deterministic (error count) |
| UX Heuristics | Snapshot + Laws of UX + Nielsen's | Structured judgment (cite specific heuristic) |
| Error States | Navigate to empty/error states | Before/after comparison |
| Data Display | Snapshot on tables/dashboards | Element match (column count, formatting) |
| Design Consistency | Screenshot baseline + changed page comparison | Visual judgment (cite specific property) |
| Exploratory | Free navigation + adversarial testing | Before/after + judgment |
Reference guides (load on demand):
- **Adversarial patterns** — [references/adversarial-patterns.md](references/adversarial-patterns.md) — load when testing forms, modals, navigation, or keyboard a11y
- **Browser recipes** — [references/browser-recipes.md](references/browser-recipes.md) — load when running deterministic checks (axe-core, console, images, form labels)
- **Exploratory testing** — [references/exploratory-testing.md](references/exploratory-testing.md) — load for Workflow B (no diff, open exploration)
- **UX heuristics** — [references/ux-heuristics.md](references/ux-heuristics.md) — load when evaluating UX quality or citing specific heuristics
- **Design system** — [references/design-system.example.md](references/design-system.example.md) — template for users to customize
- **Design consistency** — [references/design-consistency.md](references/design-consistency.md) — load when doing visual consistency checks
- **Parallel testing** — [references/parallel-testing.md](references/parallel-testing.md) — load for Workflow C (concurrent sessions)
- **Report template** — [references/report-template.html](references/report-template.html) — HTML template for Phase 7 report generation
For worked examples with exact commands, read [EXAMPLES.md](EXAMPLES.md) if you need to see the assertion protocol in action.
## Best Practices
1. **Be adversarial** — try to break things, don't just confirm they work
2. **Every assertion needs evidence** — snapshot ref, eval result, or before/after diff
3. **Before/after for every interaction** — snapshot, act, snapshot, compare
4. **Screenshot every failure** — `browse screenshot` immediately on STEP_FAIL, save to `.context/ui-test-screenshots/<step-id>.png`
5. **Deterministic checks first** — axe-core, console errors, form labels before visual judgment
6. **For localhost, start with clean local mode** — pass `--local` on the first `browse open` for reproducible runs; use `--auto-connect` only when existing local state is required
7. **Always `browse stop` when done** — for parallel runs, stop every named session
8. **Report failures with reproduction steps** — action, expected, actual, screenshot path, suggestion
9. **Parallelize independent tests** — use Workflow C with named sessions when testing multiple pages or categories on a deployed site
## Troubleshooting
- **"No active page"**: `browse stop`, retry. For zombies: `pkill -f "browse.*daemon"`
- **Dev server not responding**: `curl http://localhost:<port>` — ask user to start it
- **`browse eval` with `await` fails**: Use `.then()` instead — `browse eval` doesn't support top-level await
- **Element ref not found**: `browse snapshot` again — refs change on page update
- **Blank snapshot**: `browse wait load` or `browse wait selector ".expected"` before snapshotting
- **SPA deep links 404**: Navigate to `/` first, then click through
- **Remote auth fails**: Re-run cookie-sync with `--context <id>`, try `--verified`
- **Parallel session conflicts**: Ensure every `browse` command uses `BROWSE_SESSION=<name>` — without it, commands go to the default session
- **Session not stopping**: `BROWSE_SESSION=<name> browse stop`. For zombies: `pkill -f "browse.*<name>.*daemon"`





首页
