Skip to content
Get packs

Guide · Agent skill

How to use the Browser & Desktop Automation agent skill

Updated · written with AI from the project's own documentation and checked against our catalogue data

The browser-automation skill gives an AI agent the instructions and command set it needs to open websites, read their interactive elements, fill forms, click buttons, take screenshots and scrape content, using the agent-browser CLI built on Playwright. It can also drive Windows desktop programs such as IDA Pro or x64dbg through the OpenReverse framework. This guide explains what the skill does, how to install it, how to phrase requests so it triggers, and what it needs and where to be careful.

Browser & Desktop Automationzhaoxuya520/reverse-skill · MIT

Quick start

  1. 1Download and unzip. Download the skill ZIP from this page and unzip it into ~/.claude/skills/ or a project's .claude/skills/ folder.
  2. 2Or upload in the Claude apps. In the Claude apps, go to Settings → Capabilities → Skills and upload the skill.
  3. 3Install browser tools. Install Node.js and npm, then run npm install -g agent-browser and npx playwright install.
  4. 4Open and snapshot. Ask the agent to open the target page; it should take an interactive snapshot to get element references.
  5. 5Act on elements. The agent clicks, fills or types using the references from the latest snapshot, waiting for the page to settle.
  6. 6Close the browser. The agent runs agent-browser close so no browser process is left running.

What the browser-automation skill makes the agent do

An agent skill is a folder of instructions (a SKILL.md file plus optional reference files) that an AI agent loads when a task matches the skill's description. This one covers two areas: browser automation through Playwright and agent-browser, and Windows desktop automation through OpenReverse.

In the browser path, the agent opens a page, takes a snapshot of its interactive elements, and then acts on those elements by reference. In the desktop path, it works with Windows applications either through standard UI controls (UIA mode) or by visually identifying buttons on screen (CUA mode), and can optionally observe the network traffic those apps generate.

  • Open pages and operate elements: click, fill forms, submit
  • Scrape page content or take screenshots
  • Automate login flows and batch form submissions
  • Operate Windows GUI tools such as IDA Pro or x64dbg
  • Observe desktop app network traffic with the built-in mitmproxy path

Install the skill (ZIP download or Claude apps)

Download the ZIP of the browser-automation skill from this page and unzip it. Place the resulting folder in one of these locations: ~/.claude/skills/ to make it available for your user account, or a project's .claude/skills/ folder to make it available only inside that project.

If you use the Claude apps instead, upload the skill under Settings → Capabilities → Skills.

  • Personal install: unzip into ~/.claude/skills/
  • Project install: unzip into the project's .claude/skills/ folder
  • Claude apps: Settings → Capabilities → Skills, then upload

Set up the browser tools

The browser path needs Node.js and npm. The skill's bootstrap can install Playwright and the agent-browser CLI when they are missing, and the source lists these install methods. Node.js is listed as installable through winget.

If you install by hand, the source shows these commands for the CLI and Playwright browser binaries.

  • Playwright: installed via npm and npx playwright install
  • agent-browser CLI: npm install -g agent-browser
npm install -g agent-browser npx playwright install

Run the core browser workflow

The core loop is: open the page, take an interactive snapshot to get element references such as @e1 and @e2, act on those references, then close the browser. The source says to take a snapshot before acting and not to guess element references.

After submitting a form, wait for the page to settle with networkidle. Always close the browser when finished, because skipping close leaves the process running.

  • Use snapshot -i (interactive elements only) as the default
  • Take a fresh snapshot before acting rather than guessing references
  • Run agent-browser close when done to avoid process leaks
agent-browser open <url> agent-browser snapshot -i agent-browser click @e1 agent-browser fill @e2 "text" agent-browser close

Use the common browser commands

Beyond the core loop, the skill documents commands for navigation, reading page information, keyboard input, scrolling and waiting. Use type instead of fill when an input box has JavaScript listeners that need per-character input, as the cheatsheet notes.

Use get text, get title and get url to read page data back, and wait with an element reference, a fixed millisecond delay, or networkidle depending on what you are waiting for.

  • Interaction: click, fill, type, press Enter/Tab/Escape, scroll up or down
  • Reading: get text @e1, get title, get url
  • Waiting: wait @e1, wait 2000, wait --load networkidle
agent-browser type @e2 "password123" agent-browser press Enter agent-browser scroll down 500 agent-browser get title agent-browser wait --load networkidle

Use OpenReverse for Windows desktop apps

For desktop work, the skill relies on OpenReverse, a framework for AI agent desktop interaction and evidence capture. Choose UIA mode when the target app uses standard Windows controls like buttons, text boxes and lists. Choose CUA mode when the UI is complex or non-standard, such as IDA's disassembly view.

OpenReverse is not installed by the bootstrap. You clone the repository from GitHub, run npm install, then connect it to your agent host. The optional CUA runtime and mitmproxy are added only if you need visual mode or packet capture.

  • UIA: standard Windows controls, via the UI Automation API
  • CUA: visual recognition plus mouse and keyboard, for custom-rendered UIs
  • Network: Proxy Lane (recommended if the app can use a proxy) or Local Lane
git clone https://github.com/zhexulong/openreverse.git cd openreverse npm install npm run init:agents -- --target=all /path/to/project npm run install:cua-runtime npm run doctor:cua-runtime npm run install:mitmproxy npm run doctor:network

Trigger the skill with natural requests

The skill's description lists trigger keywords, including 浏览器自动化 (browser automation), 打开网页 (open a web page), 填表 (fill a form), 爬取 (scrape), 截图 (screenshot), Playwright, agent-browser, OpenReverse, UIA and CUA. Ask for the outcome in plain language and mention the target, such as a site or a named Windows app. Requests that mention these terms are the most likely to match the skill.

Example requests that fit the source's scope:

  • Open the staging login page, fill in the test account, submit, and screenshot the dashboard.
  • Scrape the product titles and prices from this listing page and report them back.
  • Use OpenReverse in UIA mode to open x64dbg, load this program and set a breakpoint.
  • Open IDA Pro in CUA mode, load this binary and export the function list.

Safety notes: scripts, network and permission

This skill runs scripts and uses the network. It automates logins, captchas and form submissions, so only point it at accounts and sites you own or have permission to test. Automating sites you are not authorised to test may breach terms of service or law.

The OpenReverse desktop path is labelled experimental and has heavy dependencies, so it is not suited to unattended production use. Read the commands before running them, and check the bootstrap output against the source.

  • Only use it on systems and accounts you own or are authorised to test
  • Expect it to run scripts and make network requests
  • Treat the OpenReverse desktop mode as experimental

Tips for reliable results

Most failures come from acting on stale or guessed element references. Re-run snapshot after the page changes, and use wait --load networkidle after any submit. For evidence, ask the agent to save command output, screenshots or a report, since the skill's own checklist expects reproducible evidence.

If a site relies on JavaScript key handling, switch from fill to type. For simple static pages where you only need the text, a plain fetch is usually enough and avoids this setup.

  • Snapshot again after every page change
  • Use type for JavaScript-driven inputs
  • Ask for screenshots or reports as evidence
  • Close the browser at the end of every task

What people use it for

  • Testing a web login or form flow and reporting which steps failed
  • Scraping listing or product content from a page you are authorised to read
  • Capturing page screenshots as evidence for a report
  • Driving x64dbg or IDA Pro through OpenReverse for repeatable GUI steps
  • Observing network traffic from a desktop app through the proxy or local lane

Frequently asked questions

How do I install the browser-automation skill?

Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use, or into a project's .claude/skills/ folder. In the Claude apps you can instead upload it under Settings → Capabilities → Skills.

What does the browser-automation skill require?

The browser path needs Node.js and npm, the agent-browser CLI and Playwright. OpenReverse desktop mode needs Windows and a cloned OpenReverse repository, and the optional CUA runtime and mitmproxy are needed only for visual mode and network capture.

Is the browser-automation skill free?

The skill is published under the MIT licence, as stated on this page. Check the repository licence file for the full terms before reuse.

Can I use this skill without writing code?

Yes, you describe the outcome in plain language and the agent runs the commands. You still need Node.js and npm installed so the browser tools can run.

Why does the agent say it cannot find an element?

Element references like @e1 change when the page changes. Ask the agent to take a fresh snapshot -i and act on the new references rather than old ones.

Is there an alternative to this skill for just reading page text?

Yes. If you only need static text from a page, a simple fetch is enough. This skill adds setup that is unnecessary for that task.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Use this skill if you are a developer, analyst or reverse engineer who needs an agent to drive a web page or a Windows GUI tool and return evidence such as screenshots. Skip it if you only need static text from a page, or if you want unattended production automation, since the OpenReverse desktop path is experimental with heavy dependencies.

Download the skill

Find exactly what
you're looking for