Webwright: Why AI Net Brokers Ought to Write Code, Not Click on

1. gained’t maintain nonetheless

If in case you have constructed an online agent lately, you recognize the failure sample. You give it a activity like “pull each itemizing from this listing right into a spreadsheet” and watch it inch ahead. It reads the web page. It predicts a click on. It waits for the brand new DOM, the web page construction the browser sees. It reads once more, predicts once more, waits once more.

Then, someplace round step 40, issues crumble. A modal pops up unexpectedly. The “subsequent web page” button strikes. The agent errors one aspect for one more. Any certainly one of these can derail the entire activity. The deeper drawback isn’t the unhealthy click on. It’s how the agent operates: have a look at the web page, resolve on one motion, see what modified, then resolve once more. It repeats this loop again and again, with no sturdy plan for the right way to full the duty from begin to end.

The sphere has tried just a few alternative ways to make this loop extra dependable. Some brokers, like OpenAI’s Operator and Anthropic’s Laptop Use, work from screenshots and work together with an internet site very similar to an individual would. Others, like WebVoyager, use the web page’s DOM to grasp what parts can be found and resolve which one to work together with.

Benchmarks comparable to Mind2Web and WebArena made these methods simpler to check by giving brokers a typical set of actions—click on, sort, scroll, choose. And open-source instruments like browser-use, Skyvern, Stagehand, and LaVague have packaged these concepts into APIs that engineers can extra simply construct into actual purposes.

These approaches make the loop extra dependable, however they don’t change the way it essentially works: the agent nonetheless takes one motion at a time, waits to see what occurs, then decides what to do subsequent. And when the duty is over, it hasn’t constructed something reusable—it has solely accomplished a sequence of clicks.

Webwright, a browser-agent framework from Microsoft Analysis and the College of Hong Kong, takes a unique method. Its tagline captures the concept: “A terminal is all you want for net brokers.”

As an alternative of asking the mannequin to determine the subsequent click on, Webwright has brokers writing and working code—utilizing bash and Playwright scripts to open browsers, examine pages, and perform the duty. The outcome isn’t only a lengthy sequence of browser actions. It’s a program engineers can examine, rerun, modify, and reuse.

This distinction issues most when the online is your knowledge supply: dashboards, product catalogs, search outcomes, inside instruments, JavaScript-heavy websites, and workflows you anticipate to run greater than as soon as. In these circumstances, the query isn’t simply whether or not an agent can end the duty. It’s whether or not it ought to maintain clicking by means of the browser or write a reusable program to do the work.

We’ll begin with the 4 predominant approaches to constructing net brokers and the constraints they nonetheless share. Then we’ll look inside Webwright: how its three core parts work, how a framework of roughly a thousand strains performs on benchmarks, and what the outcomes say about value and reliability. Lastly, we’ll put the method to work on three widespread scraping issues—paginated pages, JavaScript-rendered content material, and infinite-scroll feeds.

2. Why net brokers maintain breaking

The shift to “write code” issues as a result of it addresses the underlying drawback, not simply the signs. As we speak’s net brokers differ in how they perceive a web page—some have a look at screenshots, others learn the DOM—however most nonetheless work the identical means: take one browser motion, see what occurs, then resolve on the subsequent one.

That works for brief duties. However the longer the duty runs, the extra probabilities there are for one unhealthy click on, a modified web page, or a misinterpret aspect to throw every little thing off.

Household	What the mannequin sees	Why it helps	The place it breaks
Imaginative and prescient brokers	Screenshots	Works when the web page is just visually comprehensible	Format shifts, pixel ambiguity, costly screenshots
DOM / set-of-marks brokers	HTML, accessibility timber, numbered containers	Extra grounded than uncooked pixels	Large web page state, altering aspect IDs, exhausting grounding
Mounted action-API brokers	A menu like click on/sort/scroll/choose	Reproducible benchmark loop	Can not specific loops, retries, file output, or “do that for each row”
Browser frameworks	Packaged browser-control loops	Simpler to ship and observe	Typically nonetheless per-step, session-centered, and artifact-poor

Imaginative and prescient brokers are simple to grasp: they have a look at the web page very similar to an individual does and resolve the place to click on. That works nicely for a lot of browser duties, however scraping calls for extra consistency. A small structure shift can transfer a button simply sufficient for the agent to click on the incorrect place—or nothing in any respect.

There’s additionally a value to repeatedly taking a look at screenshots. Each new screenshot consumes tokens, and the agent has to hold sufficient context ahead to recollect what it already did. On an extended activity, that context turns into more durable and costlier to take care of.

DOM and accessibility-tree brokers keep away from a number of the issues that include screenshots. As an alternative of guessing the place a component is predicated on pixels, they’ll learn the construction of the web page and establish buttons, hyperlinks, types, and different parts instantly. WebVoyager, for instance, reported about 59% activity success throughout 50 real-world web sites, considerably higher than text-only baselines.

However this creates a unique drawback: an excessive amount of web page knowledge. A fancy web page can produce an accessibility tree bigger than 50KB. Because the agent strikes by means of a activity, previous web page state accumulates in its context though a lot of it’s not helpful. The references it makes use of to establish parts may change when a web page lazy-loads content material, rerenders, or navigates someplace new.

So higher entry to web page construction doesn’t essentially make lengthy browser duties dependable. On VisualWebArena, main vision-language brokers accomplished solely about 16% of duties, in contrast with roughly 89% for people.

Mounted motion APIs made net brokers simpler to construct and benchmark. Give the mannequin a small set of actions—click on, sort, scroll, choose—then let it observe the outcome and select once more. The draw back is that the agent can solely specific one small step at a time. It may’t naturally say, “maintain clicking subsequent till there are not any pages left,” “retry if this aspect doesn’t seem,” or “accumulate these 1,000 rows and save them to a CSV.” Every of these must be damaged into many particular person actions, with one other mannequin name in between.

Frameworks like browser-use, Skyvern, Stagehand, and LaVague make browser brokers a lot simpler to construct and combine. That’s helpful, however it doesn’t resolve one vital drawback for recurring knowledge work: when the duty is completed, there typically isn’t something reusable left behind. The agent could have collected the info as soon as, however subsequent week it has to work by means of the browser another time.

Analysis had already pointed towards one other method. Within the ICML 2024 paper Executable Code Actions Elicit Higher LLM Brokers, the researchers behind CodeAct changed predefined, JSON-style actions with executable Python. They reported as much as 20% increased success charges whereas utilizing roughly 30% fewer steps.

The reason being easy: code lets the mannequin do greater than take one motion at a time. It may use loops, retailer variables, retry failures, write recordsdata, and examine errors—all inside a program it may possibly run once more. Webwright brings that very same concept to browser automation.

3. Webwright’s distinctive level: make the workspace the state

Most browser brokers maintain their progress within the browser session. Shut the tab, and that state is gone.

Webwright flips this round. The browser is non permanent; the native workspace is what persists. The agent writes scripts, logs, screenshots, and output recordsdata as it really works, finally turning a profitable run right into a reusable device. This shift has just a few sensible advantages.

• Extra strong interactions — Playwright selectors and wait circumstances are extra dependable than pixel coordinates or non permanent aspect IDs.
• Higher composition — Loops and capabilities can deal with tons of of repeated actions in a single program
• Seen state — Progress is seen in recordsdata and logs.
• Reusable output — As soon as the duty works, the code might be reproduced as an alternative of ranging from scratch.

The place the mission’s 4 acknowledged benefits come from:

• Strong, reusable interactions — the agent acts by means of queries and wait-for-condition checks (web page.locator(...), wait_for_selector(...)) as an alternative of pixel coordinates or frozen aspect IDs, so a script survives structure shifts and re-renders.
• Environment friendly composition — loops, capabilities, and variables let a single flip say “do that for each row,” work a one-action-at-a-time agent has to spell out step-by-step.
• Workspace as state — progress lives in recordsdata, not in a fragile session or a context window bloated with stale web page dumps.
• Minimal by design — the entire system leans on 4 libraries (httpx, pydantic, playwright, typer) with no hidden framework beneath, and nonetheless posts state-of-the-art numbers.

3.1 Let’s do fast comparisons between Webwright and different choices in some situations.

3.1.1 Demo 1 · When clicking isn’t exact sufficient

The duty: Use Chase’s IRA calculator to check a Conventional vs. Roth IRA for somebody who’s 30, retires at 65, saves $300 a month, earns a 3% return, and has tax charges of 13% in the present day and 24% in retirement.

The problem is that the calculator makes use of six interactive JavaScript sliders.

End result:

• Webwright units the values instantly in code by updating the DOM inputs and triggering the required occasions. The values are actual, the chart renders accurately, and the working answer is saved as a reusable script.

• A imaginative and prescient agent has to control the sliders visually. It will get shut, however not shut sufficient: the $300 contribution lands at $294.

3.1.2 Demo 2 · When the identical activity comes again

The duty: Search Google Flights for a spherical journey from Seattle to San Francisco, together with the dates, and return the ranked outcomes.

End result:

• Webwright completes the search like different browser brokers may. The vital distinction is what occurs afterward: it retains the working code.

• When an identical flight search comes up later, the agent doesn’t have to determine each area, date picker, and click on once more. It may reuse the earlier script, change the inputs, and run it once more.

That’s what Microsoft means by “your searching historical past is code as an alternative of clicks.” A accomplished activity turns into a place to begin for the subsequent one, relatively than a browser session that disappears when it ends.

3.2 How Webwright differs from different browser-agent repos

The options are helpful, however they nonetheless put the browser on the middle of the workflow. Stagehand combines Playwright with natural-language instructions. agent-browser provides brokers a CLI for taking small browser actions. browser-use repeatedly reads the web page, chooses an motion, and executes it.

Webwright takes a unique method. As an alternative of selecting the subsequent browser motion, the mannequin can write a whole Python script. The browser is non permanent; the code, logs, and outputs keep within the native workspace. And when the duty is solved, the agent leaves behind a program that may be run once more.

That’s the core concept: clicking completes the duty as soon as; code completes it and retains the answer.

3.3 Inside Webwright

So what does it take to construct an agent like this? Surprisingly little.

Most net brokers put a harness—the software program connecting the mannequin to the browser—between the 2. That harness often provides the mannequin a set set of actions: click on this aspect, sort into this area, scroll the web page, learn the DOM, take a screenshot.

Webwright takes a unique method. As an alternative of giving the mannequin a menu of browser actions, it provides the mannequin a terminal and lets it resolve what instructions to run.

That makes the system surprisingly small. The core harness is roughly 1,000 strains of code throughout three parts. The total repository is nearer to 1,500 strains when you embrace the command-line interface and help for various mannequin suppliers. There isn’t a massive library of predefined browser actions. No customized DOM engine. The core system is simply three items:

• Runner (~150 strains) — Retains monitor of the duty and every little thing that has occurred to date: what the agent is making an attempt to do, the present state of its workspace, and the outcomes of earlier actions.
• Mannequin Endpoint (~550 strains) — Connects Webwright to the language mannequin. It supplies backends for OpenAI, Anthropic, and OpenRouter.
• Surroundings (~300 strains) — Offers the mannequin a terminal related to Playwright working Chromium. That is the place instructions truly execute, browser interactions occur, and recordsdata created in the course of the activity are saved.

The interplay between these items is a straightforward loop.

1. The Runner provides the mannequin the duty and the most recent context.
2. The mannequin decides what to do subsequent and returns a shell command.
3. The Surroundings runs that command and sends again what occurred—terminal output, logs, screenshots, or error messages.
4. Webwright provides these outcomes to the context and asks the mannequin what to do subsequent.

Briefly, the loop appears to be like like this:

perceive the present state → select a command → run it → see what occurred → repeat

The method continues till the mannequin believes the duty is full and a ultimate self-check agrees. Webwright doesn’t attempt to encode each potential browser interplay into the harness. It provides the mannequin a general-purpose interface—the terminal—and lets the mannequin work out the right way to use it.

The benchmarks help the design. On On-line-Mind2Web, GPT-5.4 with Webwright scores 86.7%, the very best amongst open-source AutoEval harnesses, whereas Claude Opus 4.7 reaches 84.7% and performs higher on the toughest duties.

The larger sign comes from Odysseys. GPT-5.4 utilizing coordinate-based browser management scores 33.5%. With Webwright, the identical mannequin reaches 60.1%—a 26.6-point achieve from altering the harness, not the mannequin.

Webwright’s mission web page lists 60.8%; I take advantage of the 60.1% reported in its GitHub comparability for consistency.

One other outcome helps the thesis: as soon as Webwright has constructed reusable instruments, the mannequin can get smaller. Microsoft reviews that even a 9B open mannequin (Qwen-3.5-9B) performs nicely on On-line-Mind2Web as soon as 5 or extra instruments can be found. The device doesn’t simply save work—it lowers the mannequin functionality wanted subsequent time.

There are tradeoffs. These are LLM-judged AutoEval scores, and the headline Mind2Web outcome makes use of 100 of 300 duties. It’s also not low cost: about $2.37 per activity with GPT-5.4 and $6.09 with Claude Opus 4.7. Webwright spends extra compute upfront to construct instruments which are extra strong and reusable.

4. Experiments and End result

I examined Webwright on three more and more tough websites, utilizing a separate Claude Sonnet agent for every run. That is to see how strong Webwright is the place scraping often breaks.

I used the Claude Code plugin relatively than the standalone benchmark harness. It retains the identical core setup—terminal + Playwright—however Claude Code runs the agent loop. That removes the necessity for a separate API key or per-task API invoice, although not the compute value.

The tradeoff is token utilization. In Microsoft’s instance, the Codex-hosted ability used ~3.3M tokens versus ~424K for the standalone harness—about 8× extra, largely from cached context. The price shifts into the host session relatively than disappearing.

Setup took one command:

playwright set up firefox   # the Claude Code ability drives headless Firefox, ~110MB one-time

4.1 🔧 Check 1 — static pagination · books.toscrape.com

books.toscrape.com is the best one amongst 3 circumstances: 50 numbered catalogue pages, 20 books every, served as plain HTML. The duty was to extract each guide title, worth, ranking, availability, and URL, then craft a reusable CLI with --pages and --out.

Earlier than writing the scraper, the agent inspected the location and examined its boundaries: web page 50 had no subsequent hyperlink, whereas web page 51 returned a 404. It then pulled selectors from an actual product card and constructed a easy pagination loop.

for n in vary(1, pages + 1):
    url = CATALOGUE_URL_TEMPLATE.format(n=n)
    await web page.goto(url, wait_until="domcontentloaded")
    playing cards = web page.locator("article.product_pod")
    rely = await playing cards.rely()
    log(n, f"loaded catalogue web page {n}/{pages} ({url}) -> {rely} guide playing cards discovered")

A click-based agent might deal with this web site, however code was cleaner. One refined problem was relative guide URLs, which change throughout pages. As an alternative of manually developing them, the scraper used the browser-resolved href values.

The outcome was not simply scraped knowledge, however a standalone CLI device that would run once more with out an agent.

$ python final_script.py --help
utilization: final_script.py [-h] [--pages PAGES] [--out OUT]
Scrape all books listed on books.toscrape.com's catalogue pages.
  --pages PAGES  Variety of catalogue pages to traverse ... Default: 50.
  --out OUT      Output CSV file path ... Default: books.csv.

$ python final_script.py --pages 2 --out pattern.csv
-> 40 rows written to .../pattern.csv

End result: 1,000 books throughout 50 pages, zero empty fields, in ~37 seconds.

The verification step additionally caught a bug: the primary model by accident cleared its proof log when working --help. The agent discovered the facet impact, fastened it, examined the repair, and reran efficiently. Even on this straightforward web site, the benefit was clear: a debuggable, reusable program as an alternative of a one-time click on hint.

4.2 🔧 Check 2 — JavaScript-rendered · quotes.toscrape.com/js

The second take a look at provides JavaScript. The quotes are not current within the uncooked HTML; they seem solely after the browser runs the web page’s JavaScript. The agent verified this primary: a direct HTTP request returned zero quote parts, whereas the rendered web page confirmed 10. A primary requests + BeautifulSoup scraper would silently return nothing.

Which means each web page have to be rendered earlier than extraction. There may be one other catch: web page 11 nonetheless returns HTTP 200, so standing codes can’t inform the scraper when to cease. As an alternative, this system checks the stay DOM for the subsequent hyperlink and stops when it disappears on web page 10.

whereas True:
    url = BASE_URL if n == 1 else PAGE_URL_TEMPLATE.format(n=n)
    await web page.goto(url, wait_until="domcontentloaded")
    await web page.wait_for_selector(".quote", timeout=10000)   # look forward to JS to inject the quotes
    ...                                                       # learn the ten rendered .quote playing cards
    has_next = await web page.locator("li.subsequent a").rely() > 0
    if not has_next or n >= pages:                           # cease on the DOM, not a standing code
        break
    n += 1

Once more, the output grew to become a reusable CLI with --pages and --out, in a position to run with out an agent.

The payoff — the crafted CLI. As on the books run, the working script grew to become a reusable scrape_quotes(pages, out) device with an argparse interface (--pages, default 10; --out, default quotes.csv) that re-runs standalone, no agent within the loop.

End result: 100 quotes throughout 10 pages in 8.9 seconds, with zero empty textual content or creator fields.

The run additionally uncovered a foul assumption in my temporary: I anticipated two pages to supply 40 rows, borrowing the 20-per-page rely from the books web site. This web site serves 10, so the proper outcome was 20. The agent returned the true knowledge and flagged the mismatch relatively than forcing the output to suit the spec.

4.3 🔧 Check 3 — infinite scroll · quotes.toscrape.com/scroll

The third take a look at removes pagination fully. Quotes load 10 at a time through AJAX because the web page scrolls, so there are not any web page URLs to iterate by means of. The scraper has to scroll, wait for brand new content material, measure the web page, and resolve when loading is completed.

The agent first confirmed the location’s habits: has_next turns into false at web page 10, and web page 11 returns no quotes. It then constructed a scroll-until-stable loop that stops when no new content material seems.

for i in vary(1, max_scrolls + 1):
    await web page.consider("window.scrollTo(0, doc.physique.scrollHeight)")
    await web page.wait_for_timeout(1000)
    rely = await web page.locator(".quote").rely()
    if rely == prev_count:          # no new quotes arrived
        stable_iters += 1
        if stable_iters >= 2:        # cease on stability, not a set rely
            break
    else:
        stable_iters = 0
    prev_count = rely

The DOM rely grew 10 -> 20 -> ... -> 100, held at 100 for 2 scrolls, and stopped at iteration 11. --max-scrolls was solely a security backstop. Forcing --max-scrolls 5 returned precisely 60 rows, proving the cease situation answered to the web page, not a hidden fixed.

End result: 100 quotes in 11 iterations, taking ~17 seconds.

Throughout the three exams, the worth of code turns into clearer because the websites get more durable. Static pagination is simple; JavaScript requires an actual browser; infinite scroll requires this system to motive about when to cease.

There may be additionally a helpful cross-check: Rungs 2 and three scrape the identical 100 quotes by means of two completely different interfaces—pagination and infinite scroll—and produce matching outcomes row for row. That provides us a completeness verify we are able to truly diff.

There are tradeoffs. Setup required a ~110 MB Firefox obtain and a few Home windows surroundings fixes. The Claude Code plugin additionally makes use of headless Firefox and its personal screenshot capabilities relatively than Webwright’s normal Chromium setup. For a one-off click on, this method is overkill. The payoff seems when duties contain repetition, dynamic content material, or outcomes you should confirm and reuse.

5. Conclusion

Total, these three exams present that browser-based code era is greater than a method to automate clicks. The ultimate result’s a reusable Playwright program that may be run once more with out the agent. Because the web sites grew to become extra advanced, the generated code additionally grew to become extra succesful. It moved from easy web page loops to rendering JavaScript and at last to reasoning about when an infinite scroll had completed.

The experiments additionally present the worth of verification. The agent discovered bugs in its personal code, questioned incorrect assumptions within the activity description, and confirmed that the extracted knowledge was full. The matching outcomes from the paginated and infinite-scroll variations of the quotes web site present further confidence that nothing was missed. That is tough to attain with a one-time browser recording alone.

There are prices. Operating Playwright requires a browser obtain and extra setup than easy HTTP scraping or click on recording. The generated packages are additionally longer and require some technical information to grasp. Nonetheless, these prices are outweighed when the duty must be repeated, maintained, or verified. Total, the exams counsel that browser-based code era is a sensible method for constructing dependable net scrapers that may adapt to completely different web site designs whereas producing code that’s reusable, clear, and straightforward to check.

Webwright’s contribution just isn’t an even bigger mannequin or a greater immediate. It’s a less complicated concept:

Give the mannequin a terminal, let it program the browser, and maintain the outcome as reusable code.

That modifications the web-agent loop. As an alternative of fragile clicks and fixed replanning, the agent can write, run, debug, and reuse a program.

The thought extends past browsers. When a mannequin can code and its surroundings can execute that code, it might be higher to write this system that performs the duty than predict each motion one step at a time.

The most effective net brokers don’t simply click on. They write the device—and depart it behind.

6. Sources

Webwright (main)

The panorama

Diagrams: author-created (matplotlib). Household animations: author-created. Screenshots and generated CLIs