Web Agents That Complete Real Tasks: Beyond Browser Automation

Browser automation has been stuck at "click this button" for a decade. Selenium, Puppeteer, and Playwright are excellent at executing scripts — but they break the moment a page changes. AI web agents promise to fix this, but most can't complete real multi-step tasks.

What "real tasks" means

A real task is not "go to amazon.com and search for shoes." A real task is:

• "Order the cheapest 2-pack of AA batteries from Amazon and use my saved card"
• "Fill out this 5-page insurance form using my personal details from this PDF"
• "Find every competitor of Acme Corp on LinkedIn and export their names and titles"

These require: form filling, multi-page navigation, CAPTCHA handling, authentication, file upload, data extraction, and decision-making. Most AI agents can do 1-2 of these. Ours does all of them.

Architecture: vision + DOM + planning

Our WebAgent API at mazzagrp.com uses three signals simultaneously:

1. Visual understanding: A multimodal model sees the page screenshot and identifies interactive elements (buttons, inputs, links) even when the DOM is obfuscated.
2. DOM parsing: We parse the accessibility tree — not the raw HTML. This gives us clean element descriptions even on React/Vue/Angular sites.
3. Task planning: The agent breaks the task into sub-goals and tracks progress. If a step fails (page changed, element moved), it replans instead of crashing.

Example: automated form filling

curl -X POST https://api.mazzagrp.com/v1/web-agent \
  -H "Content-Type: application/json" \
  -d '{
    "task": "Go to example.com/insurance-form and fill out the form with data from this PDF",
    "context": {"pdf_url": "https://example.com/my-details.pdf"},
    "max_steps": 20
  }'

The agent returns a step-by-step log of what it did, screenshots of each page, and the final result. You can review the full audit trail.

Handling the hard parts

Three things make web automation fail in production:

1. CAPTCHAs. We don't solve CAPTCHAs (that's a legal grey area). Instead, we detect them and return a human-handoff signal. The API consumer can solve it and resume the task.

2. Authentication. We support session cookies. You authenticate once (or provide cookies), and the agent uses that session. No password storage.

3. Dynamic content. Single-page apps (React, Vue) load content dynamically. Our agent waits for network idle + visual stability before acting. No more "element not found" errors on SPA sites.

Pricing and limits

$0.50 per task (USDC on Base). Each task can have up to 30 steps. The API returns a full audit trail — every screenshot, every DOM state, every action taken. You only pay for completed tasks, not failed ones.