Playwright vs AI browser automation is a false binary

For most teams, no. The real choice is between three stacks: (1) hand-written Playwright .spec.ts files, (2) a closed AI browser tool that uses vision and a proprietary YAML DSL, and (3) an LLM agent that drives Playwright itself through the Model Context Protocol. Option 3 gives you the determinism of option 1 (every action is a real Playwright primitive) and the ergonomics of option 2 (plain English scenarios, no selector plumbing). Assrt is option 3. The debate makes sense only if you ignore that option exists.

At startup, Assrt spawns @playwright/mcp as a child process over stdio. Code in /Users/matthewdi/assrt/src/core/browser.ts line 155 calls require.resolve('@playwright/mcp/package.json') to find the installed package, then runs it locally. The LLM agent (Claude Haiku 4.5 by default, or Gemini 3.1 Pro) sees a tool palette in agent.ts lines 16 through 195 with entries named navigate, snapshot, click, type_text, select_option, scroll, press_key, wait, screenshot, and evaluate. Those names are 1:1 with Playwright MCP's own tools. When the agent calls click, Assrt forwards the call to the Playwright MCP server, which runs the real Playwright click on a real Chromium page. No vision model guessing at pixels, no brittle xpath inventions.

Pure vision-based browser automation (agent sees pixels, agent clicks at coordinates) has two failure modes that break CI reliability. First, the agent can misread a lookalike element and click the wrong thing. Second, the trail it leaves is a sequence of (x, y) clicks that nobody can debug six months later. Driving real Playwright through MCP solves both. The agent calls snapshot to get the accessibility tree with [ref=eN] markers and uses those refs as selectors. When a step fails, the failure log shows 'clicked [ref=e14] labeled Submit, got no navigation' — the same kind of signal a human Playwright author would read. You get LLM flexibility on top of Playwright determinism.

No, and this is the intentional inversion. Playwright codegen produces TypeScript you then maintain by hand. When the UI shifts, those generated selectors break and you edit them. Assrt keeps the plan in /tmp/assrt/scenario.md as plain Markdown using #Case N: blocks. The agent interprets the Markdown against a fresh accessibility snapshot every run, so a button that moved from one container to another still works. What you commit to git is the intent (Markdown), not the fragile implementation (selectors). If you want actual Playwright code, you can write it by hand and use Assrt for the flows where the maintenance cost of that code is higher than the cost of a model call per run.

The file is a Markdown document with one or more #Case N: scene headers, each followed by numbered steps in English. The runtime writes it from scenario-files.ts line 42 (writeScenarioFile), then starts an fs.watch on it at line 97. Edit the file in any editor and a 1-second debounce fires, which PATCHes the cloud copy. The file on your disk is the source of truth; the cloud is a UUID-keyed cache. Contrast that with closed AI QA vendors that keep the test DSL inside a proprietary cloud editor. With Assrt, git add tests/ is enough. If the company shut down tomorrow, you still have the plan file, the JSON run report, the PNG screenshots, and the WebM video.

Out of the box, the agent knows 10 Playwright MCP primitives (navigate, snapshot, click, type_text, select_option, scroll, press_key, wait, screenshot, evaluate) plus 8 higher-level testing tools (assert, complete_scenario, create_temp_email, wait_for_verification_code, check_email_inbox, suggest_improvement, http_request, wait_for_stable). You can see the full TOOLS array in agent.ts lines 16 through 195. That covers the normal E2E surface. For niche Playwright features (tracing, video coverage beyond WebM, exotic network mocks), you would drop into a hand-written Playwright test and run it alongside Assrt. The two coexist; Assrt does not try to replace your existing Playwright suite, it replaces the part you keep rewriting because the UI keeps drifting.

Assrt is MIT licensed. The npm package is @assrt-ai/assrt. There is no per-seat fee and no cloud subscription. You pay for model inference on the scenarios you run: with claude-haiku-4-5-20251001 as the default, that is typically a few cents per scenario, not dollars. Closed AI QA platforms in the same space (Momentic, QA Wolf, Rainforest) run from $8,000 to $90,000 per year. The cost difference is not rhetorical; it is the reason indie teams show up in the Reddit threads asking whether Playwright is the only option. Option 3 (LLM on top of real Playwright) exists and it is free.

Yes. Assrt runs locally in Node and exposes three MCP tools (assrt_test, assrt_plan, assrt_diagnose) plus a CLI (assrt run --url ... --plan ...). In CI, spawn it as a subprocess in your existing Playwright job. The browser is a normal Chromium spawned by Playwright MCP, not a separate fleet, so you are not doubling browser infrastructure. A typical setup: hand-written Playwright for performance-critical flows and cross-browser regression (Firefox, WebKit); Assrt for signup, auth, checkout, and the broad surface where brittle selectors steal most of your maintenance time.

The agent is told to always call snapshot before interacting, and to use the ref IDs from the snapshot (e.g. ref='e5') as the preferred selector. When a click fails, the agent re-snapshots and retries with a fresh ref. After three failed attempts the scenario is marked failed with the snapshot attached as evidence, and the run continues to the next #Case. This is the strategy written into the SYSTEM_PROMPT in agent.ts. It is the same failure mode Playwright authors already handle with expect(...).toBeVisible() retries, just expressed in natural language plus a Playwright MCP call.

The cloud sync is opt-in and cache-shaped, not source-of-truth-shaped. When you run via the CLI without a scenario ID, plans live at /tmp/assrt/scenario.md and ~/.assrt/scenarios/<uuid>.json on your machine. The Firestore sync exists so you can share a shareable URL with a teammate; it is a UUID-keyed read-through cache. scenario-files.ts line 94 explicitly skips sync for scenarios prefixed 'local-'. You can unplug your network, run the CLI, and every artifact still lands on disk. The model call (Anthropic or Gemini) is the only external dependency, and that is a commodity you choose.