A Playwright AI test agent is a while loop. Here is the loop.

A function-calling chat session whose tools are a real Chromium browser. The model receives a plain-English scenario, calls snapshot to read the page as an accessibility tree, calls click or type_text against [ref=eN] ids returned from that tree, appends the new snapshot back to its message array, and continues until it emits stop_reason: end_turn. In the open Assrt agent, the entire loop fits in roughly fifty lines of agent.ts (lines 692-747). The browser part is a thin wrapper over the official @playwright/mcp server, so the agent inherits real Playwright primitives without re-implementing them.

Playwright codegen records your manual clicks and emits a TypeScript .spec.ts file with hardcoded locators. Maintenance is on you forever; when the DOM changes, the locator breaks, the test fails, you fix it. A Playwright AI test agent has no locators in the test file. The test file is Markdown like '#Case 1: a new user signs up' and the agent re-discovers the actual elements from a fresh accessibility snapshot on every step. There is no compiled selector to break. The Assrt source lives at /Users/matthewdi/assrt-mcp/src/core/agent.ts; the system prompt explicitly tells the model 'use snapshot refs (e.g. ref=e5) for reliable element targeting' (line 679 of the user-prompt builder).

Both, but not equally. The model's primary input is text. After every action, the agent calls snapshot, which returns the page as an accessibility tree: a serialized YAML-style tree where every interactable node has a stable [ref=eN] id. That tree is what the model reasons over. The screenshot tool exists too, but it is intentionally gated. Look at agent.ts:1024: the agent only emits a screenshot AFTER visual actions like navigate, click, type_text, scroll, and press_key. snapshot, wait, assert, evaluate, http_request and the rest skip the screenshot entirely. Screenshots are for the human watching the run video; the agent's decisions are made from the accessibility text. This is the single biggest mental shift versus thinking about it as 'computer use'.

The model itself. The Anthropic branch of the loop checks response.stop_reason after each turn (agent.ts:723). When the model emits stop_reason: end_turn with no tool_use blocks, the agent marks completed = true and exits the while. In practice the model does this after it calls the complete_scenario tool with its summary and pass/fail, because complete_scenario is a terminal tool: the agent records the result, and on the next turn the model has nothing to do. Failure modes: an API error tagged tool_use or invalid_request short-circuits the loop with scenarioPassed = false (agent.ts:735-742). 529, 429, 503, or 'overloaded' errors retry up to four attempts with backoff (agent.ts:728-734). The loop bound itself is set to Infinity at the top of the file because in practice the model converges on its own; the API or the user, not a counter, are the stop conditions that matter.

Fifteen, defined as Anthropic.Tool objects at the top of agent.ts (lines 16-196). Browser primitives: navigate, snapshot, click, type_text, select_option, scroll, press_key, wait, screenshot, evaluate. QA-specific: assert (records a boolean with description and evidence), complete_scenario (terminal). Email and external IO: create_temp_email (opens a mail.tm inbox), wait_for_verification_code, check_email_inbox, http_request (for API verification of webhooks and external state). Stability and feedback: wait_for_stable (injects a MutationObserver, polls every 500ms, returns when no DOM mutation for N seconds, default 2), suggest_improvement (the agent reports UX bugs it noticed during the run). Each tool is also mirrored as a Gemini function declaration at agent.ts:277-301 so the same agent works against gemini-3.1-pro-preview by swapping a flag.

Strict pairing. Every tool_use the model emits gets a tool_result appended to messages before the next turn. Look at agent.ts:1031-1036: regardless of whether the action succeeded, errored, or was unknown, exactly one ToolResultBlockParam goes back. On exception inside the switch, the catch at agent.ts:1012-1020 still produces a result string (with the page snapshot embedded for context) and feeds it back as a tool_result so the conversation stays valid. This matters because the Anthropic API rejects assistant turns that contain tool_use blocks without matching tool_result blocks on the next user turn; without that pairing the whole conversation 400s. It also helps the model: the error message includes the new accessibility tree, so the next turn can adapt instead of guessing.

Yes, all 56 lines of it. SYSTEM_PROMPT is a const at agent.ts:198-254. It tells the model six things: (1) it is an automated web testing agent, (2) call snapshot first to get [ref=eN] ids, (3) make assertions with the assert tool, (4) call complete_scenario when done, (5) on failure call snapshot to see what the page actually looks like and try a different ref, (6) for OTP inputs use the exact DataTransfer expression baked into lines 234-236 verbatim. There are no hidden instructions. This is one of the things that distinguishes an open agent from a closed SaaS: you can fork the prompt and ship a new version. The plan-generation prompt (the one that turns a URL into a #Case list) is in a separate file at /Users/matthewdi/assrt-mcp/src/mcp/server.ts:219-236, and is also auditable.

Codegen is a transcription. The user does the test once, the tool writes down what happened, the test ships, and the loop closes. There is no live decision-making after generation. An agent loops at runtime. Every tool_use is the model deciding what to do based on the latest tool_result. If a modal appeared unexpectedly, the agent sees it on the next snapshot and dismisses it. If a button moved, the new accessibility tree still has it, just at a different ref. This is why agent runs are slower than compiled spec.ts files but more durable across UI churn. It is also why the Assrt agent does not ship a 'lock the locator' optimization; the whole point is that locators are re-derived per step.