Automated QA automation: the four babysitting loops Assrt removes in code you can read

The phrase is redundant on the surface, which is why most top results hand-wave it. Taken seriously, it means automating the automation: the parts of a browser-automation run that a human still has to babysit. Those parts are not the test steps themselves (any framework can click a button). They are the four failure modes that force a human back in the loop: the test sleeps too long and gets killed, a retry picks a stale selector, an OTP arrives in a real inbox, or the accessibility tree is too large for the model. Assrt removes each one in code you can read: wait_for_stable at agent.ts lines 906-959, auto fresh-snapshot at agent.ts lines 962-970, the three email tools between agent.ts lines 115 and 130, and SNAPSHOT_MAX_CHARS = 120_000 at browser.ts line 500.

A fixed timeout is a guess at how long your content takes to load. The guess is wrong on every page that has a streaming response, a React Query refetch, or a progressively-rendered list. wait_for_stable injects `window.__assrt_observer = new MutationObserver(...)` at agent.ts line 915, sets it to watch the entire body with childList, subtree, and characterData turned on, and then polls a counter at window.__assrt_mutations every 500ms. When the counter stops incrementing for the configured quiet period (default 2 seconds, capped at 10), the function returns. This adapts automatically to both a 200ms static page and a 12-second streaming answer. The default timeout is 30s, capped at 60s, so the run never hangs indefinitely.

The catch block at agent.ts lines 962-970 runs. It takes a fresh accessibility snapshot of the current page, slices the first 2000 characters, and appends them to the error message that goes back to the LLM: 'Error: <original message>. The action X failed. Current page accessibility tree: <tree>. Please call snapshot and try a different approach.' The model now sees the up-to-date DOM structure without you writing a single line of retry logic. This is the single mechanism that turns most transient selector failures into a one-step recovery and the reason the run continues unattended through modals, navigation, and late-loading elements.

Accessibility trees can grow into the millions of characters on apps with long virtualized lists or dense marketing pages. browser.ts line 500 defines SNAPSHOT_MAX_CHARS = 120_000, which is roughly 30,000 tokens with Claude's tokenizer. When a snapshot exceeds that, resolveAndTruncate at browser.ts line 506 slices to the nearest newline and appends the literal footer `[Snapshot truncated: showing 120k of <original>k chars. Use element refs visible above to interact.]`. The LLM still sees the first section of the tree with valid element refs, so it can scroll and re-snapshot the region it needs instead of dying on a context-window overflow. Without this ceiling, the run falls apart on any real e-commerce listing page.

browser.ts line 358 declares `private static readonly TOOL_TIMEOUT_MS = 120_000;` and passes it as the per-call timeout when the agent invokes any underlying Playwright MCP tool (line 381). The Model Context Protocol SDK defaults to 60s. Real navigations on enterprise apps, cold starts, and heavy dashboards routinely exceed that. The 120s ceiling is a survival margin. If a tool still times out, the agent doesn't hard crash: it marks the MCP client as dead (line 392), clears the transport, and the next call reconnects. This is why a network blip on step 34 of 60 does not end your run.

No, and the Assrt design argues that generating code is a side-effect, not the goal. A #Case block in scenario.md is English: 'Navigate to /settings, click Danger Zone, confirm the dialog, assert the row is gone.' The agent translates that into a chain of real Playwright MCP calls (the 18 tools registered at agent.ts lines 16-196). If you need the code out, you can log the chain. If you only need the run to pass, you never look at it. That is the actual 2026 change: the code becomes a byproduct of a reliable run, not a deliverable you maintain.

Same as any other test: pass ANTHROPIC_API_KEY or GEMINI_API_KEY as an environment variable. The default model is claude-haiku-4-5-20251001 (agent.ts line 9) which is fast and cheap enough to run on every commit. Playwright MCP launches Chromium headless by default. The run directory at /tmp/assrt/<runId>/ contains the video, events.json, and every tool-call snapshot, so a failing GitHub Actions job uploads one artifact and anyone on the team can replay the run locally with player.html.

Explicitly. Agent.ts lines 940-944 run a second evaluate call: `if (window.__assrt_observer) { window.__assrt_observer.disconnect(); } delete window.__assrt_mutations; delete window.__assrt_observer;`. This matters on long runs because the observer would otherwise accumulate listeners across every subsequent page. The explicit disconnect and delete keep the window clean for the rest of the scenario; a new wait_for_stable call in the next step gets a fresh observer on the fresh page.

Stability is a property of the run, not a property of the suite. The same test is stable on Monday and flaky on Thursday when a deployment changes an aria-label. Auto-snapshot on failure is the difference between a flake that ends your CI job and a flake that costs one extra LLM turn. In a 40-step scenario, a single transient failure costs about 2-3 seconds of agent time (one snapshot, one re-read, one fresh click) and the run continues green. Without it, you are paging an engineer to read a trace viewer. This is the core of what 'automated QA automation' should mean in 2026.

Yes. Both parameters are optional on every call. timeout_seconds defaults to 30 and is capped at 60 by Math.min at agent.ts line 907. stable_seconds defaults to 2 and is capped at 10 at line 908. For a streaming chat where tokens arrive over 8-10 seconds, raise the quiet period to 4 or 5 so the observer does not return between bursts. For a static page that should settle in under a second, lower it to 1. The cap prevents anyone from setting a 60-second quiet period that would stretch one run into forever.

/tmp/assrt/<runId>/ contains events.json (every tool call with arguments and timing), execution.log (the raw step stream), results.json (pass or fail with evidence), scenario.md (a snapshot of the plan you ran), screenshots/ (one PNG per visible action), and video/ (recording.webm plus player.html at 5x default playback). For an unattended run, this folder is the entire debugger. Tar it, upload it as a GitHub Actions artifact, and a failing build has everything a human needs to diagnose the failure from a different continent without reproducing the environment.

You can absolutely build a MutationObserver quiet-period and an auto-snapshot helper into your own Playwright suite. The question is whether you want to own and maintain the helper forever. Every one of these mechanisms is about 30-60 lines of code; together they take an engineer a sprint to build, then ongoing time to keep working as Playwright evolves. Assrt's value is that it is open-source code you can read, not a vendor service you rent, so the choice is between writing it yourself and copying someone else's working version with the commit history intact. Both beat paying a closed-source vendor $7,500 a month for the same primitives.