QA automation engineering: the defensive code, not the test

The actual job

Every career guide for this keyword says the same things. Write tests, plug into CI/CD, learn Python or Java, adopt AI-assisted self-healing. None of that is wrong. It is just the outer two percent of the work.

The inner ninety-eight percent is defensive code that handles a specific enumerated list of failure modes. The test code is the easy part. The interesting engineering lives in the surface between the test runner and the real world: DNS, TCP, DOM mutations, browser lifecycle, auth state, OTP widgets. If your automation cannot distinguish a wedged dev server from a slow one, you do not have QA automation. You have a flaky CI job.

What follows is five defensive patterns. Each one fixes a specific failure mode. Each one lives at a line in assrt-mcp/src/core/agent.ts you can open and read.

Pattern 1 · Preflight HEAD probe, 8 second abort

Before we spend 4 seconds booting Chromium and another second wiring the Playwright MCP stdio bridge, we spend up to 8 seconds checking that the target server is actually there. A HEAD request is cheap. If the server returns 405 or 501 (some servers refuse HEAD), we fall back to GET. Any HTTP response counts as reachable, even 404 or 500. The only failure modes are DNS, connection refused, or abort after 8 seconds.

The win here is not the probe itself. It is that a class of opaque late-stage errors ("MCP client not connected") is replaced with an early, typed, actionable one ("Target URL did not respond within 8000ms"). An engineer debugging a CI failure now has enough information in the first log line to fix it.

Pattern 2 · Promise.race navigate, 30 second cap

Playwright has its own page-level timeouts, but they rely on the driver being responsive. When the browser process itself is stuck, a high-level wrapper never gets its callback. The defensive pattern: race every navigate against a hand-rolled timer and log a structured event on failure. If the real nav never comes back, the timer wins in bounded time and surfaces a clean error.

// The naive version: assume the browser will come back.
await browser.navigate(url);
// If navigate hangs, the whole process hangs.
// Eventually the Playwright MCP stdio pipe gets killed and the error is
// "MCP client not connected" with no hint of which URL stalled.
await runScenarios();

Pattern 3 · MutationObserver stability, not sleep()

Every flaky test suite in the world has a await sleep(2000) somewhere. It is there because an engineer got tired of asking "is the page done yet" and picked a number that worked once. That number is now too short on a slow CI runner and too long on a fast laptop. The defensive replacement is a MutationObserver injected into the page that counts DOM mutations, polled every 500ms from the Node side, with a configurable quiet window (default 2 seconds, capped at 10).

Three details matter. The observer is attached to document.body with subtree: true, so every descendant mutation counts. The polling approach is deliberate, not a workaround: the observer lives in the page context while the agent runs in Node, and evaluate() is the cheap bridge. The cleanup block runs unconditionally, so even if the observer fires an exception we do not leak an observer into a tab that will get reused by the next scenario.

Pattern 4 · Synthetic ClipboardEvent for OTP widgets

Login flows that use a six-character OTP rendered as six single-character inputs defeat every naive automation approach. Setting each input's value programmatically does not trigger React state updates. Typing into each one with the keyboard focuses the next field mid-stroke and drops characters. The defensive pattern in Assrt is a single synthetic ClipboardEvent that the widget's own paste handler distributes across the six inputs.

The model is instructed to call evaluate() with this exact expression, substituting only the code digits. It is literally in the system prompt at agent.ts:234-236. This is how QA automation engineering handles the class of "widgets the web platform did not design for automation" without shipping a widget-specific shim.

Pattern 5 · Keep the browser alive across scenarios

The run loop iterates through all scenarios inside a try / finally. Most automation frameworks close the browser in the finally. Assrt explicitly does not. The inline comment in the code says so verbatim: Don't close the browser here — keep it alive so the user can take over and interact after the test finishes.

That single choice unlocks five properties that carry across every scenario in the run. Auth state, disposable email context, wizard progress, browser devtools history, and most importantly: the ability for a human to take over the live tab. When a scenario fails, the tab is still there. You attach with noVNC, click around, figure out the fix, write it into the plan, and rerun.

Pattern bonus · Snapshot on every tool failure

A sixth pattern worth calling out, because it is the one that replaces brittle retry logic with something that actually transfers to new codebases. Every tool dispatch is wrapped in a try / catch. On failure the catch block immediately calls browser.snapshot() for a fresh accessibility tree and attaches it to the error message returned to the LLM. The next turn of the conversation has current context, not stale plan text.

The numbers, in one table

Four constants control the five patterns. They are all small, bounded, and in source. Anyone auditing the QA runtime can read them in under a minute.