Automated UI testing flakes because the selector layer is wrong, not because the framework is

Automated UI testing is having software drive the same user interface a human would (clicks, typing, navigation, selection, scrolling, file uploads, assertions on visible state) without a human in the loop. It is one slice of test automation, distinct from unit testing (which calls functions directly) and from API testing (which hits endpoints without rendering a page). The point is end-to-end confidence: if the test passes, the app loaded, the JavaScript executed, the network requests resolved, the state updated, and the result rendered the way a real user would experience it.

Three reasons, in order of frequency. First, the locator layer is wrong: most test frameworks ask you to commit a CSS selector or XPath at write time, then look it up at run time against a DOM that has changed (a streamed-in Suspense boundary, a hydrated client component, a re-rendered list). Second, the wait primitive is wrong: waitForLoadState('networkidle') was designed for a 2014 page that finishes a single XHR, not a 2026 page that keeps the response open while React Server Components stream chunks. Third, the data isn't reset between runs: a test that signed up user42@example.com on Tuesday hits a 'user exists' branch on Wednesday. Frameworks themselves are not broken. The way most teams wire them up is.

Overlap, but not synonyms. End-to-end testing means exercising the whole system top to bottom: frontend, API, database, background jobs, external integrations. Automated UI testing is the slice of end-to-end testing that drives the UI. You can do end-to-end testing without touching the UI (call the API, inspect the database). You can do UI testing without going end-to-end (mock the API, drive the rendered frontend). In practice, the test pyramid groups them together because they share the same infrastructure (a headless browser, a runner, an assertion library). The distinction matters when you are deciding what to mock.

The honest landscape: Playwright (Microsoft, open source) and Cypress (open source with a closed cloud) are the two most-used modern choices. Selenium remains in legacy stacks and CI configs that have not been migrated. Puppeteer is in the same family as Playwright but narrower. On top of those engines, AI-layer tools like QA Wolf (managed service, around $7,500/month publicly), Momentic, Testim, Mabl, and Rainforest QA add generation and dashboards. Assrt sits in the AI layer too, but writes its output as a Markdown #Case plan plus a real Playwright runner that re-resolves locators per step, all open source at github.com/assrt-ai/assrt-mcp. None of these are the right answer for every team. The choice that survives a year is the one whose generated tests you can actually read.

It is a fine starting point and a bad ending point. Recording your clicks and replaying them works the first time and breaks the second time because the recorder captured a snapshot of the DOM that no longer exists. The pattern survives only if the runner re-resolves the recorded intent against the live page each step, instead of replaying recorded coordinates or recorded CSS paths. The mechanical operation looks the same to the person recording. The difference is whether the artifact is 'click element [ref=e5]' (one-shot) or 'click the button labelled New project' (re-resolved every run). Treat record-and-playback as a way to draft test cases, not as a way to maintain them.

Stop waiting for the network and start waiting for the DOM. Network-based waits (networkidle, idleConnections, no-pending-fetches) were appropriate when a page made one or two XHRs after load. A modern app keeps the response open while React Server Components stream, opens a websocket for live updates, and re-renders after every hydration tick. The signal that the page is actually settled is that no DOM mutations have arrived for a configurable quiet window. Assrt's runner injects a MutationObserver on document.body, polls window.__assrt_mutations every 500ms, and returns when the count has been quiet for stable_seconds (default 2s, cap 10s) or the timeout (default 30s, cap 60s) elapses. The full mechanism is at agent.ts:962-994 in the open source.

No, and the framing is wrong. Automated UI tests replace the part of QA that is repetitive and mechanical: regression coverage of flows you have already verified. The part of QA that is creative (figuring out which flow the customer actually walks, designing the test cases, deciding what 'this looks broken' even means) is still a human job. The realistic effect of good automated UI testing is that the same QA engineer covers a system three times bigger without the test count exploding into a maintenance liability. The unrealistic claim, that you can fire QA and let the machine sort it out, is the one that produces the test suites everyone hates.

Three levers. Parallelize at the worker level (Playwright workers, Cypress parallel containers, separate browser contexts per scenario); your suite is as slow as your slowest scenario, not the sum. Reuse auth state across scenarios so each test does not log in from scratch (Assrt's agent shares cookies and storage between #Case blocks for this reason, see agent.ts:692-747). And keep the wait primitives dynamic, not fixed: a 500ms sleep multiplied by 200 assertions across 60 scenarios is the difference between a five-minute CI and a thirty-minute one. None of these three are exotic. They are just the ones most suites skip until the build time forces it.

Decide on three axes. Output portability: do the generated tests live in your repo, or in a vendor dashboard? If you can't grep them, you can't audit them. Hosting cost: do you pay per scenario per month, or do you run on your own CI minutes? Managed services are often the right answer at low scale and a tax at high scale. Lock-in if you leave: if the vendor goes away tomorrow, do you have to rewrite the suite or can you keep running it? Assrt's bias is open source on all three axes (Markdown #Case files in your repo, run on your CI, no vendor to leave). Managed services like QA Wolf have a real dashboard and managed maintenance that an open tool does not match. The honest answer depends on which axis hurts more for your team.

The MCP server, agent loop, browser wrapper, and CLI live at github.com/assrt-ai/assrt-mcp. The snapshot/click/type pipeline is at src/core/agent.ts (tool declarations at lines 27-53, dispatch at lines 778-797) and src/core/browser.ts (browser_snapshot at line 589, browser_click at 600, browser_type at 610). The MutationObserver stability wait is at agent.ts:956-1009. The npm package is @m13v/assrt; npx @m13v/assrt discover https://your-app.com generates a #Case plan and runs it. Every file and line cited on this page is checkable on the main branch.