QA automation engineer in 2026: a Markdown plan and a JSON report

No, it is changing shape. The slice that goes away is the locator-by-locator authoring layer: the page object models, the brittle CSS selectors, the wrappers around wrappers around fixtures. The slice that grows is the part that was always the actual job: deciding what to verify, modelling user identity and state, owning the flake budget, gating CI, and writing assertions that catch real product regressions. When the runtime is an LLM agent reading a live accessibility tree, the engineer authors plain-English #Case blocks (see /Users/matthewdi/assrt-mcp/src/core/agent.ts line 620 for the parser) and reviews structured JSON reports (see writeResultsFile at /Users/matthewdi/assrt-mcp/src/core/scenario-files.ts lines 77-84). The skill that matters is good test design, which has not changed.

One Markdown file per concern. The reference path is /tmp/assrt/scenario.md (defined at /Users/matthewdi/assrt-mcp/src/core/scenario-files.ts line 17). Each block in that file starts with a #Case heading and contains 3 to 8 bullet points in plain English. The parser at /Users/matthewdi/assrt-mcp/src/core/agent.ts line 620 splits on the regex /(?:#?\s*(?:Scenario|Test|Case))\s*\d*[:.]\s*/gi and runs each block as an independent scenario, sharing browser state. There is no separate fixture file, no page object, no locator file, no test runner config. The plan is the test, and a Markdown diff in a pull request is exactly what review looks like.

Eighteen tool schemas, all defined as Anthropic tool input_schema objects between lines 16 and 196 of agent.ts. The list: navigate, snapshot, click, type_text, select_option, scroll, press_key, wait, screenshot, evaluate, create_temp_email, wait_for_verification_code, check_email_inbox, assert, complete_scenario, suggest_improvement, http_request, and wait_for_stable. That is the entire authoring surface. Anything an automation engineer would need a Cypress plugin or a Selenium addon for is one of those eighteen calls; the agent picks which to invoke based on the English in your plan.

With Selenium or Cypress, the engineer's day is roughly 60% locator wrangling, 20% fixture and CI plumbing, and 20% actual test design. With raw Playwright (or Playwright MCP) plus an LLM, the locator share drops to near zero because the agent reads a fresh accessibility tree before each action (see the system prompt at agent.ts lines 206-218 for the snapshot-act loop). The remaining time goes to test design, which is where the engineering judgment actually lives. The reference loop wraps Playwright MCP and adds the test execution layer (#Case parser, scenario continuity, JSON report shape, video recording per run). The README at /Users/matthewdi/assrt-mcp/README.md frames it as adding a QA engineer on top of the Playwright tools.

Yes, but not at the locator level. You need to know how a real browser actually behaves: when an event fires, when a form submits, when a navigation happens, what an accessibility tree looks like. Those mental models still drive how you write a #Case. What you do not need is muscle memory for page.locator() chains or expect().toHaveText() variants. The agent emits the equivalent calls under the hood through the Playwright MCP server. browser.ts at line 296 of the reference implementation shows the actual Playwright MCP launch args, including --viewport-size 1600x900 and --output-mode file. Knowing what those flags mean is part of the modern automation engineer's literacy; writing them out by hand on every project is not.

One bash line. The CLI emits a JSON TestReport (typed at /Users/matthewdi/assrt-mcp/src/core/types.ts lines 28-35) which wraps a ScenarioResult[] (lines 19-26). Each scenario carries name, passed, assertions[], and steps[]. In CI you run assrt run --url ... --plan-file plans/regression.md --json, pipe through jq, and exit non-zero on any .scenarios[].passed === false. There is no proprietary report format to parse, no SaaS dashboard to log into, no quota on how often you can run. The same JSON file lives at /tmp/assrt/results/latest.json (see scenario-files.ts line 20) for local debugging and at /tmp/assrt/results/<runId>.json for historical comparison.

It changes the failure mode. Locator flake (the test failed because the button got renamed from 'Submit' to 'Save changes') goes away, because the agent reads the accessibility tree on every step and finds the new label. Timing flake (the test failed because the API response had not arrived) is partly addressed by wait_for_stable at agent.ts line 187, which waits until the DOM stops mutating for a configurable stable_seconds window. What does not go away is intent flake: a #Case that says 'verify dashboard appears' will pass under multiple legitimate dashboard renderings, and that is the engineer's design problem. The fix is the same as it has always been: tighter assertions and explicit pass criteria, which the run interface accepts as a passCriteria string at server.ts line 343.

Five places, roughly in order of leverage. First, scenario design across the user model: which user, with which permissions, doing which sequence. Second, the assertion layer: what counts as 'passed', written as explicit pass criteria. Third, the regression boundary: which #Case files run on every commit, which on every release, which on every deploy. Fourth, the test data layer: disposable email is built in via create_temp_email at agent.ts line 115, but synthetic fixtures (orgs, projects, billing states) are still your problem to seed. Fifth, the report-review habit: a passing build with passing assertions but suspicious step traces (twenty retries on the same button, a screenshot that shows an unexpected modal) is something only a human catches.

The reference runner at github.com/m13v/assrt-mcp is MIT licensed. The Playwright MCP package it spawns (@playwright/mcp) is Microsoft's, also open source. The only closed thing in the loop is whichever LLM you point at it; the default is Claude Haiku 4.5 (DEFAULT_ANTHROPIC_MODEL at agent.ts line 9), but the Gemini path is also wired up at lines 354-367. There is an optional hosted runner at app.assrt.ai for sharing scenarios in a browser, but the local CLI produces identical reports without it. No proprietary YAML, no closed-source rule engine, no per-seat license. A senior automation engineer's evaluation budget for this stack is: clone the repo, read agent.ts top to bottom, decide.

The engineer writes English. The agent translates to tool calls. Inside a #Case, every step is roughly: agent calls snapshot to get the accessibility tree (with [ref=eN] handles), reasons about which ref matches the English instruction, calls click or type_text with that ref, waits for the page to settle, calls snapshot again. The system prompt at agent.ts lines 206-218 documents this loop explicitly: 'ALWAYS call snapshot FIRST', 'After each action, call snapshot again', 'If a ref is stale (action fails), call snapshot again to get fresh refs'. The engineer never writes a ref. The agent never writes a sentence the engineer did not authorise.

Both, with judgment. The agent does what your plan says and nothing else. A read-only #Case (login as a synthetic user, navigate to a public report, assert a number) is fine in production with a dedicated test account. A destructive #Case (create and delete projects, mutate billing state) belongs in staging with a fixture dataset. The --extension flag at /Users/matthewdi/assrt-mcp/src/core/browser.ts lines 299-306 even lets the agent attach to your real running Chrome with your SSO session intact, which is the right answer for verifying that a feature works under your actual identity provider state without re-implementing it in a fixture. The trade-off is that you must be sure the plan does only what you intend; the same way you would not run a flaky destructive Cypress suite against prod, you would not run a flaky destructive #Case there either.

Faster than onto Selenium or Cypress. There is no DSL to learn. Step one: read /Users/matthewdi/assrt-mcp/README.md (under 100 lines). Step two: read the system prompt at agent.ts lines 198-254. Step three: read the parser at line 620 and the eighteen tool schemas at lines 16-196. That is roughly an hour of reading, and at the end of it the engineer can author a working #Case. Compare to a typical Selenium codebase, where the page-object hierarchy and the custom-fixture conventions are several days of orientation before the first test lands. The remaining ramp is on test design, which is the part that was always actually hard.