QA Tester Automation: A Practical Guide to Bridging Manual Testers and Real Test Code

1. The Handoff Problem Nobody Talks About

A manual QA tester writes a test case in a doc, a ticket, or a spreadsheet. It reads like: "Open the signup page, fill out the form with a valid email, click Submit, confirm the welcome email arrives." Five sentences. Five minutes to write. Any tester on the team can run it and any product manager can read it.

An automation engineer receives this and rewrites it. Now it is sixty lines of Playwright, or two hundred lines of proprietary YAML inside a hosted test platform. The tester cannot read it. The engineer has to translate every future edit. When the signup form changes, two artifacts have to change: the tester's checklist and the automation code. In practice, one of them rots, and eventually they represent different things.

This is the actual reason QA tester automation is hard. The artifact a tester produces is not the artifact an automation suite needs. Teams bridge the gap with record-and-replay tools, hosted codeless platforms, or full-time translator engineers. All three approaches trade one problem for a different one: lock-in, flaky recordings, or permanent headcount.

There is a better pattern. Make the scenario itself the shared artifact. Keep it in plain text. Make it editable by both humans and coding agents. Generate the Playwright code from it every run, not once. Throw the generated code away if you want to. The spec is the source.

2. The Scenario File as the Shared Artifact

Instead of hiding scenarios inside a hosted platform, put them on disk, in markdown, where testers and engineers already work. That sounds obvious, and it is how the Assrt MCP server is actually built. When you run a test, three files appear in a predictable place:

/tmp/assrt/
├── scenario.md          # the plan, in #Case N: name format
├── scenario.json        # metadata: {id, name, url, updatedAt}
└── results/
    ├── latest.json      # most recent run
    └── <runId>.json     # every prior run, by ID

You can verify this in the source. The paths are declared in src/core/scenario-files.ts in the assrt-mcp repository, as the exported PATHS constant. When a test runs, writeScenarioFile writes the plan, then startWatching opens an fs.watch on it. Any edit you make, from VS Code, Vim, or a coding agent, triggers a 1 second debounce and then syncs the new plan to central storage. The debounce is deliberate: 1 second is long enough to coalesce rapid keystrokes, short enough that a save and a re-run feel instant.

This is the piece that is different from every hosted QA automation tool. The scenario is not trapped behind a web UI. It is a file. A tester can open it. A product manager can comment on it in a pull request. A coding agent can read it, rewrite it, and save it back. The file is the contract between the humans and the automation.

3. The #Case Format, In Detail

The scenario file uses a minimal, line-based format that a tester can learn in under a minute. Each test case is a block that starts with #Case N: short name, followed by plain English steps. That is the entire syntax.

#Case 1: signup with valid email
Go to /signup. Fill out the form with a test email.
Click Submit. Verify the welcome message appears.

#Case 2: signup with existing email
Go to /signup. Use an email that is already registered.
Verify the inline error says "already in use".

#Case 3: password strength meter
Go to /signup. Type a weak password.
Verify the strength indicator shows red.

This is not pseudocode. It is not a domain-specific language. A non-technical stakeholder reads it and understands what is being tested. A Playwright engineer reads it and knows what selectors to target. An AI agent reads it and produces a real browser-driven run. Crucially, any of the three can also edit it, and the edit is saved to one place.

The format is enforced by convention, not by a parser, which matters. If you want to add a fourth case, you add #Case 4: ... and write two sentences. There is no schema to learn, no nested YAML to indent, no proprietary keyword list. If your team decides to move to a different automation runner later, the scenario file is still a readable checklist. That is the whole point.

4. The Handoff to Real Playwright Code

Plain English scenarios are great for collaboration, but they are a terrible debugging format when a test fails in a subtle way. You need to drop into real code. Two design choices make this tractable.

First, the generated code is Playwright, not a proprietary runtime. When the agent drives the browser, it uses @playwright/mcpunder the hood. Playwright is open-source, runs anywhere, has a twelve-thousand-word documentation site and a full community on StackOverflow. When your scenario graduates from "smoke test" to "critical pre-deploy gate" and you want to own the code, you can. You copy the Playwright script into your repo and it keeps working. There is nothing to export, nothing to convert, no migration tool to run.

Second, results are structured JSON, not a proprietary report format. After each run, writeResultsFile produces /tmp/assrt/results/latest.json and a timestamped copy in the same directory. You can pipe that into your existing dashboards. You can diff it between runs. You can feed failures back into a coding agent to fix them. Because it is a file, not an API response that expires, the record is yours.

The practical flow looks like this. A tester writes #Case 3: password strength meter. The agent runs it, fails, writes a diagnosis to disk. A developer opens latest.json, sees that the selector for the strength bar changed class names, edits the scenario to be more specific, and commits both the product fix and the updated scenario.md in the same pull request. The tester reviews the scenario change. Everyone is reading the same artifact.

5. What to Automate, What to Keep Manual

The promise of QA tester automation is tempting but dangerous. Not every tester-written check should be automated, even when it is easy to do so. A useful rule of thumb: automate the things whose failure is boring and repeated, keep manual the things whose failure is judgment-heavy.

The scenario-as-markdown pattern supports both. Automated cases live in scenario.md and run every deploy. Manual-only cases live in a separate manual.md with the same format, which a tester reads before a release. The format is identical so a case can migrate from manual to automated by copying the block into the other file. There is no conversion step.

6. Ownership, Portability, and Vendor Lock-In

The hidden cost of most codeless QA platforms is that the test artifact lives on someone else's server. If the vendor raises prices, pivots, or goes out of business, your test suite evaporates. Several enterprise QA SaaS tools charge $7,500 a month per seat and store every scenario as an opaque YAML blob you cannot read outside their UI. That is fine if you plan to stay on their platform forever. It is catastrophic if you ever want to leave.

Three checks will tell you whether a QA tester automation tool respects your ownership:

1. Can you read the test artifact in a text editor, without the vendor's app? If not, you have a lock-in problem.
2. Is the runtime open-source? Playwright, Selenium, and Cypress are. Most "AI QA" startups run a proprietary runtime. When they shut down, your tests do too.
3. Can you self-host? For teams in regulated industries, this is not optional. For everyone else, it is leverage: the ability to move vendors without rewriting your test suite.

Assrt scores three for three on these checks. The scenarios are markdown on your disk. The runtime is Playwright, open-source under the Apache 2.0 license. The MCP server is open-source and self-hostable; the source is what this guide references directly. You can pay zero dollars forever and keep everything.

7. FAQ

Where exactly are Assrt scenarios stored on disk?

In /tmp/assrt/scenario.md for the plan, /tmp/assrt/scenario.json for metadata, and /tmp/assrt/results/latest.json for the most recent run. Historical runs are kept in the same results directory indexed by run ID. You can verify this in src/core/scenario-files.ts.

What is the #Case format, precisely?

A test case is a block that starts with #Case N: short name on its own line, followed by one or more sentences describing what to do and what to verify. No indentation, no YAML, no quotes. Multiple cases stack in one file separated by blank lines.

How is the scenario file kept in sync when I edit it?

The MCP server opens an fs.watch on the file as soon as a scenario is loaded. Edits are debounced by 1 second, then pushed to central storage. The debounce value matters: rapid keystrokes coalesce into a single sync, and a save-then-rerun cycle still feels instant.

Can I use the generated Playwright code without Assrt?

Yes. The browser runtime is @playwright/mcp and the scripts it produces are standard Playwright. Copy the code into your own repo, install Playwright, and it runs. There is no proprietary glue layer to replicate.

Is this a replacement for manual QA testers?

No. It is a replacement for the translation step between manual testers and automation engineers. Manual testers still write scenarios, still do exploratory testing, still catch judgment-heavy failures. What goes away is the weeks of lag where a tester's scenario sits in a ticket waiting to become automated.

What happens if the scenario file is deleted?

Each scenario has a UUID stored in scenario.json. You can re-fetch the plan from central storage by ID and writeScenarioFile will recreate the markdown locally. The file on disk is a convenient editable cache; the record of record is the scenario ID.