Playwright test maintenance vs generation: it is one decision, not two stages

Generation is a one-time cost per test. Maintenance is recurring, and on a locator-bound suite it dominates the total over the life of the project, because every UI change can break a selector that was correct when it was generated. The useful reframe is that the two are not separate problems. The artifact your generator emits decides your maintenance bill. If generation produces TypeScript with selectors baked in, you have signed up for maintenance the moment the file is written. If generation produces plain-English intent that gets re-derived against the live page on every run, there is almost nothing to maintain.

Playwright ships three test agents: a Planner that explores the app and writes a Markdown plan, a Generator that compiles that plan into Playwright test scripts in TypeScript, and a Healer that runs failing tests in debug mode and patches them. The Generator step is a one-way compile: Markdown plan goes in, TypeScript with concrete selectors comes out. From that point the Healer maintains the compiled TypeScript, not the plan. The plan becomes stale documentation. Assrt removes the compile step entirely. The plain-English #Case file is the test, and assrt_test executes it directly every run.

No, and that is deliberate. Assrt's test artifact is a #Case block in plain-English Markdown, stored at /tmp/assrt/scenario.md and meant to be checked into your repo as Markdown. There is no generated TypeScript spec sitting between generation and maintenance to go stale. If you specifically want a static, agent-free .spec.ts that runs deterministically with zero LLM calls, that is what playwright codegen and the Generator agent give you, and Assrt does not. The trade is covered in the comparison table on this page.

Both emit the same grammar. assrt_plan, the generator, runs on a system prompt that says to generate test cases in an exact format starting with #Case 1:, keep each case to 3 to 5 actions, and produce 5 to 8 cases. assrt_diagnose, the maintenance tool, runs on a different system prompt that forks the failure between an app bug, a flawed test, and an environment issue, then emits a Corrected Test Scenario in the exact same #Case format. Both prompts live in assrt-mcp/src/mcp/server.ts and both call Claude Haiku. Two tools, two separate model calls, one output format.

Only when the app's actual behavior changed in a way your intent no longer describes. A renamed button, a re-wrapped element, or a swapped data-testid is not a maintenance event for Assrt, because the agent re-snapshots the accessibility tree before every action and finds the element by role and label. You maintain a #Case when, say, signup grows a new required step. Then you call assrt_diagnose, it returns a corrected #Case, and you paste it back into scenario.md. That edit is the only maintenance, and it is in the same format the generator produced.

Yes. Because Assrt re-derives the executable test from the live page on every run, each run spends Claude Haiku tokens to drive the agent. A compiled Playwright spec spends nothing per run: it is static and deterministic. That is a real advantage of the compiled approach and you should weigh it honestly. Assrt earns its keep when your UI changes often enough that maintenance, not running, is your cost center. If your UI is frozen and a test almost never changes, a compiled spec is cheaper to run forever.

Yes. Assrt runs on Playwright under the hood, driving a real Chromium process through the Playwright MCP bridge, so a scenario.md file can sit next to a Playwright project with no conflict. A common pattern is to keep your stable, rarely-changing flows as compiled specs and move the flows that churn most, the ones generating the maintenance PRs, into #Case files. You are comparing two artifacts, not two test runners.