Playwright test scenario generator: what one URL actually becomes

The four-stage pipeline, with numbers

A scenario generator is a small pipeline with five or six tunable knobs. The defaults are the most interesting part. Below is what Assrt's open-source generator does to one URL, in order. The line numbers point at the actual source so you can verify each step yourself.

The anchor fact (verifiable, open source)

Most write-ups of a scenario generator stop at the marketing diagram. The interesting question is what the budget actually is. Here are the five numbers that define Assrt's generator, all traceable to ~/assrt-mcp/src/mcp/server.ts:

• 1.3 screenshots at scroll offsets 0, 800, 1600 pixels (lines 794-805). Each captures both a JPEG and the accessibility tree.
• 2.8000-character slice on the concatenated accessibility text (line 809). The three trees are joined with a blank line, then truncated.
• 3.Model is claude-haiku-4-5-20251001 by default, overridable via the model parameter (line 830).
• 4.max_tokens 4096 on the output (line 831). Enough for 5-8 #Case blocks with 3-5 actions each, not enough to ramble.
• 5.The system prompt caps output at 5-8 cases of 3-5 actions each (lines 234, 236). The cap is policy, not a parser, but it holds in practice.

The repo is at github.com/assrt-ai/assrt-mcp. Clone it, open src/mcp/server.ts, jump to line 766, and the numbers above are right there.

Plan as artifact, or plan as a draft of code?

The single biggest decision in this category is what the artifact looks like after the generator runs. Playwright's Generator agent and Codegen both compile to a TypeScript spec; that spec is what your repo holds and what CI runs. Assrt stops at Markdown; the same file you commit is the one the runner reads at execution time. The trade is concrete: a spec file lets you eyeball the locator strategy at PR time, a plan file lets the runner re-decide on every run.

The constraints the prompt actually enforces

The cap of 5-8 cases is not the only thing the prompt does. Six rules sit at the heart of the generator, and the model treats them as load-bearing. Every rule maps to a behavior that, if you remove it, makes the suite worse on a real app. The full prompt is at server.ts:219-236.

What you get back: scenario.md

The output is one file. If you point the generator at a generic SaaS landing page, what comes back looks roughly like this. Every #Case is a separate execution unit; each numbered line is one tool call away in the runner.

# /tmp/assrt/scenario.md

#Case 1: Visitor lands on the homepage and sees the hero
1. Navigate to /
2. Assert: the page title contains "Acme"
3. Assert: a heading reading "Ship faster" is visible
4. Assert: a button reading "Start free" is visible

#Case 2: Visitor opens the pricing page from the navbar
1. Navigate to /
2. Click the "Pricing" link in the top navigation
3. Assert: the URL path is /pricing
4. Assert: a heading reading "Plans" is visible

#Case 3: Visitor submits the contact form
1. Navigate to /contact
2. Type "test@assrt.dev" into the email field
3. Type "Just curious" into the message field
4. Click the "Send" button
5. Assert: a confirmation reading "Thanks" is visible

#Case 4: Visitor signs up with a disposable email
1. Navigate to /signup
2. Create a temporary email
3. Type the temporary email into the email field
4. Type "Hunter2-Hunter2" into the password field
5. Click the "Create account" button
6. Wait for the verification code
7. Assert: the URL path is /onboarding

The file is what you commit. To re-run later, pass scenarioId (the UUID written to scenario.json alongside) to assrt_test and the same plan runs against the same URL.

How this stacks up against Playwright's v1.56 agents

Playwright shipped its Planner, Generator, and Healer agents in version 1.56 in October 2025. They are well-built and well-documented. The Planner writes a test-plan.md, the Generator compiles it to a .spec.ts, the Healer repairs the .spec.ts when a locator drifts. The pipeline assumes you proceed from plan to spec and live with the spec.

Assrt fits the same niche from the other direction. The plan stays the artifact; there is no Generator step to compile, and no Healer step is needed because the locator is re-derived from a fresh accessibility snapshot on every run. The trade-off is real: a spec file lets you statically analyze the locator strategy and run it with plain npx playwright test with no agent in the loop. A plan file requires the Assrt runner at execution time, in exchange for never having to re-bind locators after a redesign.

One specific gap worth noting: pages all over the web describe what the v1.56 agents dobut very few publish the budget inside them. The official docs are intentionally open-ended so the agents can be configured per project. That makes them flexible, but also makes the user's mental model fuzzier. Assrt's generator is the opposite: a 100-line tool with five tunable knobs, all in one file. If you want to understand the shape of a scenario generator without reverse-engineering it, the open source path is shorter.

When a scenario generator helps, and when it does not

A scenario generator earns its keep on a page where the testable affordances are visible and the user's job-to-be-done is obvious from the layout. Landing pages, marketing sites, pricing pages, public app dashboards, and most onboarding flows fit this shape. The generator looks at the hero, sees the CTA, sees the secondary nav, and writes the cases a careful human would.

It does worse on three shapes. The first is anything gated by auth: the generator sees the login form and writes a case that signs in, but it cannot guess the post-login flow without exploring it. The second is highly state-dependent apps where the visible UI depends on which row of data you happen to be viewing; the generator picks one path and misses the rest. The third is anything that depends on timing or animation; the model sees a static snapshot and does not know whether a button was about to slide in. For these, the right workflow is to generate a draft, then hand-edit scenario.md for the cases that need it.

Try the generator on your URL

The whole thing is one command, no API key signup, no cloud step. The MCP server runs in your terminal or inside Claude Desktop. You point it at any URL, get back a draft scenario.md, and decide what to keep.

# install the CLI
npx @m13v/assrt discover https://your-app.com

# or use it from inside Claude as an MCP tool:
#   assrt_plan({ url: "https://your-app.com" })
# the plan lands at /tmp/assrt/scenario.md.

# then run the cases:
#   assrt_test({ url: "https://your-app.com" })

Frequently asked questions