Real Playwright Tests, Not Automation Scripts: How Assrt Eliminated the .spec.ts File

1. The Code Generation Trap

When most tools say they "generate Playwright tests," they mean one of two things. Either they produce .spec.ts files with hardcoded selectors and assertions, or they output proprietary YAML that wraps Playwright underneath. Both approaches share the same fundamental problem: they create a code artifact that immediately starts decaying.

A generated .spec.ts file works perfectly the moment it is created. The selectors match the current DOM. The assertions pass against today's UI. But the moment a developer changes a button label, rearranges a form, or updates a component library, the file becomes a maintenance liability. Someone has to update the selectors. Someone has to fix the assertions. Someone has to run the suite and figure out which failures are real bugs and which are stale tests.

Proprietary YAML formats add another layer to this problem. The tests are locked to a specific vendor's runtime. If you outgrow the tool or the vendor raises prices (and $7,500/month is not uncommon for enterprise testing platforms), your YAML files are worthless outside their ecosystem. You cannot run them with standard Playwright. You cannot port them to another tool without rewriting every test from scratch.

The root issue is not the quality of the generated code. The issue is that generating code is the wrong abstraction for tests that need to survive UI changes.

2. How Assrt Actually Works: The Agent as Test Runner

Assrt takes a fundamentally different approach. Instead of generating code that a test runner executes, an AI agent is the test runner. You write (or let AI generate) test scenarios in plain English. The agent reads each scenario, interprets the steps, and drives a real Playwright browser session to execute them.

The architecture is a two-stage pipeline. First, a planning model (Gemini) analyzes a page's screenshots, DOM structure, and interactive elements to generate test scenarios in a simple #Case N: Markdown format. Each case contains 3 to 5 steps describing what to do and what to verify. Second, an execution model (Claude Haiku) reads those scenarios and translates each step into real Playwright API calls inside an ephemeral browser VM.

There is no intermediate code generation step. The agent does not produce a .spec.ts file and then run it. The agent reads "click the Login button" and calls browser_click with the appropriate element reference. It reads "verify the dashboard shows Welcome text" and calls browser_wait_for with that text string. Every interaction is a genuine Playwright MCP tool invocation against a real Chromium browser.

This means the agent adapts in real time. If a button label changed from "Submit" to "Save Changes," the agent uses Playwright's accessibility tree snapshot to find the right element by role and context, not by a hardcoded selector from three months ago. The scenario still says "click the save button." The agent figures out which element that refers to right now, in the current state of the page.

3. The 10 Real Playwright APIs Under the Hood

This is not a wrapper that pretends to use Playwright. The Assrt agent calls actual Playwright MCP tools through convenience methods defined in browser.ts. Each method maps directly to a Playwright browser operation:

Each of these methods includes cursor overlay injection and keystroke visualization, so test runs produce watchable video recordings showing exactly what the agent did. The click method waits 400ms for a CSS cursor glide animation before executing, so the recorded video is human-readable. This is not a headless script blasting through a page in 200ms. It is a visible, auditable browser session.

4. Scenarios, Not Scripts: What the Test Input Looks Like

Traditional test automation requires you to learn an API, write code in a specific language, and structure your tests according to a framework's conventions. Assrt scenarios are plain Markdown. Here is what an actual scenario looks like:

#Case 1: Login with valid credentials
Navigate to the login page.
Fill in test@example.com in the email field.
Fill in the password field with a valid password.
Click the Login button.
Verify that the dashboard loads with "Welcome" text visible.

#Case 2: Empty form submission shows validation
Navigate to the login page.
Click the Login button without filling in any fields.
Verify that an error message appears for the email field.

The agent's parseScenarios function splits this text on the #Case delimiter using a regex pattern. Each block becomes an independent test scenario with a name and steps. The agent then executes each scenario in sequence, collecting pass/fail results, screenshots, and assertion evidence into a structured JSON report.

Because scenarios are plain text, they are version-controllable, diffable, and readable by anyone on the team. A product manager can write a scenario. A QA engineer can refine it. A developer can review it in a pull request. Nobody needs to understand Playwright's locator API, assertion library, or test runner configuration.

And because the scenarios are interpreted at runtime rather than compiled into code, the same scenario works across UI changes without modification. "Click the Login button" works whether that button is a <button>, an <a> styled as a button, or a <div role="button">. The agent resolves the reference against the live accessibility tree each time it runs.

5. What You Stop Maintaining

The practical difference comes down to what disappears from your workflow:

The trade-off is real: you lose the fine-grained control of hand-written Playwright code. If you need to test a specific WebSocket message sequence or validate an HTTP response header, a .spec.ts file is still the right tool. Assrt is for the 80% of E2E testing where the question is "does this user flow work from start to finish?" and the test should be as easy to write as describing the flow in a sentence.

Frequently Asked Questions