Manual Playwright vs Assrt: Same Browser Engine, Zero Test Code

1. How Manual Playwright Tests Work

A standard Playwright test is a TypeScript file that imports @playwright/test, defines test cases with test() blocks, and uses locator APIs to interact with the page. You write selectors, assertions, and control flow in code. The test runner executes the file, launches a browser, and reports pass/fail.

import { test, expect } from '@playwright/test';

test('user can log in', async ({ page }) => {
  await page.goto('https://myapp.com/login');
  await page.getByLabel('Email').fill('user@test.com');
  await page.getByLabel('Password').fill('secret123');
  await page.getByRole('button', { name: 'Sign In' }).click();
  await expect(page.getByText('Dashboard')).toBeVisible();
});

This works well. But every line is a commitment. The selector getByLabel('Email')assumes an element with that accessible label exists. If the label changes to "Email address" during a redesign, the test breaks even though the feature still works. Multiply this across hundreds of tests and maintenance becomes a second job.

2. How Assrt Works (the No-Code Layer)

Assrt uses the same browser engine (Chromium via Playwright) but removes the code layer between your intent and the browser. You write a test plan in plain English:

#Case 1: User can log in
1. Navigate to the login page
2. Type user@test.com into the email field
3. Type secret123 into the password field
4. Click the Sign In button
5. Verify the Dashboard heading is visible

No imports. No async/await. No selectors. This text gets parsed and handed to an AI agent (Claude Haiku) that reads the live page through the accessibility tree, decides which elements correspond to your descriptions, and issues tool calls through Playwright MCP to interact with them. The plan stays as text. The code never gets written.

If the label changes from "Email" to "Email address," the AI agent still finds the email input because the accessibility tree still reports an input with a name that semantically matches. No test update required.

3. Inside the TestAgent: What Actually Happens at Runtime

The core of Assrt is the TestAgent class in the SDK (located in src/core/agent.ts of the @assrt-ai/assrt npm package). Here is what it does for each test run:

1. Parse the plan. The agent splits your text on the regex /(?:#?\s*(?:Scenario|Test|Case))\s*\d*[:.]/gi. Each match becomes one scenario to execute independently.
2. Call snapshot. For each scenario, the agent first calls the Playwright MCP snapshot tool, which returns the accessibility tree as text. Every interactive element gets a ref ID like e5, e12, e47.
3. Match and act. The LLM reads the accessibility tree and your natural language step, then responds with a tool call: click(element: "Sign In button", ref: "e5"). This call goes to Playwright MCP, which calls the real Playwright API.
4. Assert. The agent uses an assert tool that takes a description and a boolean. No expect() calls, no assertion library. The agent describes what it checked and whether it passed, with human-readable evidence.
5. Report. Results are structured JSON: passedCount, failedCount, totalDuration, and per-scenario breakdowns with each assertion and its evidence string.

At no point does the TestAgent invoke a code generation function. It does not produce a .spec.ts, .test.ts, or any JavaScript file. The entire execution is tool calls interpreted at runtime. Your test plan in /tmp/assrt/scenario.md is the only artifact, and a file watcher syncs edits to cloud storage within one second.

4. Side-by-Side Comparison

5. When to Use Which

Use manual Playwright when you need sub-second test execution, cross-browser coverage (Firefox, WebKit), pixel-level visual regression testing, or when your team already has a mature test suite and dedicated QA engineers who maintain it.

Use Assrt when you have no test suite and need coverage fast, when your frontend changes frequently and selector maintenance is eating engineering hours, when your team lacks Playwright expertise, or when you want to integrate testing into an AI coding workflow (Claude Code, Cursor) where the agent can run tests after every change without writing code.

The two approaches are not mutually exclusive. Assrt can cover your happy-path flows and catch regressions with zero code investment while your team writes targeted Playwright tests for performance-critical paths or visual edge cases that need pixel precision.

6. Frequently Asked Questions