Playwright e2e testing, and the video layer nobody writes about

An end to end test in Playwright is a script that opens a real Chromium, Firefox, or WebKit instance, walks through a user scenario by calling page.goto, page.click, page.fill, page.waitFor and page.expect, and fails the test if any assertion does not match. The defining features are (1) a real rendering engine rather than a mocked DOM, (2) a single-file scenario that starts from a URL and ends on a verified outcome, and (3) a context object that persists cookies, localStorage and permissions across steps. Assrt runs on top of the official @playwright/mcp server, so every action is still a Playwright primitive. What changes is who writes the scenario and how you watch it back.

Playwright's video feature is a screencast of the page. It does not show a cursor, it does not show click coordinates, and it does not show keystrokes. On a human-written test you already know what the script did, so you are using the video mostly to confirm a visual state, and that is fine. On an agent-driven run the video is the only artifact that preserves ordering, intent and timing across 40-plus tool calls. Without a visible cursor and keystroke indicator you are reduced to scrubbing through still frames to figure out which button the agent actually clicked. Assrt fixes this by injecting a DOM overlay layer before every action, so the recording shows a red cursor gliding to each element, a green keystroke toast at the bottom of the page, and a click ripple that expands on every interaction. Source: browser.ts lines 33 to 98.

Four absolutely positioned DOM elements, all on z-index 2147483647 (the highest 32-bit integer, so nothing covers them): a 20px red cursor with a white border and a soft red shadow (id __pias_cursor), a 40px ring ripple that scales from 0.5 to 2 on every click (id __pias_ripple), a black rounded toast at the bottom center of the screen that shows typed characters in green monospace (id __pias_toast), and a 6px green dot in the bottom right corner that pulses forever (id __pias_heartbeat). The injection is idempotent: a window.__pias_cursor_injected flag prevents re-injection across re-renders. The exact script is CURSOR_INJECT_SCRIPT in /Users/matthewdi/assrt-mcp/src/core/browser.ts at lines 33 to 98. It is called via browser_evaluate before every action through injectOverlay() at line 429.

It is the least obvious element and the most load-bearing. Chromium's compositor only produces video frames when something on the page has actually changed. On an idle dashboard between an agent's tool calls, nothing changes, so the Playwright CDP recorder skips frames or drops chunks entirely, and the video timeline ends up with one-second gaps that look like the agent froze. The heartbeat is a 6px dot with a CSS Web Animations API keyframe that runs 800 milliseconds per iteration, iterations: Infinity, direction: alternate. It forces a compositor repaint every 400 ms forever, so the Playwright recorder always has something to encode. When you watch the WebM at 5x, the dot is a tiny blinking green pixel in the bottom right. The video stays smooth.

McpBrowserManager tracks the last cursor position server-side as two numbers, cursorX and cursorY (initialized to 640, 400 which is roughly screen center). Before every action the agent calls injectOverlay(), which reinjects the four DOM elements if they are not already there, then restores __pias_cursor to the stored (x, y) with a one-frame transition: none so it does not animate in from off-screen, and then re-enables the smooth transition 50 ms later. The result: every new page load starts with the cursor already where the last page left it, and animates smoothly to the next element.

Assrt ships a self-contained HTML player that is written to disk and auto-opened when a run finishes. It is a plain <video> tag with custom keyboard controls: Space toggles play and pause, the number keys 1, 2, 3, 5 and 0 set playback rate to 1x, 2x, 3x, 5x and 10x, left and right arrows seek by five seconds. The default speed is 5x because an agent run is often four to eight minutes, and that is too long to watch at real time when you are looking for one bug. Behind the scenes, the .webm is served from a persistent Node HTTP server on a random port that implements HTTP Range requests (status 206 Partial Content). That matters because it lets the browser seek to any frame instantly instead of redownloading the whole file. Source: server.ts lines 35 to 215.

Yes. The overlay is applied by the Assrt agent loop, not by Playwright itself. If you invoke @playwright/mcp directly (for example from your own agent or a Playwright spec file), you get the stock Playwright behavior and a stock WebM. If you call Assrt's CLI or the assrt_test MCP tool, you get the overlay plus the persistent video server plus the 5x player. Nothing about the overlay requires Assrt to own your test plans. The agent loop layer is optional, though the overlay is where most of the value lives on day one.

Neither in the usual sense. You write a plain-English plan file at /tmp/assrt/scenario.md with #Case blocks: a one-line header and three to five bullet steps under each header. The runner reads that file, builds a Playwright-backed agent loop on top of @playwright/mcp, and decides what to click and type at each step from a fresh accessibility tree snapshot. You never see a locator string. The scenario file is watched via fs.watch() with a one-second debounce (scenario-files.ts:97-103), so you can edit it while a run is in progress and the next case picks up the new version. If you want a Playwright .spec.ts file at the end, Assrt can export one, but the primary artifact is the scenario + the video + a JSON event log.

1600 by 900 WebM with the VP9 codec, hard-coded in browser.ts at lines 296 and 628 as --viewport-size 1600x900 and size: { width: 1600, height: 900 }. The resolution is chosen because it is wide enough to see sidebar and main content together on most modern SaaS apps, and because the compression on a mostly-static page at that size keeps a ten-minute run under about 15 MB. You can override the viewport by passing the viewport option into assrt_test, but there is no current knob for the output codec. WebM is what @playwright/mcp writes; Assrt does not re-encode.

No. Every overlay element has pointerEvents: 'none' in its inline style, so clicks pass through to the real page. The cursor is a pure visual, not an actual system cursor; the agent clicks elements via Playwright's element reference, not by moving a real mouse. The ripple, the toast and the heartbeat also do not receive events. A test that uses the overlay and a test that does not are equivalent from the page's point of view; the only difference is what appears in the recorded frames.

The Assrt agent and the @assrt-ai/assrt CLI are MIT licensed. When you run a test locally, the URL, the accessibility snapshot and any screenshots go directly from your machine to the LLM endpoint you configured (Anthropic by default, configurable via ANTHROPIC_BASE_URL). The video file stays on disk at /tmp/assrt/ and is served only over localhost by the persistent video server. Nothing leaves the machine unless you explicitly opt in to cloud sync, which uploads the plan and artifacts to app.assrt.ai under a scenario UUID. Compare against the $7.5K per seat per month closed runners that route every DOM through their backend by design.