Web application testing tools, graded by what they leave on your disk

The argument: features are fungible, artifacts are not

Pick any three guides covering the best web application testing tools for 2026 and you will see the same matrix: parallel execution, AI test generation, codeless recorder, real-device cloud, CI integration, free tier. Every serious tool has every one of those, or can ship the missing one in a sprint. The matrix ranks how the tool sells itself, not how it actually behaves on your machine after a run finishes.

The behavior that matters is what files exist on your filesystem when the run is done. Specifically: where they live, whether you can hand the path to a different process, and whether the format is something a coding agent can read without a vendor SDK. That is the substrate for everything downstream of a run, which is where the value lives. A failed test is only useful if you can turn it into a fix, and you can only turn it into a fix if your tools can read the failure.

Most reviews avoid this dimension because it makes the table uglier. Half the tools store the canonical test in a vendor database. Some write artifacts only to the dashboard. Some write to disk but in a binary format you need a viewer to open. The tools that win on this axis are the ones that treat the filesystem as a first-class output: every run writes a known set of files to known paths, and the tool tells you the paths.

Anchor: the directory Assrt writes after every run

Assrt is a useful reference because the layout is small, documented, and grep-able. There are two roots. The first is stable across runs and holds the editable plan plus the latest result. The second is per-run, named by a UUID, and holds the recording, the events, and the screenshots.

// assrt-mcp/src/core/scenario-files.ts:16-20
const ASSRT_DIR     = "/tmp/assrt";
const SCENARIO_FILE = join(ASSRT_DIR, "scenario.md");
const SCENARIO_META = join(ASSRT_DIR, "scenario.json");
const RESULTS_DIR   = join(ASSRT_DIR, "results");
const LATEST_RESULTS = join(RESULTS_DIR, "latest.json");

A real run from a recent test session on this site produced run id 06778d6e-574b-430c-8449-2b2ffc1157df. The artifacts directory under $TMPDIR/assrt/ contained:

• events.json, full structured event trace from the run
• execution.log, human-readable log of every step, assertion, and reasoning line
• screenshots/, 29 PNGs named 00_step0_init.png, 01_step2_click.png, on through 28_step29_assert.png
• video/recording.webm, the Playwright recording of the whole session
• video/player.html, a self-contained HTML5 player with 1x / 2x / 4x speed buttons, served on a local port so the player URL works as soon as the run finishes

The reason every file name is predictable is so a downstream process (your IDE agent, a CI script, a flake-tracking job) can find it without a callback URL or an SDK. The MCP tool response carries the absolute paths in named keys (artifactsDir, videoFile, videoPlayerFile, videoPlayerUrl, resultsFile, scenarioFile) so an agent does not have to guess.

The reviewer's table: artifact set per tool

The table below grades tools on the dimension that matters: what appears on your disk after a run, and whether a coding agent working from outside the test runner can read it.

What the agent loop looks like when artifacts cooperate

With a known on-disk layout, the loop between your IDE agent and the test runner is just file reads. No webhooks, no SDK, no dashboard scraping. The sequence below is what happens when you tell Claude Code to test a local dev server and then explain a failure.

The loop closes because every link is a file read. Compare to a tool whose canonical artifact lives in a hosted dashboard: the agent cannot Read the dashboard, so the post-mortem step collapses into "open this URL in your browser and tell the AI what you saw", which puts the human back in the slow path.

Owning the files vs. renting the dashboard

The split between local-files-first and dashboard-first is the old open vs. closed argument with a new wrinkle. The new wrinkle is that AI coding agents make the dashboard friction much worse: every dashboard becomes a place your agent cannot reach.

Where each tool fits if you grade by what they leave on disk

A short take, given the lens of this page. None of these are wrong choices in absolute terms; they fit different priorities.

• Playwright is the best raw on-disk tool for a team that writes tests by hand. The trace.zip is unbeatable for human debugging. The friction for an AI coding agent is the opaqueness of trace.zip and the single-bundle HTML report.
• Cypress is fine until you adopt Cypress Cloud. Once results live in the dashboard the agent loop frays.
• Selenium and WebdriverIO give you total freedom over the artifact set and total responsibility for designing it. If your team has the time to specify a layout and ship a custom reporter, you will end up with something better than any tool's default. Most teams do not have that time.
• QA Wolf, Mabl, and testRigor are real choices for organizations that want to outsource the testing problem entirely. The trade is that the canonical test, the canonical run, and the canonical artifact all live in the vendor.
• BrowserStack and Sauce Labs are browser grids; you bring your own runner. Their on-disk footprint is whatever your runner produces, plus a hosted video and log per session you can download.
• Assrt is the local-files-first option built for the agent loop: the plan is a markdown file, the result is JSON at a stable path, the artifacts are in a known per-run directory, and every path is returned in the MCP tool response. The generated test code is standard Playwright you can keep and run anywhere.

Try the lens on your current shortlist

Pick whatever web application testing tool you are evaluating right now and answer four questions. Do not consult marketing pages; run a real test and look at your filesystem.

1. After a run, what files exist on disk, and at what paths? Are the paths stable across runs?
2. Could you point a coding agent at one of those paths and expect a useful answer about the run?
3. If you uninstalled the tool tomorrow, would your tests still run on something else without rewriting them?
4. Where does the canonical version of the test live: in your repo, on your filesystem, or in the vendor's database?

Tools that answer "yes, yes, yes, my repo" stay in the shortlist. Tools that answer "in the dashboard" go on a different shortlist, the one for teams that want to outsource the QA problem.

Frequently asked questions