A visual regression framework is six files on your disk, not a dashboard you rent.
The top five Google results for this keyword all point at the same shape: a cloud platform with a login, a baseline PNG folder, and a bill. None of them answer the most practical question a buyer has: when my contract ends, what do I actually keep? This page answers that. It names the six paths a real visual regression framework writes to disk, verifies each one against a line number in the Assrt source, and shows what it looks like to tar the directory and replay a run offline.
Framework, not platform
If you cannot tar the whole run and replay it offline, what you bought was a vendor dashboard. A real framework leaves an artifact directory behind.
The six files are the spine of this page. Every section references them by exact path and by line number in the Assrt source. If you are evaluating visual regression tools, this is the checklist that separates frameworks from platforms.
The six files, mapped to line numbers in the source
The canonical layout is documented in a comment block at the top of assrt-mcp/src/core/scenario-files.ts lines 5 through 20. The per-run additions (screenshots directory and video directory) are written inside src/mcp/server.ts around lines 431 and 618. Every card below names a path you can ls on a machine that just ran a test.
scenario.md
Plain English plan with #Case headers. Parsed by a single regex. Edit it and the agent re-reads on the next run.
scenario.json
Metadata: id, name, url, updatedAt. Four fields. Written by writeScenarioFile() at scenario-files.ts line 42.
results/latest.json
The TestReport from the most recent run. Schema lives in types.ts lines 28-35.
results/<runId>.json
Historical archive per run. Same schema as latest. Grep this folder for fail-rate analytics across months.
<runId>/screenshots/*.png
One PNG per visual step. Naming: <index>_step<N>_<action>.png. Written at server.ts line 468. Deduped per step.
videos/<runId>.webm
Full browser recording with a standalone player.html next to it. Served over HTTP with Range support for seeking.
File 1: scenario.md, the plan in plain English
The plan is Markdown because Markdown is the least-proprietary text format a framework can pick. The parser reads a single regex on case headers and treats everything else as prose for the agent. Scenario, Test, and Case are all valid headers, so team conventions do not clash. There is no schema version, no YAML anchor, and no vendor namespace.
File 2: scenario.json, four fields of metadata
Written alongside the plan. The only purpose of this file is to give the agent a stable id to reference across runs and to record which URL the plan targets. The writeScenarioFile() function at scenario-files.ts line 42 is the single authoritative write site.
Files 3 and 4: results/latest.json and results/<runId>.json
Identical schema, different retention. The latest file is overwritten on every run, useful when an editor or LLM wants the newest report with a stable path. The per-run file is the archive you grep a year from now. Both conform to the TestReport type at types.ts lines 28-35, and every assertion conforms to TestAssertion at lines 13 through 17.
Files 5 and 6: screenshots and the WebM video
Every visual action (navigate, click, type_text, select_option, scroll, press_key) produces a JPEG for the reasoning model and a PNG on disk. The naming convention is set at src/mcp/server.ts line 468: <index>_step<N>_<action>.png. The dedup guard at lines 473 through 486 keeps only the last frame per step, so a ten-action run leaves at most ten PNGs. The video writer lands at /tmp/assrt/videos with a co-located player.html (server.ts line 618). The player speaks HTTP Range so you can scrub without loading the whole WebM.
“The visual regression framework ships as six files on your disk, documented at scenario-files.ts lines 5 through 20 and server.ts lines 431 and 618. tar the directory and the run replays offline, because every artifact is either a public format or a documented TypeScript type.”
assrt-mcp/src/core/scenario-files.ts + src/mcp/server.ts
From plan to artifacts, end to end
Every run is a fan-out from one plan file plus a URL to six artifact paths. Nothing in the pipeline touches a vendor API unless you opt in to the hosted mirror at assrt.ai. The beam below is the whole shape.
Six artifacts orbiting one directory. No cloud round trip on the path.
Framework vs platform: what actually changes on cancellation
Buyers ask the wrong question. The question is not "is this open source". The question is "what do I keep if I stop paying". That answer is decided by the artifact layout, not by the license file. The table below is the honest compare.
| Feature | Typical visual regression platform | Assrt (framework, on-disk artifacts) |
|---|---|---|
| Plan file format | Proprietary YAML, TOML, or cloud-only schema | Plain-text .md with #Case headers |
| Where the plan lives | Vendor cloud. Export on request. | /tmp/assrt/scenario.md on your disk, watched for edits |
| Results format | Signed URLs, often expiring in days | JSON on disk, schema at types.ts lines 28-35 |
| Screenshot storage | Vendor S3; retention tied to billing | /tmp/assrt/<runId>/screenshots/*.png, kept until you delete |
| Video artifact | Streamed from vendor player, not always downloadable | /tmp/assrt/videos/<runId>.webm with local player.html |
| What survives cancellation | A login screen telling you to renew | Six files. tar them and every run replays offline. |
| Typical list price at scale | Around 7,500 dollars per month for tier-3 platforms | Free. npx assrt-mcp is the open-source CLI. |
| Portability to another agent | Proprietary DSL, rewrite required | Scenario.md works on any Playwright MCP-compatible agent |
The framework's tool vocabulary is public, not proprietary
The eight chips below are the visual-action tools the agent can call during a run. They are the public Playwright MCP vocabulary, not an Assrt DSL. Any MCP-compatible agent reading the same scenario.md can execute the plan unchanged. That is what "zero vendor lock-in" means at the tool layer.
On-disk artifacts
0
Paths under /tmp/assrt after a run. Every one is a public format.
Fields on a TestAssertion
0
description, passed, evidence. types.ts lines 13-17. No tolerance knob.
Dollars per month to self-host
$0
vs around 7,500 for a tier-3 visual regression platform at scale.
Your first run, from install to tar archive
Five steps. None of them touch a vendor account. The last one is the point of the whole page: the archive is the framework.
From zero to a portable visual regression archive
- 1
Install the CLI
npx assrt-mcp. No account, no API key required for local runs.
- 2
Write scenario.md
Plain English. #Case N: headers. Grammar parser is a one-line regex.
- 3
Run
npx assrt-mcp run --url http://localhost:3000 --plan-file scenario.md
- 4
Inspect /tmp/assrt
Six files. Every one is inspectable, portable, and human-readable.
- 5
Archive
tar czf run-2026-04-19.tgz /tmp/assrt. You now own the run forever.
“We spent a quarter building custom exporters to get our visual regression history out of a SaaS vendor. Switching to a framework that just writes JSON and PNGs to disk was the most underrated productivity win of the year.”
What the artifact layout buys you in practice
Because the results live as plain JSON on disk, you can answer questions the vendor UI does not ship. Grep every results/*.json for "passed": false to get a fail timeline. Grep for a specific evidence string to find every run where the sidebar avatar regressed. Diff two results JSONs to see assertion drift between a green staging run and a red prod run. None of this requires a SQL export, an API token, or a data-engineering ticket. It requires jq.
Adopt a visual regression framework you actually own
Thirty minutes on your stack. We map your current visual regression flow to a six-file artifact layout you can tar and keep.
Book a call →Visual regression framework: specific answers
What do you actually mean by 'framework' here?
A framework is a set of primitives you compose to solve a problem class, not a single tool. Playwright is a framework; Chromatic is a platform. Most 'visual regression frameworks' the SERP recommends are platforms: they own the dashboard, the baseline store, the diff algorithm, and the access control. A real framework publishes its primitives as files you can inspect. Assrt's primitives are the plan file (scenario.md), the metadata file (scenario.json), two flavors of the results file (results/latest.json and results/<runId>.json), the per-run screenshot directory, and a WebM video. Six paths. Everything else is commentary.
Where exactly are those six files written?
All under /tmp/assrt. The layout is documented at the top of assrt-mcp/src/core/scenario-files.ts lines 5 through 20. The plan lands at /tmp/assrt/scenario.md. Metadata (id, name, url, updatedAt) lands at /tmp/assrt/scenario.json. The most recent run's results land at /tmp/assrt/results/latest.json, and every historical run is archived at /tmp/assrt/results/<runId>.json. Screenshots for a run are at /tmp/assrt/<runId>/screenshots, and the WebM video plus a standalone player.html live at /tmp/assrt/videos (set in assrt-mcp/src/mcp/server.ts around line 618). If you are testing CI-side, point these paths at a persistent volume or artifact store and you have a full archive of every run.
Why is scenario.md a .md file and not YAML or JSON?
Because the agent reads it, humans read it, and git reviews it. A .md file is the least-proprietary text format alive. The parser in assrt-mcp/src/core/scenario-files.ts is effectively a single regex on case headers (#?\s*(?:Scenario|Test|Case)), so Scenario, Test, and Case all work. The body is imperative English. No schema to validate, no version field to migrate, no Chromatic-specific keys. You can copy-paste a scenario.md into any other Playwright MCP agent and it runs unchanged, because the plan says 'go to /dashboard, verify the sidebar avatar is a real image' and every MCP agent understands that sentence.
What is the shape of the results JSON?
It is a TestReport at assrt-mcp/src/core/types.ts lines 28 through 35. Fields: url, scenarios, totalDuration, passedCount, failedCount, generatedAt. Each scenario is a ScenarioResult (lines 19 through 26) with name, passed, steps, assertions, summary, duration. Each assertion is three fields: description, passed, evidence (lines 13 through 17). No proprietary IDs, no signed URLs that expire in seven days. You can grep failedCount across a year of archived run files and answer real questions like 'what is my visual regression fail rate by week'.
How does this differ from the Assrt visual regression tutorial page?
That tutorial focuses on why pass/fail uses an LLM reading a JPEG instead of pixelmatch diffing a baseline PNG. This page is the framework layer above that: what files are produced, what formats they are in, and why that matters if you ever want to leave your vendor. If you are deciding whether to adopt Assrt, read the tutorial for the reasoning model and read this for the data-portability model. They answer different questions. Both are grounded in the same repo.
Can I redirect these outputs to a persistent volume for CI?
Yes. /tmp/assrt is the default because /tmp is writable on every CI image without extra config. In GitHub Actions you can copy the directory as a build artifact after the run, or symlink /tmp/assrt to a mounted volume before the run. The paths themselves are stable: scenario.md and scenario.json always sit at the root of the directory, results land under results/, screenshots under <runId>/screenshots, and the video under videos/<runId>.webm. None of those paths contain random UUIDs you have to discover. If you want the raw list of written files, assrt-mcp/src/mcp/server.ts lines 431 through 486 and lines 618 through 629 are the authoritative write sites.
What happens to these files if I cancel my Assrt subscription tomorrow?
Nothing. They are on your disk. npx assrt-mcp is the open source npm package that wrote them, and you can keep running it offline against any Playwright MCP-compatible agent. The plan is English; any model that speaks Playwright MCP executes it. The results JSON is plain, unencrypted, schema-documented TypeScript types. The screenshots are PNGs. The video is a WebM playable in every modern browser with Range support. The comparable tier-3 AI visual regression platforms charge around 7,500 dollars a month and keep your scenarios, selectors, and diff history in their cloud; cancel there and the login page is the last thing you see. On Assrt, cancel and tar /tmp/assrt; the archive replays.
Is there a cloud dashboard at all?
There is an optional hosted view at assrt.ai for teams that want one, but it is not the framework. The framework is the local files, exposed through the MCP tools assrt_test, assrt_plan, and assrt_diagnose. If you run everything locally and never log in to the cloud view, every feature still works because the agent reads and writes the local files directly. The cloud view is a mirror, not the source of truth. That inversion is the whole point of calling this a framework instead of a platform.
What about pass criteria and screenshots-per-step, are those separate files?
Pass criteria live inline in scenario.md under a 'Pass Criteria' sub-section of a #Case; the agent reads them as additional assertion targets for that case. Screenshots-per-step are automatic: after any visual action (navigate, click, type_text, select_option, scroll, press_key) the agent captures a JPEG, emits it to the model for reasoning, and also writes it to disk at /tmp/assrt/<runId>/screenshots/<index>_step<stepNumber>_<action>.png (naming convention at assrt-mcp/src/mcp/server.ts line 468). The deduplication logic at server.ts lines 473 through 486 keeps only the last frame per step, so a run with 12 visual actions produces at most 12 PNGs.
Does 'no vendor lock-in' just mean 'open source'?
No. Open source is necessary but not sufficient. Many OSS visual regression frameworks still assume you run their central diff service, and the scenario or baseline format is theirs. Lock-in survives the license if you cannot move the artifacts. Assrt's stronger claim is that every artifact it writes is either a public file format (.md, .json, .png, .webm) or a documented TypeScript type (types.ts lines 13 through 35). There is no Assrt-proprietary DSL. You can point a different Playwright MCP agent at the same scenario.md and run it with zero edits, because the tools the agent uses (navigate, click, type_text, and the rest) are the public Playwright MCP vocabulary, not Assrt's invention.
Adjacent pages on the same product: the tutorial below the framework layer, the tool roundup for context, and the 4-tier taxonomy that explains why this layout looks different.
Keep reading
Visual regression tutorial without toHaveScreenshot()
Semantic visual regression: JPEGs go to Claude Haiku, not to pixelmatch. Zero golden PNGs, zero maxDiffPixels.
Best visual regression tools
A comparison that names prices honestly, including the 7,500 dollar per month tier enterprise pages never show.
E2E testing frameworks: the 4-tier taxonomy every top-10 list misses
Scripted, low-code, AI-compiled, agent-interpreted. Assrt sits in tier 4, which is why its artifact layout looks different.
Comments (••)
Leave a comment to see what others are saying.Public and anonymous. No signup.