The framework read as files, not as a login screen

A visual regression framework is six files on your disk, not a dashboard you rent.

The top five Google results for this keyword all point at the same shape: a cloud platform with a login, a baseline PNG folder, and a bill. None of them answer the most practical question a buyer has: when my contract ends, what do I actually keep? This page answers that. It names the six paths a real visual regression framework writes to disk, verifies each one against a line number in the Assrt source, and shows what it looks like to tar the directory and replay a run offline.

M
Matthew Diakonov
9 min read
4.8from Assrt MCP users
Six on-disk artifacts, layout documented at scenario-files.ts lines 5-20
Zero proprietary DSL; plan is a .md file, results are plain JSON
Open source npx assrt-mcp vs tier-3 platforms at around 7,500 dollars per month

Framework, not platform

If you cannot tar the whole run and replay it offline, what you bought was a vendor dashboard. A real framework leaves an artifact directory behind.

The six files are the spine of this page. Every section references them by exact path and by line number in the Assrt source. If you are evaluating visual regression tools, this is the checklist that separates frameworks from platforms.

The six files, mapped to line numbers in the source

The canonical layout is documented in a comment block at the top of assrt-mcp/src/core/scenario-files.ts lines 5 through 20. The per-run additions (screenshots directory and video directory) are written inside src/mcp/server.ts around lines 431 and 618. Every card below names a path you can ls on a machine that just ran a test.

scenario.md

Plain English plan with #Case headers. Parsed by a single regex. Edit it and the agent re-reads on the next run.

scenario.json

Metadata: id, name, url, updatedAt. Four fields. Written by writeScenarioFile() at scenario-files.ts line 42.

results/latest.json

The TestReport from the most recent run. Schema lives in types.ts lines 28-35.

results/<runId>.json

Historical archive per run. Same schema as latest. Grep this folder for fail-rate analytics across months.

<runId>/screenshots/*.png

One PNG per visual step. Naming: <index>_step<N>_<action>.png. Written at server.ts line 468. Deduped per step.

videos/<runId>.webm

Full browser recording with a standalone player.html next to it. Served over HTTP with Range support for seeking.

File 1: scenario.md, the plan in plain English

The plan is Markdown because Markdown is the least-proprietary text format a framework can pick. The parser reads a single regex on case headers and treats everything else as prose for the agent. Scenario, Test, and Case are all valid headers, so team conventions do not clash. There is no schema version, no YAML anchor, and no vendor namespace.

/tmp/assrt/scenario.md

File 2: scenario.json, four fields of metadata

Written alongside the plan. The only purpose of this file is to give the agent a stable id to reference across runs and to record which URL the plan targets. The writeScenarioFile() function at scenario-files.ts line 42 is the single authoritative write site.

/tmp/assrt/scenario.json

Files 3 and 4: results/latest.json and results/<runId>.json

Identical schema, different retention. The latest file is overwritten on every run, useful when an editor or LLM wants the newest report with a stable path. The per-run file is the archive you grep a year from now. Both conform to the TestReport type at types.ts lines 28-35, and every assertion conforms to TestAssertion at lines 13 through 17.

/tmp/assrt/results/<runId>.json

Files 5 and 6: screenshots and the WebM video

Every visual action (navigate, click, type_text, select_option, scroll, press_key) produces a JPEG for the reasoning model and a PNG on disk. The naming convention is set at src/mcp/server.ts line 468: <index>_step<N>_<action>.png. The dedup guard at lines 473 through 486 keeps only the last frame per step, so a ten-action run leaves at most ten PNGs. The video writer lands at /tmp/assrt/videos with a co-located player.html (server.ts line 618). The player speaks HTTP Range so you can scrub without loading the whole WebM.

ls /tmp/assrt after a two-case run
6 files

The visual regression framework ships as six files on your disk, documented at scenario-files.ts lines 5 through 20 and server.ts lines 431 and 618. tar the directory and the run replays offline, because every artifact is either a public format or a documented TypeScript type.

assrt-mcp/src/core/scenario-files.ts + src/mcp/server.ts

From plan to artifacts, end to end

Every run is a fan-out from one plan file plus a URL to six artifact paths. Nothing in the pipeline touches a vendor API unless you opt in to the hosted mirror at assrt.ai. The beam below is the whole shape.

/tmp/assrt
scenario.md
scenario.json
results/latest.json
results/<runId>.json
screenshots/
videos/<runId>.webm

Six artifacts orbiting one directory. No cloud round trip on the path.

Framework vs platform: what actually changes on cancellation

Buyers ask the wrong question. The question is not "is this open source". The question is "what do I keep if I stop paying". That answer is decided by the artifact layout, not by the license file. The table below is the honest compare.

FeatureTypical visual regression platformAssrt (framework, on-disk artifacts)
Plan file formatProprietary YAML, TOML, or cloud-only schemaPlain-text .md with #Case headers
Where the plan livesVendor cloud. Export on request./tmp/assrt/scenario.md on your disk, watched for edits
Results formatSigned URLs, often expiring in daysJSON on disk, schema at types.ts lines 28-35
Screenshot storageVendor S3; retention tied to billing/tmp/assrt/<runId>/screenshots/*.png, kept until you delete
Video artifactStreamed from vendor player, not always downloadable/tmp/assrt/videos/<runId>.webm with local player.html
What survives cancellationA login screen telling you to renewSix files. tar them and every run replays offline.
Typical list price at scaleAround 7,500 dollars per month for tier-3 platformsFree. npx assrt-mcp is the open-source CLI.
Portability to another agentProprietary DSL, rewrite requiredScenario.md works on any Playwright MCP-compatible agent

The framework's tool vocabulary is public, not proprietary

The eight chips below are the visual-action tools the agent can call during a run. They are the public Playwright MCP vocabulary, not an Assrt DSL. Any MCP-compatible agent reading the same scenario.md can execute the plan unchanged. That is what "zero vendor lock-in" means at the tool layer.

navigateclicktype_textselect_optionscrollpress_keyscreenshotassert

On-disk artifacts

0

Paths under /tmp/assrt after a run. Every one is a public format.

Fields on a TestAssertion

0

description, passed, evidence. types.ts lines 13-17. No tolerance knob.

Dollars per month to self-host

$0

vs around 7,500 for a tier-3 visual regression platform at scale.

Your first run, from install to tar archive

Five steps. None of them touch a vendor account. The last one is the point of the whole page: the archive is the framework.

From zero to a portable visual regression archive

  1. 1

    Install the CLI

    npx assrt-mcp. No account, no API key required for local runs.

  2. 2

    Write scenario.md

    Plain English. #Case N: headers. Grammar parser is a one-line regex.

  3. 3

    Run

    npx assrt-mcp run --url http://localhost:3000 --plan-file scenario.md

  4. 4

    Inspect /tmp/assrt

    Six files. Every one is inspectable, portable, and human-readable.

  5. 5

    Archive

    tar czf run-2026-04-19.tgz /tmp/assrt. You now own the run forever.

We spent a quarter building custom exporters to get our visual regression history out of a SaaS vendor. Switching to a framework that just writes JSON and PNGs to disk was the most underrated productivity win of the year.
A
Anonymous platform engineer
Series B SaaS, 40-engineer org

What the artifact layout buys you in practice

Because the results live as plain JSON on disk, you can answer questions the vendor UI does not ship. Grep every results/*.json for "passed": false to get a fail timeline. Grep for a specific evidence string to find every run where the sidebar avatar regressed. Diff two results JSONs to see assertion drift between a green staging run and a red prod run. None of this requires a SQL export, an API token, or a data-engineering ticket. It requires jq.

Adopt a visual regression framework you actually own

Thirty minutes on your stack. We map your current visual regression flow to a six-file artifact layout you can tar and keep.

Book a call

Visual regression framework: specific answers

What do you actually mean by 'framework' here?

A framework is a set of primitives you compose to solve a problem class, not a single tool. Playwright is a framework; Chromatic is a platform. Most 'visual regression frameworks' the SERP recommends are platforms: they own the dashboard, the baseline store, the diff algorithm, and the access control. A real framework publishes its primitives as files you can inspect. Assrt's primitives are the plan file (scenario.md), the metadata file (scenario.json), two flavors of the results file (results/latest.json and results/<runId>.json), the per-run screenshot directory, and a WebM video. Six paths. Everything else is commentary.

Where exactly are those six files written?

All under /tmp/assrt. The layout is documented at the top of assrt-mcp/src/core/scenario-files.ts lines 5 through 20. The plan lands at /tmp/assrt/scenario.md. Metadata (id, name, url, updatedAt) lands at /tmp/assrt/scenario.json. The most recent run's results land at /tmp/assrt/results/latest.json, and every historical run is archived at /tmp/assrt/results/<runId>.json. Screenshots for a run are at /tmp/assrt/<runId>/screenshots, and the WebM video plus a standalone player.html live at /tmp/assrt/videos (set in assrt-mcp/src/mcp/server.ts around line 618). If you are testing CI-side, point these paths at a persistent volume or artifact store and you have a full archive of every run.

Why is scenario.md a .md file and not YAML or JSON?

Because the agent reads it, humans read it, and git reviews it. A .md file is the least-proprietary text format alive. The parser in assrt-mcp/src/core/scenario-files.ts is effectively a single regex on case headers (#?\s*(?:Scenario|Test|Case)), so Scenario, Test, and Case all work. The body is imperative English. No schema to validate, no version field to migrate, no Chromatic-specific keys. You can copy-paste a scenario.md into any other Playwright MCP agent and it runs unchanged, because the plan says 'go to /dashboard, verify the sidebar avatar is a real image' and every MCP agent understands that sentence.

What is the shape of the results JSON?

It is a TestReport at assrt-mcp/src/core/types.ts lines 28 through 35. Fields: url, scenarios, totalDuration, passedCount, failedCount, generatedAt. Each scenario is a ScenarioResult (lines 19 through 26) with name, passed, steps, assertions, summary, duration. Each assertion is three fields: description, passed, evidence (lines 13 through 17). No proprietary IDs, no signed URLs that expire in seven days. You can grep failedCount across a year of archived run files and answer real questions like 'what is my visual regression fail rate by week'.

How does this differ from the Assrt visual regression tutorial page?

That tutorial focuses on why pass/fail uses an LLM reading a JPEG instead of pixelmatch diffing a baseline PNG. This page is the framework layer above that: what files are produced, what formats they are in, and why that matters if you ever want to leave your vendor. If you are deciding whether to adopt Assrt, read the tutorial for the reasoning model and read this for the data-portability model. They answer different questions. Both are grounded in the same repo.

Can I redirect these outputs to a persistent volume for CI?

Yes. /tmp/assrt is the default because /tmp is writable on every CI image without extra config. In GitHub Actions you can copy the directory as a build artifact after the run, or symlink /tmp/assrt to a mounted volume before the run. The paths themselves are stable: scenario.md and scenario.json always sit at the root of the directory, results land under results/, screenshots under <runId>/screenshots, and the video under videos/<runId>.webm. None of those paths contain random UUIDs you have to discover. If you want the raw list of written files, assrt-mcp/src/mcp/server.ts lines 431 through 486 and lines 618 through 629 are the authoritative write sites.

What happens to these files if I cancel my Assrt subscription tomorrow?

Nothing. They are on your disk. npx assrt-mcp is the open source npm package that wrote them, and you can keep running it offline against any Playwright MCP-compatible agent. The plan is English; any model that speaks Playwright MCP executes it. The results JSON is plain, unencrypted, schema-documented TypeScript types. The screenshots are PNGs. The video is a WebM playable in every modern browser with Range support. The comparable tier-3 AI visual regression platforms charge around 7,500 dollars a month and keep your scenarios, selectors, and diff history in their cloud; cancel there and the login page is the last thing you see. On Assrt, cancel and tar /tmp/assrt; the archive replays.

Is there a cloud dashboard at all?

There is an optional hosted view at assrt.ai for teams that want one, but it is not the framework. The framework is the local files, exposed through the MCP tools assrt_test, assrt_plan, and assrt_diagnose. If you run everything locally and never log in to the cloud view, every feature still works because the agent reads and writes the local files directly. The cloud view is a mirror, not the source of truth. That inversion is the whole point of calling this a framework instead of a platform.

What about pass criteria and screenshots-per-step, are those separate files?

Pass criteria live inline in scenario.md under a 'Pass Criteria' sub-section of a #Case; the agent reads them as additional assertion targets for that case. Screenshots-per-step are automatic: after any visual action (navigate, click, type_text, select_option, scroll, press_key) the agent captures a JPEG, emits it to the model for reasoning, and also writes it to disk at /tmp/assrt/<runId>/screenshots/<index>_step<stepNumber>_<action>.png (naming convention at assrt-mcp/src/mcp/server.ts line 468). The deduplication logic at server.ts lines 473 through 486 keeps only the last frame per step, so a run with 12 visual actions produces at most 12 PNGs.

Does 'no vendor lock-in' just mean 'open source'?

No. Open source is necessary but not sufficient. Many OSS visual regression frameworks still assume you run their central diff service, and the scenario or baseline format is theirs. Lock-in survives the license if you cannot move the artifacts. Assrt's stronger claim is that every artifact it writes is either a public file format (.md, .json, .png, .webm) or a documented TypeScript type (types.ts lines 13 through 35). There is no Assrt-proprietary DSL. You can point a different Playwright MCP agent at the same scenario.md and run it with zero edits, because the tools the agent uses (navigate, click, type_text, and the rest) are the public Playwright MCP vocabulary, not Assrt's invention.

How did this page land for you?

React to reveal totals

Comments ()

Leave a comment to see what others are saying.

Public and anonymous. No signup.