Why no single WCAG test SaaS actually covers WCAG.

No, and I want to be honest about that up front. Assrt does not output a WCAG 2.2 violations report. It does not classify issues by success criterion. It does not know what a Level AA contrast ratio is. What Assrt does is run real Playwright tests against a real Chromium browser, where every click and type targets an element pulled from the live accessibility tree via a snapshot taken right before that step. The rule is in SYSTEM_PROMPT at /Users/matthewdi/assrt-mcp/src/core/agent.ts lines 206-218: ALWAYS call snapshot FIRST to get the accessibility tree with element refs. If the accessibility tree on your page is missing a label, the agent will pick the wrong element or fail to find one at all. That is a different signal than a scanner gives you, and that is the half of WCAG coverage nobody else is selling.

If you only buy one tool, buy something axe-core based: axe DevTools is the commercial wrapper, Pa11y is the free CLI, Storybook ships axe in @storybook/test-runner, GitHub Actions has the deque-systems/axe-core-action. They all use the same engine and catch the same approximately 57 percent of WCAG issues (Deque's 2021 study, link in the proof banner below). For the rest, you need a process: per-page manual review for keyboard traps and focus order, a screen reader check on the critical paths, and runtime tests that exercise your modals, dialogs, async loads, and route transitions. Assrt fits the runtime-tests slot. It is not a substitute for the scanner; it is the layer above the scanner.

Two reasons. The first is structural: many WCAG criteria require human judgment to evaluate, like 'is this alt text actually meaningful' or 'does this focus order match the visual flow.' A scanner can flag a missing alt attribute but cannot grade a present one. The second is temporal: a static scan happens at one moment in the page lifecycle, usually after load. It does not click anything. It does not open the modal that traps focus. It does not submit the form that triggers the loading spinner that should announce itself via aria-live but does not. Deque's 57 percent number is measured over 13,000 pages and roughly 300,000 issues, weighted by volume rather than success criterion. That weighting is what most people miss when they argue automated coverage is closer to 30 percent. Both numbers are right; they are measuring different things.

By depending on the accessibility tree to drive the test. Assrt's browser_snapshot tool at /Users/matthewdi/assrt-mcp/src/core/browser.ts line 589 calls into the @playwright/mcp official server, which returns the computed accessibility tree, the same tree a screen reader consumes. If a modal opens and traps focus on a hidden element, the next snapshot has no clickable ref for the dialog primary action, and the test fails on that step. If a button loses its accessible name during a refactor (someone wraps it in a styled div that swallows the role), the agent cannot find it by intent and reports the failure with the snapshot as evidence. If a form posts and the success message arrives via aria-live but the markup is broken, the agent's wait_for_stable returns and the assertion against the success heading fails. None of these failures need WCAG-specific logic in the test framework. They fall out for free because the test is reading the same surface the assistive technology reads.

Yes, and that is the obvious path if you want both halves in one runner. The Assrt agent emits real Playwright instructions (it drives @playwright/mcp), and Playwright has a first-party integration at @axe-core/playwright that injects axe into any Page or Frame and returns the violations array. You write the scenario in Markdown for Assrt to execute the flow, and at the assertion step you call a Playwright helper that runs axe.analyze() and asserts the violations array is empty. The combined result is one suite that fails on flow-time accessibility breakage (the Assrt half) and on WCAG 2.2 violations on each page state visited (the axe half). That is closer to a real audit than any single SaaS gives you, and it costs whatever your CI minutes plus your LLM tokens cost.

They are accessibility overlays first and a scanner second. The scanner functionality piggybacks on the same axe-core or in-house engines as everyone else; the overlay is a separate, controversial product that injects a widget into your page to remediate issues client-side. The W3C, WebAIM, and a long list of disability advocacy groups have argued the overlay approach is harmful, and there is ongoing litigation around it. If you want to test WCAG compliance, use the scanner side; do not deploy the overlay as a fix. If you want runtime coverage on top of the scanner, that is what an accessibility-tree-driven E2E agent gives you, and it does not modify the production page in any way.

It degrades gracefully. The accessibility tree @playwright/mcp returns is the computed tree, which infers roles from HTML semantics even when ARIA is absent. A button without aria-label still appears with its visible text. A div with an onclick still appears as a generic interactive node. The agent can fall back to evaluate for arbitrary JavaScript if a page is genuinely opaque, and the SYSTEM_PROMPT documents an evaluate-based workaround for OTP inputs split into single-character fields (agent.ts line 235). What you do not get is reliability: the worse your accessibility is, the more your tests will pick the wrong element, take a wrong path, or fail on a stale ref. The fix is to improve the accessibility, which is also what your users with screen readers, switch controls, and keyboard-only navigation need. The test suite turns into a forcing function for the work that should have happened anyway.

Pa11y runs a static axe-style scan on a list of URLs you give it. It is great. It catches the 57 percent half. It does not click anything, open any modal, submit any form, or navigate any in-app state, so it cannot catch the 43 percent that lives in flows. Pa11y plus a manual QA checklist is the most common shape of an accessibility program today, and the manual checklist is the part everyone hates. Assrt automates the checklist: write the flow in Markdown, run it on every PR, fail the build when the flow can no longer be completed via the accessibility tree. Pa11y for the static scan, Assrt for the flow regression, axe-core/playwright as the shared engine between them. That is the actual recipe.

A plain Markdown file with #Case blocks. Example for a focus-after-modal-close test: '#Case 1: Closing the settings modal returns focus to the trigger\n1. Navigate to /dashboard\n2. Click the Settings button in the top nav\n3. Assert: a dialog with the heading Settings is visible\n4. Click the Close button in the dialog\n5. Assert: focus is on the Settings button.' The agent runs the case, calls snapshot between steps, and the focus assertion either finds a focused ref matching Settings or fails. The scenario file lives at /tmp/assrt/scenario.md by default, and the format is defined at /Users/matthewdi/assrt-mcp/src/core/scenario-files.ts. You can commit it to your repo next to a Playwright project, version-control it, diff it, review it. There is no proprietary YAML, no vendor dashboard, no seat-based pricing.

Substantially, for most teams. AudioEye, Siteimprove, and similar enterprise platforms price into the five to six figures per year for sites of any meaningful scale. UserWay and accessiBe are in the $490 to $4,900 per year range for the overlay plus scanner, plus the reputational and legal cost of running an overlay. axe DevTools Pro is around $40 per developer per month. Pa11y is free. Assrt is open source, self-hosted, and free beyond the Anthropic API tokens your runs consume; a typical flow run is on the order of cents. The right comparison is not Assrt vs an enterprise WCAG SaaS, it is Pa11y plus Assrt vs an enterprise WCAG SaaS. The open stack covers the same approximately 57 percent of issues as the enterprise scanner, plus the runtime flow regressions enterprise scanners do not test, at roughly two orders of magnitude less cost. The trade is that you do not get the white-glove audit and the legal posture a vendor sells; if you need those, buy them separately.