Self-healing test maintenance hours, the math nobody publishes

Where the 42 hours actually go

The 42-hour number is a composite, not a survey response from one source. Webomates publishes a 80%-of-test-cost-is-maintenance figure. QA Flow on Medium puts it at 50% of total QA time. Mabl, Virtuoso, and Testim quote case studies in the 200 hour-per-quarter and 40 hour-per-month range. They roughly agree on the bucket sizes.

The line items are not equally addressable. The bottom one (genuine product evolution) is genuine work. No automation strategy returns that hour and pretending otherwise is dishonest. The other three are the structural budget self-healing tries to attack, and which a no-stored-locator architecture removes outright.

The 34 hours is the bucket Mabl, Virtuoso, Functionize, and Testsigma all aim at. The disagreement is over what fraction comes back. The unspoken question is what the bucket would look like if the model underneath did not store a locator at all.

Two architectures, two hour curves

The toggle below shows the same monthly maintenance budget under two different architectures, against the same fictional 200-case suite. Numbers in the "repair pipeline" column are the published best case from the SaaS side. Numbers in the "no locator stored" column are derived from the actual snapshot-per-action loop the Assrt agent runs.

The seven words that erase the bucket

The mechanism is three lines of system prompt and one error-recovery path. It is not magic and it is not a model upgrade. It is a choice about where the locator gets resolved. In a stored-locator world, resolution happens at authoring time and the storage rots. In a snapshot-first world, resolution happens at the moment the action fires and there is nothing to rot.

The seven words that do the work are ALWAYS call snapshot FIRST to get the accessibility tree. The rest of the rules block enforces that this happens every action and that stale refs trigger a fresh snapshot rather than a retry against a stale one. The agent never accumulates a cache of selectors. Every interaction is resolved against a tree that did not exist a tick ago.

The recovery path is the normal path

Most self-healing products structure recovery as a separate pipeline: action fails, fault detection fires, a model is called to propose a new locator, the proposal is logged, a human reviews it. Each of those steps has a cost line. Assrt collapses recovery into the normal control flow at agent.ts line 933. When an action fails, the runner inlines the first 2,000 characters of a fresh accessibility tree directly into the failed-tool result that the model sees on its next turn. The model does not get told "a heal happened". It gets told the action failed and here is the page right now. Continue.

The 2,000-char slice is the entire heal protocol. There is no confidence score, no proposed-locator audit trail, no separate model call. The next turn either succeeds against the new tree, or fails again, in which case the agent re-snapshots, tries a different approach, and after roughly three failed attempts marks the case as failed for human review. That last hour is real and honest. It is the only hour the snapshot-first architecture spends.

What disappears from your weekly maintenance loop

Concrete tasks, not abstractions. If you have run a Playwright suite past 100 cases, you have done some version of every line below inside the last 90 days. Every one of these is a budget item that does not exist in the no-locator-stored model.

Wait timing is the other big bucket and it is also structural

If your team spends fewer than 4 hours a month on timing flakes, either the suite is small or someone has done careful work on waits. Most do not. The standard pattern is await page.waitForTimeout(5000) which makes the test slow on fast pages and still flaky on slow ones, or await expect(...).toBeVisible({ timeout: 10000 }) which is better but still a fixed ceiling that you tune by superstition.

The wait_for_stable tool at agent.ts:872-925 measures actual DOM quiet rather than wall-clock time. It injects a MutationObserver on document.body, polls every 500ms, and exits the moment 2 seconds pass with zero new childList, subtree, or characterData mutations. The 30-second ceiling is there to prevent infinite loops on a page that never settles, not as the typical wait. A typical wait on a Next.js dashboard with a streaming widget is under 2 seconds plus the 2-second quiet window. A typical wait on a static page is under 500ms. You do not write any number into your scenario; the page tells the test how long to wait.

The honest residual

A page that claims zero maintenance hours would be lying, so here is what stays. About one hour per month goes into env and data drift, the same as any approach: a seeded user expires, a staging DB resets, a third-party API contract changes. About one more hour goes into genuine product evolution, where a flow your scenario describes no longer exists in the product. You read the failure, rewrite the #Case block, ship. assrt_diagnose will emit a corrected #Case block in Markdown if you want a head start, but the rewrite itself is the actual work and there is no way to automate that without telling the test what the new flow is. Total residual on a mature suite, two hours a month per QA. Compare 6 to 11 hours residual on the SaaS-self-healing path.

The other thing that goes away is the per-seat subscription cost that closed AI QA platforms charge. Comparable products price in the $7,500 per month per seat range for the bundle that includes AI execution. The runner under @m13v/assrt is open source and self-hosted; the bill is whatever Anthropic charges for the model calls your runs consume, and you can swap to a local proxy via the ANTHROPIC_BASE_URL environment variable if you want the bill at zero.

How to verify the math on your own suite this week

Before you trust any vendor (this one included) on hour-savings claims, measure your own baseline. Three commands plus an afternoon of attention is enough to know whether your team sits at 10 hours a month or 50.

1. Run git log --oneline --since="90 days ago" -- tests/ and grep for "flaky", "selector", "locator", "testid", "timing". Multiply the count by 20 minutes (the rough cost of a fix including review). That is your floor.
2. In CI, query for tests that failed and went green on retry within the same week without a code change to the test file. That is stale-selector or timing flake mass. Multiply by 15 minutes per investigation.
3. For one week, ask every engineer who touches a test file to tag the commit either "repair" or "extend". Sum repair hours. Multiply by 4 to project monthly. This usually matches vendor case studies within 10-20%.

If your team comes in under 10 hours a month, you have a stable suite or a very small one and any self-healing product (this one included) will be marginal. If you come in over 30 hours a month you have the bucket the whole category aims at, and the question becomes whether you want a 95% repair pipeline or no pipeline at all.