Compare AI testing tools by the model doing the work and the finite list of actions it can take.

Most comparison pages sort tools by price, logo count, and bullet lists of features. Those are marketing inputs, not engineering inputs. The two facts that predict what an AI testing tool can and cannot do are the underlying model (because models have different capabilities, cost profiles, and context windows) and the finite tool surface the model is allowed to call (because anything outside that surface is something the tool flat out cannot do). Almost no vendor prints either one. Assrt prints both, in a single file agent.ts, and this page reads them aloud.

The default is Claude Haiku 4.5, declared on line 9 of assrt-mcp/src/core/agent.ts as DEFAULT_ANTHROPIC_MODEL = "claude-haiku-4-5-20251001". Line 10 declares a Gemini fallback at DEFAULT_GEMINI_MODEL = "gemini-3.1-pro-preview". You can override at runtime with the --model flag or the ASSRT_MODEL environment variable. The model string is passed verbatim into this.anthropic.messages.create({ model, ... }) at line 714, so any Anthropic model ID that supports tool use will work. Haiku is the default because it is fast, cheap, and strong enough to drive an accessibility tree through tool calls.

Eighteen, defined as the TOOLS constant between lines 16 and 196 of agent.ts. In order: navigate, snapshot, click, type_text, select_option, scroll, press_key, wait, screenshot, evaluate, create_temp_email, wait_for_verification_code, check_email_inbox, assert, complete_scenario, suggest_improvement, http_request, wait_for_stable. That is the entire action surface. There is no dynamic plugin registry, no vendor-private sub-tool, no hidden reasoning action. If you can describe your test in terms of those eighteen calls, the agent can execute it. If you can't, neither can the agent, and the honest answer is to open a pull request.

You do, directly to Anthropic. The Anthropic client is initialized in the agent constructor with process.env.ANTHROPIC_API_KEY. Every test run invoices against that key at Haiku 4.5 rates. A typical ten step scenario lands at a few cents in tokens. Compared to commercial AI testing platforms that start around one thousand dollars a month and climb to seventy five hundred plus at enterprise tier, that is one to three orders of magnitude cheaper, and you get an itemized Anthropic invoice instead of a flat SaaS line.

Because marketing features are generated from the tool surface, not the other way around. Every bullet a vendor prints about 'auto heals broken selectors' or 'tests OTP flows' is the output of a particular tool doing a particular thing. If you know the tool surface, you can predict which features are real and which are theatrical. If you do not know the tool surface, a plausible sounding feature list can hide enormous gaps. Assrt's assertion that tests survive UI changes is not a feature, it is a consequence of snapshot returning fresh refs on every call. Someone else's 'self healing' claim might be the same consequence, or might be an LLM guessing at a new CSS selector after the old one breaks. Reading the tool surface tells you which.

Every tool call in that list of eighteen maps to a real Playwright action. click calls page.getByRole(...).click() against the ref returned from snapshot. type_text calls page.getByRole(...).fill(). evaluate calls page.evaluate(). wait_for_stable runs a MutationObserver via page.evaluate with a polling loop in TypeScript around it. There is no intermediate DSL that compiles to Playwright at runtime. The agent drives Playwright directly through the @playwright/mcp bridge. If you want to see exactly what runs, open src/core/agent.ts and src/core/browser.ts in the assrt-mcp repo, the full path from model tool call to Playwright API is about two files and a few hundred lines.

The http_request tool. It accepts a URL, method, headers, and body, runs with a 30 second AbortController timeout, and returns the status and up to 4000 characters of the response body. In a test, the agent signs up, triggers the action that should post to Slack, then fires http_request against the Slack web API or a webhook verification endpoint and makes an assert call against the response. That promotes the test from a DOM-only checker to an end to end integration verifier. Same pattern works for Stripe webhook confirmations, Telegram bot deliveries, and GitHub comment posts.

It is defined immediately after the TOOLS array, also in agent.ts, about sixty lines. It tells the model to always call snapshot first, to use ref identifiers over text matching, to call snapshot again after any action fails, and to prefer wait_for_stable over fixed sleeps when the page is still rendering. It also contains a few dozen lines of very specific edge case guidance, including a literal JavaScript expression the model pastes verbatim into evaluate() when it encounters split-digit OTP inputs. Because it is in the source, you can read it, disagree with it, and submit changes.

The MCP server is a plain Node.js process you run locally. It does not phone home. The only outbound call during a test is to api.anthropic.com (or generativelanguage.googleapis.com if you pick Gemini) to talk to the model. Browser automation runs on your machine through a local Playwright, so the agent can hit localhost URLs your dev server is serving without tunnels, VPNs, or public exposure. Test results and scenarios sit on your disk under /tmp/assrt. There is also an optional cloud sync to Firestore for teams that want a shared dashboard, controlled by an environment flag and off by default.

The MCP package is MIT licensed and available at npm install assrt-mcp. The full source, including agent.ts with all the line numbers referenced above, is at github.com/assrt-ai/assrt-mcp. Your only running cost is the Anthropic invoice for Haiku tokens consumed during test runs, which in practice is a rounding error next to any team's existing LLM spend. There is no seat fee, no build quota, no flaky test surcharge, no enterprise tier.