Automated open source testing: when the runner itself is a forkable file

Playwright, Selenium, and Cypress are browser automation frameworks. They are open source runners that execute tests you write by hand in code: locator strings, waits, assertions, the whole thing. Assrt sits one layer above that. The plan you write is English prose (#Case blocks in /tmp/assrt/scenario.md), and an MIT-licensed LLM agent translates each sentence into real @playwright/mcp calls at runtime. In other words: Playwright is what the agent uses; Assrt is the agent. Both are open source, but they solve different halves of the problem. If you already love Playwright, you can keep using it as the driver and move only your intent layer into prose.

Literally that. /Users/you/assrt-mcp/src/core/agent.ts is 1,087 lines of TypeScript. It contains the TOOLS array (lines 16-196, 18 tools), the system prompt, the provider dispatch (Anthropic on line 9, Gemini on line 10), the conversation turn loop, and the try/catch error recovery that re-snapshots on any failure. If you want to audit what the LLM can do to your browser, you do not have to read a server repo, a worker repo, and a dashboard repo. You read one file. That is what MIT + self-contained actually buys you compared to a closed platform.

No. keychain.ts lines 33-38 check for ANTHROPIC_API_KEY in env before any other auth path. If it is set, the runner uses it directly and makes zero calls to any Assrt endpoint. scenario-files.ts:94 explicitly skips Firestore sync for any scenario whose ID starts with local-. If you pass --isolated and use a local- scenario, the entire data path is: your disk, your LLM provider, the target URL, back to your disk. app.assrt.ai exists for teams that want shareable run URLs and history; it is never on the hot path for a test to pass.

Yes, via --model. Line 9 of agent.ts defines DEFAULT_ANTHROPIC_MODEL = 'claude-haiku-4-5-20251001'. Line 10 defines DEFAULT_GEMINI_MODEL = 'gemini-3.1-pro-preview'. Passing --model gemini-3.1-pro-preview (or any other Gemini model ID) routes the conversation through @google/genai instead of @anthropic-ai/sdk. Adding a new provider is a pull request, not a ticket: implement the tool-call loop in core/agent.ts for your provider, point the --model flag at it, and ship. The dispatch table is right there in the file.

Closed vendors typically give you a visual recorder that emits a proprietary recording file (YAML, JSON, or a custom binary). Play it back inside the vendor runner and it works; try to run it anywhere else and it does not. Assrt's runtime artifact is different. The scenario is a markdown file, and every browser action the agent takes is a regular @playwright/mcp tool call (navigate, browser_snapshot, browser_click). If you swap Assrt for a raw Playwright MCP loop you drive yourself, the same underlying tool calls still work. The plan is prose, and the calls below it are standard Playwright. Nothing in your on-disk artifact is vendor-specific.

assrt-mcp is MIT (see LICENSE lines 1-3, Copyright 2026 Assrt). Its only browser dependency is @playwright/mcp ^0.0.70 from the Playwright team (Apache 2.0). The agent pulls from @anthropic-ai/sdk ^0.39.0 (MIT) and @google/genai ^1.46.0 (Apache 2.0). The MCP framework dependency is @modelcontextprotocol/sdk ^1.29.0 (MIT). Every runtime dependency is either MIT or Apache 2.0. There is no AGPL component, no 'source available' license, no 'open core with a closed enterprise feature' split.

Three environment pieces: ANTHROPIC_API_KEY (or GEMINI_API_KEY), a checked-in /tmp/assrt/scenario.md (or passed via --plan-file), and --isolated to keep the browser profile in memory. The CLI supports --json which writes a structured TestReport to stdout. Pipe that to your CI artifact store. The cloud-sync code path is scenario-files.ts, and it only activates if the scenario ID is not prefixed with local-; if you are running --plan-file directly, the local-only path is what kicks in. The only outbound network call is to your LLM provider for tool-use inference. Everything else is your own infra.

Add an entry to the TOOLS array in src/core/agent.ts (anywhere between lines 16 and 196 is fine; order does not matter). Each tool is a plain object with name, description, and input_schema. Then add a case to the tool dispatch switch further down in the same file to implement it. Rebuild with npm run build, reinstall from your fork, and the LLM can now call your tool by name. That is the whole extension model: no plugin API, no vendor approval, no marketplace review. Changes to your fork take about as long as adding a function to any TypeScript file.

You keep running. The MIT license grants irrevocable rights to use, modify, and redistribute. The runtime artifacts are a markdown plan file and webm videos you already own. The only external service in the hot path is your LLM provider, which you picked. If the Assrt org disappears from GitHub, your existing clone still works and you can fork it. Contrast with a closed SaaS: when the vendor disappears, so do your recordings, your dashboards, and your test runs. The 'open source' part of automated open source testing is insurance against exactly this.

Not here. The runtime is real @playwright/mcp (the same engine closed vendors build on top of), and the model can be Claude Haiku or Gemini Pro, both frontier-tier for this workload. The difference between Assrt and a $7,500/month closed platform is not browser engine quality. It is packaging: onboarding dashboards, screenshot libraries, video sharing, managed test runners. You can add those yourself on top of the MIT core, or accept that your test artifacts are raw files and move on. Quality of the underlying automation is bounded by Playwright and the LLM, not by the license.