Open source testing tools comparison: the one column every matrix skips.

It means that parsing a test plan from Markdown into a list of scenarios happens on one line of source. In assrt-mcp/src/core/agent.ts at line 621, the full parser is const scenarioRegex = /(?:#?\s*(?:Scenario|Test|Case))\s*\d*[:.]\s*/gi;. The parseScenarios function uses that regex to split the input text and returns an array of { name, steps } pairs where steps is the original English prose, untouched. No AST, no schema, no visitor pattern. Anything an agent could figure out from a plain-English test description lives in the LLM, not in the parser. This is the opposite of Cypress or Playwright, where the test file is a program that runs in a dedicated runner process with describe/it/beforeEach/afterEach semantics.

It is in a different category, and that is the honest answer. Selenium, Cypress, Playwright, Puppeteer, and WebdriverIO are test runners: you write code in a supported language, the runner executes that code against a browser, the return value is pass or fail. Assrt is a test agent: you write English in a Markdown file, an LLM agent interprets each sentence at runtime and emits Playwright MCP tool calls through the @playwright/mcp server Microsoft maintains. The underlying browser driver is the same Playwright you would use directly; what differs is who writes the click-and-type sequence. In a comparison table for 2026, leaving Assrt off the list because it is 'different' is the same mistake a 2014 Selenium comparison made by leaving off Cypress because Cypress was 'just a JS test framework.'

Selenium (webdriver-based, 2004), Cypress (in-browser runner, 2017), Playwright (multi-browser, MIT-licensed by Microsoft, 2020), Puppeteer (Chrome-focused, Google, 2017), WebdriverIO (WebDriver Protocol wrapper, 2011), Nightwatch (Node-based runner, 2014), TestCafe (no WebDriver, 2016), and Assrt (Markdown-driven agent over Playwright MCP, 2025). The picks are the ones that keep showing up on every list of 'open source end-to-end testing tools' plus Assrt. Anything that is a paid-first product (QA Wolf, Momentic, Testim, Mabl) is excluded because the keyword is specifically open source.

Gherkin Given/When/Then steps are still a DSL. Every Given clause in a .feature file maps to a step definition function in JavaScript or TypeScript that you have to implement and maintain. Your BDD engineer writes 'Given the user is signed in' once, and then you write a step def that calls cy.login(user) or page.goto('/login'). The English is a fiction layered on top of code. In Assrt there is no step-definition registry. 'Sign in with a disposable email' gets passed to a Haiku agent, which calls create_temp_email, then type_text, then click, then wait_for_verification_code, all based on reading the page's accessibility tree at runtime. The maintenance cost of a step-definition library is zero, because there is no library.

A .md file. For example, tests/checkout-regression.md starts with '#Case 1: Add a product to the cart' on one line, then on the next line 'Navigate to /products/widget. Click Add to cart. Verify a toast containing Added appears within 2 seconds.' That is the full test. Commit it to git. Diff it in a pull request. Let a product manager read it and suggest an edit. Because the file is Markdown and the runner only recognizes #Case, #Scenario, or #Test as a scenario delimiter, you can interleave any other Markdown content (headings like ## Context, paragraphs of explanation, commented-out scenarios) and the parser will ignore it. The file is half documentation, half executable.

Numbered cases: covered directly, because the regex matches a digit block (\s*\d*). #Case 1 and #Case 17 are both valid; the digit is captured as part of the name and stripped. Nested cases: not a concept in Assrt. There is no describe block. If you want grouping, use filesystem folders, e.g. tests/auth/signup.md and tests/auth/signin.md. Parameterized scenarios: covered by the Test Variables feature, which interpolates {{email}} and similar placeholders into the plan text before the agent reads it. The plan remains a single Markdown file; variables travel alongside it as a separate JSON object. The absence of a test-runner tree is a feature, not a gap.

Slower per run, much lower per maintenance. A scripted Playwright test that takes 3 seconds might take 15 to 40 seconds in Assrt, because each plan step is an LLM tool-calling round trip. The default model is claude-haiku-4-5-20251001, which is cheap per call but not free. The trade is that when the signup button moves from the header to a drawer, a scripted test breaks and needs a selector update, while the Assrt plan ('Click Get started') keeps passing because the agent reads the accessibility tree at runtime. For a long-lived regression suite where selectors drift, that trade is usually worth it. For a tight inner dev loop where you rerun the same click 100 times in an hour, scripted Playwright is still faster and cheaper.

It is apples-to-oranges if you think the column 'uses an LLM' is the interesting one. It is apples-to-apples if the column you care about is 'does a human read the test file and understand what it tests without knowing the implementation.' Every open source testing tool on the list can answer 'yes' to 'can I write reliable tests with this', so that column does not separate them. The column that does separate them is readability of the artifact. Under that column, Gherkin BDD tools come closest (English-ish Given/When/Then), but they still require a step-definition layer. Assrt sits alone at the end of that axis.