Stop hand-writing the broken-test exercise. Auto-draft it, then break three Cases on purpose.

It is a take-home where the interviewer hands the candidate a small web application and a test suite that includes a handful of intentionally broken Cases. The candidate has to decide, for each red test, whether the app is broken, the test is broken, or the environment is misconfigured, fix the right thing, and hand back a green run. The exercise is popular because it mirrors a real day at the office: you inherit a repo, something is on fire, and the first skill is not coding, it is triage. The mistake most hiring managers make is hand-crafting the broken test suite from scratch, which takes half a day per role and produces an exercise nobody else on the team can reproduce or grade consistently. This page is about flipping that workflow: auto-draft the canonical plan from your running app, break a few Cases on purpose, hand every candidate the exact same artifact.

The handler lives at /Users/matthewdi/assrt-mcp/src/mcp/server.ts lines 768-862. It launches a local Playwright MCP browser, navigates to the URL you provide, then takes three sequential screenshots at scroll offsets 0, 800, and 1600 pixels. Between each scroll it pulls an accessibility-tree snapshot of the visible viewport. All three snapshot texts are concatenated and sliced to the first 8000 characters, then sent along with the three JPEG screenshots to claude-haiku-4-5-20251001 with max_tokens set to 4096. The system prompt at lines 219-236 forces the output into the #Case format with six numbered rules, including 'Each case must be SELF-CONTAINED' and 'Keep cases SHORT — 3-5 actions max per case'. The response is 5 to 8 cases you can review, prune, and ship as your canonical plan.

Three reasons. Speed: one tool call against a running URL takes about ten seconds and produces a draft plan that covers the flows a candidate would actually exercise. Consistency: the generator uses the same PLAN_SYSTEM_PROMPT for every role, so two interviewers reviewing candidates for the same app start from the same canonical plan instead of their own idiosyncratic tests. Coverage honesty: it surfaces gaps. If the plan that pops out does not cover a flow you care about, the signal is that your UI does not expose that flow at all or exposes it badly, which is useful feedback independent of the hiring exercise. None of this is magic; the generator is just a forcing function that makes you actually look at what the app affords before grading someone on it.

Three seeded-failure patterns work best because they map cleanly to the three triage buckets. For 'app bug', leave the Case intact and introduce a one-line regression in the app (e.g., drop a column from a SELECT, forget to await a Promise, flip a comparison operator). For 'flawed test', keep the app correct but mutate the Case to assert stale text, a deleted selector, or a timing that stopped holding after a recent refactor. For 'environment issue', keep both app and test correct but misconfigure something out-of-band: a missing env var, a seed script that no longer runs, a port the dev server does not bind to. Ship a README that tells the candidate 'three of the Cases in /tmp/assrt/scenario.md are red; fix the right thing in each bucket, submit a writeup'.

The entire thing is three files on disk: scenario.md (plan text), scenario.json (metadata like name and URL), and optionally a snapshot of /tmp/assrt/results/latest.json from a baseline green run. This layout is fixed at /Users/matthewdi/assrt-mcp/src/core/scenario-files.ts lines 16-20. Because scenario.md is plain Markdown with #Case blocks, you can check it into the interview repo next to the app source. Every candidate clones the same repo, runs the same dev server, and executes `npx assrt-mcp` against the same URL with the same plan. There is no dashboard to provision, no vendor account, no seat to allocate. If a candidate submits a fix, you copy their branch, run the same command, and the Pass/Fail outcome is a video recording plus an events log written under /tmp/assrt/<runId>/ rather than a subjective diff review.

Once the candidate's branch is cloned, you run a single assrt_test call against the same URL with the same scenario.md. The runner writes a video (recording.webm) plus per-step screenshots plus an events.json and an execution.log into /tmp/assrt/<runId>/. A passing submission produces a green report with three Cases listed as 'passed: true' and evidence strings showing the exact accessibility refs and text content that satisfied each assertion. The grading signal is: (a) did the plan go green end-to-end, (b) did the candidate weaken any assertion to fake a green (you diff scenario.md before vs after), and (c) did they add a regression Case after each app-bug fix. You can read the video at 5x speed via the auto-opened player and make a decision in under three minutes per candidate.

Yes on both counts. The candidate only needs Playwright MCP and Node.js to execute the plan; the LLM is only involved when they invoke assrt_diagnose or assrt_plan themselves. They can read the #Case blocks by eye, fix the code, and the runner will still execute the plan deterministically against real Chromium. Your side of the grading pipeline (assrt_test with a fixed scenario.md) does not call any LLM either; it is a scripted browser walkthrough with assertion bookkeeping. That matters if the hiring rules at your company forbid candidates from using AI assistants in take-homes. The exercise format survives that rule; only the candidate's drafting tools change.

Assrt is open source and self-hosted. There is no per-seat pricing, no per-run fee, no dashboard subscription. Your only out-of-pocket cost is LLM tokens if you use assrt_plan or assrt_diagnose, and plan generation with Haiku 4.5 at max_tokens 4096 typically comes in under a cent per call. Comparable closed vendor platforms in this space start around $7.5K per month for seat-based access to hosted test runners and proprietary test formats. For a hiring pipeline that evaluates ten candidates a month on a take-home, the realistic cost difference is roughly $7,500 versus a few dollars of Anthropic billing.

The watcher at /Users/matthewdi/assrt-mcp/src/core/scenario-files.ts lines 90-111 picks up edits, debounces for one second, and syncs the new plan text back to Firestore under the same scenario UUID. For a hiring exercise you want to disable this: set ASSRT_NO_SAVE=1 in the candidate's environment so local edits stay local and do not mutate any shared record. You can also gate on the fact that scenario IDs starting with 'local-' skip the watcher entirely. Either way, when you re-grade the candidate's branch you run against your own clean copy of scenario.md, not theirs, so a candidate cannot slip an altered plan into the grading run without you noticing the diff.

Boot the app you want candidates to work against. Run `npx assrt-mcp` and invoke assrt_plan with the dev server URL. Copy the 5 to 8 Case blocks from the JSON response into a file called scenario.md under the repo root. Pick two Cases and deliberately regress the underlying app code. Pick one Case and mutate its assertion to something stale. Commit, push, and write a one-paragraph README pointing the candidate at scenario.md and giving them the dev server boot command. End to end, this takes about fifteen minutes the first time and under five on repeat. Compared to hand-writing a broken-test exercise, that is a roughly tenfold reduction in prep time.