Pause and takeover during AI test runs, the part most cloud runners skip

Why the model is take-over-after, not pause-mid-step

A lot of the people who land on this topic want true mid-step pause: freeze the agent on step 3, click into the browser, do a thing, then let the agent continue from step 4. It is the obvious feature. It is also the feature that breaks if you build it badly.

The reason it breaks is that the AI agent is holding the CDP session. Every click, type, and assertion the agent issues is a message on the same Chrome DevTools Protocol channel that a human taking over would use. Splitting that channel between two controllers turns into a racy mess: half-typed inputs, double clicks, a captured screenshot that the agent thinks it took but the human moved the page before the response landed. The honest version of mid-step pause requires a transactional handoff protocol, and no open source runner I have seen ships one that actually works under load.

Assrt makes the smaller, more defensible choice. The agent runs to completion or to a stop signal. When it lets go, the keyboard and mouse are yours. That is take-over-after. It covers ninety percent of why people ask for pause-and-takeover (the agent failed and I want to finish the flow), without inventing a handoff protocol that loses inputs.

The one comment that makes this possible

The whole feature hangs off a single design decision in the agent loop: when a run finalises, do not destroy the VM. In src/core/agent.ts the comment on finalizeVideo() says it out loud:

/** Encode and download the screencast recording from the VM.
 *  Does NOT destroy the VM (it stays alive for user takeover). */

Most cloud E2E products release the VM immediately. The reasoning is economic, every running VM costs money, and the user already has the video. The Assrt design rejects that trade. The VM is the most expensive single artifact of the run; throwing it away in the same second a developer wants to debug it is the worst possible time.

Three things release the VM instead: an explicit Stop click from the user, a pagehide beacon when the user closes the tab, or a server-side reaper that kills idle sessions after a timeout. Until one of those fires, you can come back to the tab an hour later and the browser is still sitting on the failure page.

The handoff, step by step

From the moment the agent reaches appState === "done" to the moment your mouse moves are five small things in sequence. Each one is a few lines of code; none of them is magic.

Two channels, two phases

Drawn as a sequence diagram, the takeover handoff is a small dance between three actors: your browser tab, the input WebSocket, and the noVNC canvas. The thing to notice is that the screencast channel never disconnects. It is the same pixel stream throughout, which is why the picture does not flicker when you take over.

The input WebSocket carries one extra job. While you are in control, an Inspect button is visible. Click it and the same WebSocket carries a single JSON message ( { type: "highlight", action: "inspect", enabled: true }) that switches CDP into element-picker mode. Hover the canvas, watch the element under your cursor highlight, click to capture its selector. That selector flows back to the React app as an inspectNode event, which dispatches a custom DOM event the rest of the UI listens for.

What changes between watch-only and interactive

The same JSX element renders the browser in both phases. The flip is one prop, no reconnect.

// src/app/app/test/page.tsx
<VncViewer
  url={vncUrl}
  viewOnly={true}
/>
// rfb.viewOnly = true; rfb.showDotCursor = false

The six-line useEffect inside the viewer that actually does the flip:

useEffect(() => {
  if (rfbRef.current) {
    rfbRef.current.viewOnly = viewOnly;
    rfbRef.current.showDotCursor = !viewOnly;
  }
}, [viewOnly]);

That is it. The RFB instance is alive, the WebSocket to the VNC server is open, the pixel stream is still flowing. Toggling rfb.viewOnly on the existing instance is what unblocks input forwarding from noVNC down to the VM. The cursor dot appears so you can see where your pointer is on the remote display.

What this gets you that a recording does not

Every E2E test product gives you a video and a screenshot. The mental model people pick up from that is "watch the failure, then go reproduce it on my laptop." That mental model breaks on three flows that come up constantly.

How this compares to other AI test runners

Most of the AI test products you can buy run scenarios on a managed fleet of cloud VMs, then destroy the VM as soon as the run finishes. That makes sense from a unit-economics angle and is fine if the only thing you ever do with a run is look at the report. The moment you want to touch what the agent touched, it falls apart.

When this is the wrong feature for you

Honesty section. Take-over-after is not the right shape for every team. Three cases where it does the wrong thing:

• You need true frame-accurate pause inside step execution so a human can patch the agent's next action and let it continue. That requires a handoff protocol Assrt does not ship today, and anyone telling you their tool does ship it should be asked to demo it under load.
• Your CI does not run on a VM that can run a VNC server, so a head-up canvas is not reachable from a developer's laptop. In that environment the same scenario should run locally first and only flow to CI as a smoke job.
• Your test data is so sensitive that letting a developer drive the test browser after a run would violate the same access policies that gate production access. The takeover is exactly as powerful as the test account that ran the scenario; if that is too powerful, lock it down before opening the canvas.

Try it on your own app

The whole pattern is open source and runs from one command. Point it at a URL, watch the agent crawl and write scenarios, run a scenario, then click Take Over on the result.

npx @m13v/assrt discover https://your-app.com

The CLI is the same surface used by the Model Context Protocol integration, so if you are running this from inside Claude Code or Cursor, the same tools (assrt_test, assrt_plan, assrt_diagnose) drive the same VM lifecycle. The Take Over overlay shows up on the same browser view the agent was just on.

Pause and takeover, the questions that come up