navigator.onLine, setOffline, and Playwright: the two-step pattern every offline test actually needs
context.setOffline(true) blocks the network. It does not, on its own, flip the value your UI cares about: the offline event your component registered with window.addEventListener. That gap is why so many offline tests pass while the real offline UI silently does nothing. Here is the full recipe, plus the exact 4-line snippet our agent runs.
Why setOffline alone is incomplete
The Playwright API for offline emulation is browserContext.setOffline(true). It flips a flag inside the browser context that aborts any new fetch, blocks new WebSocket negotiations, and prevents service workers from reaching the network. It is a network-layer simulation. The first time you use it, the obvious test is to call page.goto and assert it throws.
That assertion is a layer below where most offline UIs live. Real apps render an offline banner, disable a Send button, queue mutations to flush on reconnect, or show a stale-data badge. The code that drives those UIs almost always reads the same two browser primitives: the navigator.onLine property and the window.addEventListener('offline', handler) listener. Neither of those is reliably toggled by setOffline alone.
What the existing playbooks teach vs. what real offline tests need
// What the existing playbooks teach.
// Looks complete. Misses half the surface area.
import { test, expect } from '@playwright/test';
test('offline banner shows when network drops', async ({ context, page }) => {
await page.goto('https://app.example.com/');
// Block all new HTTP and new WebSocket negotiations.
await context.setOffline(true);
// Try to load something. The fetch fails. We assert that.
await expect(page.getByText('Network error')).toBeVisible();
});
// Problems that stay invisible:
// - The window 'offline' event may not have fired,
// so the banner key'd off addEventListener never appears.
// - navigator.onLine may still read 'true' if a closure
// captured it on first paint.
// - Already-open WebSocket connections still flow.
// - Service worker keeps serving cached responses,
// the 'offline' UI never trips.“Tests that exercise the network simulation but skip the offline event silently pass while the offline UI does nothing.”
Playwright tools Assrt's agent calls — none of them is setOffline
The four-step recipe
Two for going offline, two for coming back. Pair the network simulation with a JS-side override and a manual event dispatch on each transition. That is the whole pattern.
Block the network with context.setOffline(true)
This is the part every guide covers. setOffline is a per-context flag that aborts new HTTP requests, blocks new WebSocket negotiations, and prevents service workers from reaching the network. It does not affect already-open sockets, and it does not reliably fire the 'offline' Event your UI listens to. It is necessary, not sufficient.
Override navigator.onLine before first paint
Use page.addInitScript or context.addInitScript with Object.defineProperty(window.navigator, 'onLine', { configurable: true, get: () => false }). This guarantees that every document the context creates, including iframes and reload navigations, sees onLine as false from the very first script. Without this, any closure that read navigator.onLine on first paint will keep returning the original value forever.
Dispatch the standard offline Event from window
page.evaluate(() => window.dispatchEvent(new Event('offline'))) is what makes the listener your component registered with window.addEventListener('offline', ...) actually run. The event is parameterless. Do not dispatch from document or document.body, the standard listener is on window. This is the line that flips your offline banner from invisible to visible.
Assert the offline UI, then reverse all three for online
Assert on the visible state, not on internal flags. Banner shown, action buttons disabled, queue counter incrementing. Then run the inverse for back-online: setOffline(false), redefine the getter to return true, dispatch new Event('online'). Assert the banner clears, queued actions flush, and the badge resets. The full round trip, in one test.
The four lines of JS the Assrt agent runs
When a scenario step says "simulate going offline," the agent picks the evaluate tool from its 18-tool palette and forwards an expression to Playwright MCP's browser_evaluate. The wiring is /Users/matthewdi/assrt-mcp/src/core/browser.ts:665-670. That is the entire bridge between "go offline" in plain English and a real property-descriptor swap on window.navigator. No proprietary DSL, no YAML, no recorded session to replay.
How the call flows through Assrt
One scenario step maps to one tool call. The agent is the hub; on the left are the inputs that decide which tool to pick; on the right are the side effects in the real browser.
evaluate is the sole pathway from 'go offline' to a real DOM event
Why patch via addInitScript and not page.evaluate
Most apps read navigator.onLine once on first paint inside a framework hook (useState(navigator.onLine), $: online = navigator.onLine, or similar) and then rely on the online and offline events to update state. If you call page.evaluate to override the property after navigation, the framework has already cached the original value, and your override is invisible to the component. Only addInitScript guarantees the override runs before any other script in every new document, including iframes and full reloads.
The same test, written for the Assrt agent
You do not need the agent to use the recipe. The four-line JS snippet works in any hand-written Playwright spec. But if you want plain-English scenarios that recompute their locators on every run, here is what the same test looks like as a scenario.md file, two cases, ten bullet steps total.
What it looks like when the run goes through
One evaluate call per transition, one snapshot to refresh the accessibility tree, one assert per visible behavior. Two cases, eleven steps, four assertions, finished in roughly five seconds wall-clock.
The tools the agent uses for an offline test
Six of the eighteen tools defined at agent.ts:16-196. The whole offline round trip uses fewer than a third of them.
browser_evaluate
The Playwright MCP tool that runs arbitrary JS in the page. Defined at /Users/matthewdi/assrt-mcp/src/core/browser.ts:665. The agent passes a closure (`() => (...)` or `() => { ... }`) and gets back the return value as a string. This is the single tool that turns the four-line offline snippet into a one-step scenario action.
evaluate (agent-level)
The 8-line tool definition the LLM sees. Lives at /Users/matthewdi/assrt-mcp/src/core/agent.ts:106-113. Just one parameter: { expression: string }. The agent picks it whenever a scenario step asks to flip JS state, paste OTPs across multiple inputs, or, in this case, simulate offline.
snapshot
Calls Playwright MCP's browser_snapshot to get an accessibility tree of the page. Used right after the evaluate call to confirm the offline banner is now visible and to grab a fresh ref for the assert step.
wait_for_stable
MutationObserver-based stability detector. Source: agent.ts:956-1005. Useful between offline and back-online for apps that batch reconnect logic in a debounced effect.
assert
Records pass/fail with description, passed boolean, and evidence string. The agent calls it once per visible behavior: banner visible, button disabled, queue badge equals zero. Failures land in the WebM with their evidence, so review is one scrub away.
complete_scenario
Marks the case done with a one-line summary and an overall pass/fail. The runner uses this to write /tmp/assrt/results/latest.json and to short-circuit downstream reporting.
Service workers and WebSockets
Two known gaps that the existing pages either bury or skip. Issue microsoft/playwright#2311 tracks setOffline not affecting service workers, which can keep serving cached responses long after the network simulation has gone offline. Issue #4910 tracks the same gap for already-open WebSocket connections, which continue to send and receive frames. The navigator.onLine override does not fix either, but combined with page.route('**/*', r => r.abort()) and an explicit ws.close() stub, you can get within a few percent of a real disconnect. The point of this page is the JS-state half, which is the half almost everyone misses.
What changes when an LLM agent drives the test
The recipe is identical. The difference is what you maintain. In a hand-written spec, the test file is the source of truth: locators, evaluate snippets, assertions. When the offline banner gets renamed from You are offline to Connection lost, the spec breaks and someone has to update it. With Assrt, the scenario file just says "assert the offline banner is visible." The agent recomputes which DOM node is the banner from a fresh accessibility tree on every run, so the banner copy can change without a single edit to the scenario.
The evaluate step itself stays as plain JS in the scenario, because there is nothing about navigator.onLine or the offline event that requires reasoning, only execution. The agent forwards it verbatim to browser_evaluate and reports back what the page looked like before and after.
Want this recipe wired into your test suite?
Show us your app and we'll wire up an offline-and-back-online scenario that runs in CI on every PR.
Frequently asked questions
Does context.setOffline(true) automatically fire the window 'offline' event in Playwright?
Not reliably. context.setOffline(true) flips the network simulation flag in the browser context so any new fetch, XHR, or WebSocket negotiation fails, and it sets navigator.onLine to false on the next read. It does not consistently dispatch the 'offline' Event object that your code added with window.addEventListener('offline', handler). The behavior differs between Chromium, Firefox, and WebKit, and it differs again across page reloads. If your UI keys its banner, sync queue, or retry button off that listener, you cannot rely on setOffline alone. The portable recipe is to call setOffline AND fire the event yourself with page.evaluate(() => window.dispatchEvent(new Event('offline'))).
Why does navigator.onLine sometimes return true even after setOffline(true)?
Two reasons. First, navigator.onLine is computed lazily; some pages cache its value in a variable on first paint and never read it again, so flipping the underlying flag mid-test does nothing visible. Second, if you reload the page after calling setOffline(true), the new document boots with whatever navigator.onLine returns at parse time, which can race with the offline flag in a way that surfaces 'true' for the first frame. The fix is to override the property itself with addInitScript, so every document the context loads sees navigator.onLine as a getter that returns the value you control.
What is the minimal four-line JavaScript snippet that turns navigator.onLine off and notifies listeners?
Object.defineProperty(window.navigator, 'onLine', { configurable: true, get: () => false }); window.dispatchEvent(new Event('offline')); That is it. Two statements: redefine the getter to return false, then synchronously dispatch the offline event. Reverse it for back-online: redefine the getter to return true, dispatch new Event('online'). This is exactly what Assrt's agent runs through Playwright MCP's browser_evaluate tool when a scenario step says 'go offline.' Source: the evaluate handler at /Users/matthewdi/assrt-mcp/src/core/agent.ts lines 843 to 845, which forwards to browser.evaluate at /Users/matthewdi/assrt-mcp/src/core/browser.ts lines 665 to 670.
Does setOffline(true) work with service workers and WebSockets?
Partially. The Microsoft Playwright issue tracker has long-running threads (#2311 for service workers, #4910 for WebSockets) that document the gaps. setOffline blocks new HTTP requests and prevents new WebSocket negotiations, but already-open WebSocket connections continue to send and receive frames, and service workers can still serve cached responses without ever touching the network simulation. If your offline test is verifying that a chat reconnects after a drop, you usually need to combine setOffline with route('**/*', r => r.abort()) and an explicit ws.close() simulation. The 'real offline' you are testing is almost never the same as the 'real offline' setOffline simulates.
Why patch navigator.onLine via addInitScript instead of just calling page.evaluate after navigation?
page.evaluate runs after the document has loaded, which means any inline script, framework bootstrap, or service worker registration that read navigator.onLine on first paint already saw 'true' and cached it. By the time your evaluate runs, the offline banner has already been suppressed. addInitScript runs before any other script in every new document the context creates, including iframes and pages opened via window.open. That guarantees the override is in place before the first read, and it survives full reloads. Pair addInitScript('navigator.onLine = false') with setOffline(true) for both the network simulation and the JS state, and you cover both sides.
How do I test the back-online flow specifically? Re-enabling setOffline is not enough.
Three steps. First, await context.setOffline(false) to lift the network block so any retry the app makes can actually reach your server. Second, page.evaluate(() => window.dispatchEvent(new Event('online'))) to fire the event your UI listens to so the banner dismisses, the sync queue drains, and the optimistic-UI reconciliation kicks in. Third, assert on whatever your UI is supposed to do after coming back: a 'Synced' toast, a count of pending mutations dropping to zero, an empty offline-queue badge. Skip step two and your test will look offline-still even though the network is restored, because nothing told your code the state changed.
Does Assrt's MCP server expose setOffline directly?
No, deliberately. Assrt routes the agent through Playwright MCP's stdio interface and exposes 14 underlying tools to the LLM: browser_snapshot, browser_navigate, browser_click, browser_type, browser_select_option, browser_resize, browser_take_screenshot, browser_start_video, browser_stop_video, browser_press_key, browser_scroll, browser_wait_for, browser_evaluate, and browser_close. setOffline is intentionally not in that list, because the JS-side override is portable, observable, and works for both the network and event sides at once. When a scenario says 'simulate going offline,' the agent runs Object.defineProperty plus window.dispatchEvent through browser_evaluate, and the test exercises exactly what a real-world offline event would. The wiring lives at /Users/matthewdi/assrt-mcp/src/core/browser.ts:665-670 and is called via the evaluate tool defined at /Users/matthewdi/assrt-mcp/src/core/agent.ts:106-113.
Will Object.defineProperty on navigator.onLine work in WebKit and Firefox?
Yes, with one caveat. In all three engines (Chromium, Firefox, WebKit) navigator.onLine is configurable, so Object.defineProperty(window.navigator, 'onLine', { configurable: true, get: () => false }) replaces the getter cleanly. The caveat is WebKit's stricter same-origin handling for cross-frame access; if your app reads navigator.onLine from inside an iframe, you have to repeat the override on every frame. addInitScript handles that automatically because it injects into every new document the context creates.
Is dispatching new Event('offline') enough, or do I need a CustomEvent with detail?
new Event('offline') is enough for the standard window.addEventListener('offline', handler) pattern, which is what 99% of apps use. The native event is parameterless, no detail payload, no useful properties beyond .type. If your code somewhere reads e.detail and treats absence as a signal, switch to new CustomEvent('offline', { detail: { synthetic: true } }). Either way, do not bubble it from document or document.body; the standard listener is on window, and dispatching from elsewhere will silently miss it.
How does this translate to an Assrt scenario, in plain English?
You write the scenario as plain Markdown in /tmp/assrt/scenario.md with #Case blocks. Example: '#Case 1: offline banner appears after disconnect' with bullets like 'Open the app', 'Run evaluate to override navigator.onLine and dispatch offline', 'Snapshot and assert the offline banner is visible'. The agent reads each step, picks the right tool from its 18-tool palette, and executes. For step 2 it picks evaluate, passes the four-line JS snippet, and continues. No spec file to maintain, no locator updates when the banner copy changes, and the recipe stays portable: the same evaluate snippet works in any hand-written Playwright spec.
Open source, MIT licensed. 0 lines of JavaScript are all that stand between an offline test that passes for the wrong reasons and one that exercises the real UI.
Comments (••)
Leave a comment to see what others are saying.Public and anonymous. No signup.