context.setOffline flips navigator.onLine, but the 'offline' event never fires
Every article about testing offline behaviour in Playwright ends the same way: call await context.setOffline(true), move on. That advice hides a gap most apps fall through. navigator.onLine does flip to false, but window.addEventListener('offline') is never called.
If your app subscribes to the event to flip a banner, drain a queue, or pause a WebSocket retry, your test silently passes and proves nothing. This page walks through why, where the gap lives inside Playwright's source, and the three-line fix that closes it for every test in your suite.
“The entire offline implementation in Playwright's Chromium backend is a single CDP call: session.send('Network.emulateNetworkConditions', { offline: true, latency: 0, downloadThroughput: -1, uploadThroughput: -1 }). No window.dispatchEvent. No navigator patching. No injected script. File: playwright-core/lib/server/chromium/crNetworkManager.js, lines 131 to 137.”
node_modules/playwright-core, read directly
The failure mode: a test that passes because nothing happened
Here is the common pattern. A React app shows an offline banner driven by a hook that subscribes once on mount:
useEffect(() => { const on = () => setOffline(false); const off = () => setOffline(true); window.addEventListener('online', on); window.addEventListener('offline', off); return () => {...}; }, []);
That hook never reads navigator.onLine again after mount. It is waiting for the browser to tell it. When the test below runs, Playwright flips the flag, the app never gets the notification, and the offline banner stays hidden forever. Engineers then add a reload to the test, the mount-time read picks up the flipped value, and the test goes green. Production users never reload when their wifi dies, so the bug ships anyway.
Broken test versus the fixed version
Left tab: the test most articles show you, the one that either flakes or silently passes depending on whether the app reads navigator.onLine on mount. Right tab: the same test with a three-line page.evaluate that dispatches the missing event. It's small, boring, and it works.
OFFLINE TEST
import { test, expect } from "@playwright/test";
test("shows offline banner when network drops", async ({ page, context }) => {
await page.goto("/");
// This flips navigator.onLine to false...
await context.setOffline(true);
// ...but NEVER fires the 'offline' event.
// If the app registers:
// window.addEventListener('offline', () => setOnline(false))
// that handler is still waiting.
// This assertion passes because a plain reload
// now reads navigator.onLine at mount time.
// Without the reload, the banner never appears.
await expect(page.getByText("You're offline")).toBeVisible();
// In real apps that only listen for events: FLAKY / SILENT PASS.
});What Playwright actually does under the hood
Three layers. Your test calls context.setOffline(true). Playwright-core translates that into a single CDP message, Network.emulateNetworkConditions. Chromium receives it, updates its internal network state, and propagates a new value for navigator.onLine. The DOM event system sits in a different pipeline and never learns the state changed.
Your test -> Playwright -> CDP -> Chromium. The DOM event system is off-path.
The verification test you can paste in now
Don't take my word for it. Here is a 20-line Playwright test that attaches listeners before the flip, waits for them after, and asserts two things: the getter changed, the listener did not fire. Both expectations pass on Playwright 1.56 against Chromium. If the behaviour ever changes, one npm install will tell you.
Run it:
The reusable fixture so you never write this by hand again
A per-test page.evaluate call gets old fast. Put it in a fixture once. Every spec that imports from ./fixtures/offline inherits a patched context.setOffline that also dispatches the matching online or offline event on every open page. You write it once; your whole suite is fixed.
Why this gap exists, summarised in five tiles
The behaviour is not a Playwright bug. It is a consequence of CDP being a network-layer emulation, not a DOM event emulation. The five cards below are the full mental model.
The getter flips instantly
CDP's Network.emulateNetworkConditions updates the NetworkStateNotifier that backs navigator.onLine. A fresh eval reads false the moment setOffline resolves. Code that reads the getter on mount, on router change, or before a fetch sees the right value.
The event never dispatches
Chromium only fires window 'offline' and 'online' when the real OS connectivity observer changes state. CDP emulation does not route through that observer. Listeners wait forever.
The common app pattern breaks
Most React/Vue/Svelte offline hooks subscribe in a useEffect: addEventListener('offline', ...). They never read navigator.onLine again until the next event. Under test: permanently online from the app's point of view.
Reloading hides it
Because navigator.onLine IS flipped, reload() shows an offline banner that depends on the mount-time check. Engineers add a reload to their test, see the banner, ship a bug. Users do not reload when their wifi dies.
The fix is three lines
page.evaluate dispatches window.dispatchEvent(new Event('offline')). Wrap setOffline once in a fixture and every test in the suite inherits it. Cost: a dozen lines in tests/fixtures/offline.ts.
The numbers
The whole story is 0 lines of Playwright core, 0 dispatched events, and a 0-line fix. Read that again. This is not a deep architectural problem; it is a small hole that nobody documents, and a small patch closes it permanently.
How Assrt fits around this
Assrt's agent deliberately excludes network error testing. The plan-generation system prompt at /Users/matthewdi/assrt-mcp/src/mcp/server.ts:219 reads literally: “The agent can: navigate URLs, click buttons/links by text or selector, type into inputs, scroll, press keys, and make assertions. It CANNOT: resize the browser, test network errors, inspect CSS, or run JavaScript.” That line exists on purpose. Low-level browser state belongs in real Playwright code that you author, review, and own.
Because Assrt generates real Playwright tests (not a proprietary YAML DSL, not a cloud-locked editor), the offline fixture you wrote above drops into the same tests/ folder as your Assrt-driven scenarios. One repo, two kinds of tests, one npx playwright test command. No vendor lock-in, no separate dashboard to learn, no DSL to translate.
The full workflow, from English scenario to offline-event fixture
Write the scenario in Assrt Markdown first
Describe the offline UX in a #Case block in /tmp/assrt/scenario.md. 'Navigate to /. Assert the Add to cart button is visible. Then the connection drops. Assert the banner reads Youre offline.' The agent handles the parts it can.
Let assrt_plan generate the happy-path assertions
The MCP tool writes #Case blocks for everything the agent can verify: navigation, visible text, URL matches, button enabled state. It skips the network-manipulation step on purpose. That line 219 rule is not a bug; it keeps the agent honest.
Drop an offline.spec.ts next to it
In the same tests/ folder, add the Playwright file that calls context.setOffline and dispatches the event. No YAML. No vendor editor. A real .spec.ts in your repo.
Run both from one command
npx playwright test runs your offline.spec.ts in the normal Playwright runner. assrt_test runs the scenario.md via Playwright MCP. Results land in the same artifacts directory. Review is one folder.
Commit the fixture once
tests/fixtures/offline.ts lives with the repo. Every future spec that imports from './fixtures/offline' gets the patched setOffline for free. You never write window.dispatchEvent by hand again.
Edge cases this fix does not cover
The dispatch workaround handles window.addEventListener('offline') and navigator.onLine. It does not fix three related surfaces that the same emulation also misses. If your app relies on them, budget extra code.
- Service workers: tracked as microsoft/playwright#2311. In older versions
setOfflinedid not stop SW-controlled fetches. Usecontext.routeto force the service worker's fetch path to fail deterministically. - WebSockets: tracked as microsoft/playwright#4910. Open WebSocket connections keep sending and receiving frames after
setOffline(true). If your app reconnects on a ping timeout, simulate it withws.close()insidepage.evaluate. - navigator.connection.effectiveType: the NetworkInformation API is a separate emulation surface.
setOfflinedoes not flipconnection.effectiveTypeto'slow-2g'. Patch it in anaddInitScriptand dispatchconnection.dispatchEvent(new Event('change')).
Want help closing this gap across your whole test suite?
Walk through your Playwright setup with the Assrt team. We ship a working offline fixture and patched agent config on the call.
Book a call →Frequently asked questions
Does await context.setOffline(true) actually change navigator.onLine to false?
Yes. When Playwright sends Network.emulateNetworkConditions { offline: true } over the Chrome DevTools Protocol, Chromium internally flips the navigator.onLine getter to return false for every execution context in the browser context. You can verify it with await page.evaluate(() => navigator.onLine); the value is the boolean false. The call is synchronous from the page's perspective: by the time the setOffline promise resolves, a fresh script eval will read false. The subtle part is that no event is dispatched to announce the change. Code that reads navigator.onLine at a specific moment sees the new value; code that listens for the browser to tell it the value changed waits forever.
Why does window.addEventListener('offline') never fire during a Playwright test?
Because the entire offline implementation in Playwright's Chromium backend is four lines: crNetworkManager.js:131-137 calls session.send('Network.emulateNetworkConditions', { offline: true, latency: 0, downloadThroughput: -1, uploadThroughput: -1 }). That CDP command affects the network stack, not the DOM event system. Chromium does not piggyback an 'offline' CustomEvent on the state change, and Playwright does not inject one. Any application code that starts its disconnected UI inside a window.addEventListener('offline', () => setState('offline')) handler will keep waiting for a handler call that is never coming, and its banner will never render.
What is the canonical fix?
Dispatch the event yourself. The shortest version lives inside page.evaluate right after the setOffline call: await page.evaluate(() => window.dispatchEvent(new Event('offline'))). A more robust version is an addInitScript on the context that wraps context.setOffline so every invocation dispatches the matching event. The page includes a copy-paste Playwright test fixture that does this once and every test in the suite inherits the fix. The cost is three lines; the benefit is that your app's real event handler finally runs under test.
Why not override navigator.onLine with Object.defineProperty instead?
Because Playwright already does that work for you through the CDP call, and overriding the getter is actively harmful. If you defineProperty the getter before your app loads, your override wins and the browser's own offline emulation cannot update it later (setOffline(false) becomes a no-op from JavaScript's point of view). The right separation is: let Playwright manage the value of navigator.onLine, then wrap setOffline so it also dispatches the matching DOM event. The property stays accurate; the event listener finally wakes up.
Does this also affect the 'online' event after context.setOffline(false)?
Yes, symmetrically. Flipping the network back on through setOffline(false) does not trigger window.addEventListener('online') either. If your app reconnects to a WebSocket or refills a stale cache inside an online-event handler, that work never runs under test. The dispatchEvent wrapper must handle both directions: after setOffline(true) fire 'offline'; after setOffline(false) fire 'online'. Either one alone leaves half your offline story unverified.
What does Assrt do with network error testing?
The Assrt agent explicitly refuses it. The plan-generation prompt at /Users/matthewdi/assrt-mcp/src/mcp/server.ts line 219 reads: 'The agent can: navigate URLs, click buttons/links by text or selector, type into inputs, scroll, press keys, and make assertions. It CANNOT: resize the browser, test network errors, inspect CSS, or run JavaScript.' That exclusion is intentional. Low-level browser state (offline mode, request interception, CDP flags) belongs in a Playwright test you write and own, not in an English scenario handed to an agent. Assrt generates real Playwright, so the offline fixture below drops into the same tests folder and runs next to your Assrt-driven scenarios.
Does this gotcha apply to service workers, WebSockets, and navigator.connection too?
Partly. Service workers hit a separate bug tracked as microsoft/playwright#2311: setOffline did not stop SW-controlled fetches in older versions. WebSocket connections stay alive after setOffline; see microsoft/playwright#4910. navigator.connection.effectiveType does not change with setOffline because CDP's Network.emulateNetworkConditions is distinct from NetworkInformation emulation. Each surface needs its own workaround. The 'offline' event dispatch fix on this page addresses the biggest one, the one most apps actually subscribe to, but an app that also relies on service worker fetch failures or connection.type changes needs additional code.
Where do I put the wrapper so every test in my suite inherits it?
In a Playwright test fixture. The page shows the full fixture file: tests/fixtures/offline.ts exports a context fixture that patches context.setOffline to also dispatch the event via an addInitScript hook. Every spec that imports from './fixtures/offline' gets a patched context automatically. No per-test boilerplate. You can also drop the script into playwright.config.ts under use.storageState or a custom projects entry if you want the behavior to apply to every project in the config without an import.
Keep reading
The wait problem after context.setOffline(true)
networkidle hangs forever once the network is dead. Fixed waits flake. Here is the MutationObserver-based wait_for_stable Assrt ships in agent.ts, with line numbers.
AI-generated Playwright tests review: watch the run, not the .spec.ts
The .spec.ts file is not the test. The run is. Four artifacts, three minutes per test, zero hallucinated selectors to audit by eye.
Playwright vs AI browser automation: which stack to pick
Assrt drives real Playwright via Playwright MCP with an LLM agent on top. Every click is a Playwright call; every scenario is Markdown in git.
How did this page land for you?
React to reveal totals
Comments (••)
Leave a comment to see what others are saying.Public and anonymous. No signup.