Open source API testing tools have a UI-shaped blind spot. Here is the category gap, and the 13-line fix.
Bruno, Hurl, Hoppscotch, SoapUI, Karate, REST Assured, Schemathesis, EvoMaster. Every open source API testing tool in 2026 operates at the same layer: the HTTP boundary. You send a request, you assert on a response, you go home. That is the right shape when the API is consumed by partners reading your OpenAPI spec. It is the wrong shape when your API's biggest consumer is your own UI, because now you own two test pyramids that never touch. This guide is the category map plus the one thing missing from it.
The category, in 2026
These are the eight open source API testing tools that appear on every category roundup you will find, and why each is sharp at one slice of the work. If you are here for the shortlist, this is your shortlist. The analysis starts below.
Bruno
Git-friendly desktop client. The .bru spec format is plain text, version-controllable, and a clean escape hatch from Postman. Best for teams that hated the cloud account requirement.
Hurl
Single static binary. Plain-text files with chained requests and JSONPath assertions. Fast enough to run on every commit. Best for smoke suites in CI.
Hoppscotch
Browser-native client with a self-hostable stack. WebSocket, SSE, MQTT in one UI. Best for teams that want a Postman replacement without installing a desktop app.
SoapUI
Java desktop app, open source community edition. Deepest SOAP and WSDL support in the category. Best if you still maintain SOAP services in 2026.
Karate DSL
Gherkin-style .feature files that collapse HTTP, WebSocket, and GraphQL testing into one grammar. Readable by non-developers. Best for BDD shops.
REST Assured
Java library you drop into JUnit. The go-to pick when your test suite already lives in a Maven project and you want HTTP assertions next to unit tests.
Schemathesis
Property-based fuzzer that generates cases from an OpenAPI or GraphQL schema. Catches 500s on malformed payloads. Not for happy-path testing, for catching the weird stuff.
EvoMaster
White-box evolutionary fuzzer. Reads the service under test and evolves inputs to maximize branch coverage. Research-grade power, learning-curve cost.
Used by teams who outgrew Postman
What every listicle leaves out
I read the five top-ranking articles on this topic before writing this one. Between them they named about 25 tools and compared on eight criteria: REST support, SOAP support, GraphQL support, WebSocket support, CI integration, reporting, pricing, learning curve. They are good shopping guides. They all miss the same architectural question.
When a user in your product clicks Connect Slack, what actually happens? A click, then a redirect, then an OAuth code exchange, then a session cookie, then a webhook, then a rendered success state. A pure API test hits /api/integrations/slack/connect and asserts the JSON. It tests one of those six steps. The other five are the ones that break in production.
Every open source API testing tool in the list above assumes the browser and the HTTP layer are separate concerns. They were right for a while. In a world where the app is the UI and the UI is a thick client that mediates between the user and twelve internal and external APIs, they are no longer right.
It proves the endpoint returns 200 with the right shape. It does not prove the button that fires that endpoint is wired to the right handler, that the session cookie is honored, that the webhook ever landed.
A real browser session that sets real state. An HTTP call that runs inside that session. An assertion that checks the external side effect, not just the internal response.
The fix, by the numbers
You do not need a new runtime to close this gap. You need an agent that has both a click primitive and an http_request primitive in the same tool list, and a system prompt that knows when to use each. In Assrt's agent, the tool definitions are small and the pattern is explicit.
Thirteen lines of tool definition. One shared TOOLS array. One SYSTEM_PROMPT line that teaches the model the pattern. That is the whole mechanism. Everything else is the agent loop you already have if you are running any MCP-based browser agent.
One runner, both layers
The architecture diagram nobody else in the category can draw. Three inputs on the left feed one agent loop, which emits three outputs. The agent has both browser tools and an HTTP tool in its hand at all times.
Mixed UI + HTTP scenario: one agent, two layers
The 13-line tool definition
Here is the whole mechanism. This block lives in the same TOOLS array as the browser tools. It is not a plugin, not a separate package, not a second agent. The agent can reach for it on any step inside any scenario.
The second half of the mechanism is the system prompt. The agent is not just given the tool, it is taught when to use it. This block is the training signal that turns a generic browser agent into a tool that does UI-triggered API verification.
What happens when a mixed scenario runs
A typical Connect-Telegram test. The reader, the agent, the real browser, the product backend, and the Telegram Bot API, in one message trace. Notice that the agent never leaves the loop. No handoff to a second runner. No second test file.
Mixed UI + HTTP trace
The scenario file that covers both layers
A single file on your disk. Two cases. The first verifies that connecting Telegram actually delivers a real bot message. The second verifies that disconnecting fires the revocation webhook. Both cases mix click steps with http_request steps, and the agent runs them in one loop.
What the run actually prints
Browser steps and HTTP steps in the same log, the same video, the same pass/fail report. No split between suites. No correlating runIds across runners.
Five steps across two layers
The anatomy of a UI-triggered API test. Each step names the exact thing the agent does and where it happens.
Step 1. The reader clicks a UI control that triggers an API call
This is where every pure API testing tool stops being useful. The click sets state in the browser: session cookie, CSRF token, local storage. A standalone HTTP test would have to replicate all of that manually.
Step 2. The agent picks the element ref from a live accessibility snapshot
No selectors in the scenario.md file, so nothing to rot. The ref is fetched at run time from @playwright/mcp. When the button moves, the selector does not break because there is no selector.
Step 3. The click executes in the real browser, setting real cookies
The browser is not mocked. It is a Chromium instance under Playwright control. Every cookie, header, and piece of session state that your production backend sets is actually set here.
Step 4. The http_request tool fires inside the same agent turn
agent.ts line 857: await fetch(reqUrl, fetchOptions). With a 30-second AbortController timeout. The response is truncated at 4000 chars and handed back to the agent as a tool result. The HTTP call runs in the Node process, but any cookie header the scenario passes is the cookie the browser has already set.
Step 5. The agent asserts across both layers and completes the scenario
The assertion can check an element on the page, a field in the JSON response, or a correlation between the two (the user ID in the UI matches the user ID in the webhook payload). That single cross-layer assertion is the thing pure API tools cannot do at all.
Where the HTTP call actually runs
The switch case for http_request is a few dozen lines inside the same tool-dispatch function that handles click, type_text, and navigate. A 30-second AbortController guards the fetch. The response is truncated at 4000 characters and handed back to the agent as the tool result. The whole thing lives in the same process as the browser driver.
Category view: where this fits next to Bruno, Hurl, and friends
The other tools are not wrong. They are sharp at what they do, and most teams should keep running at least one of them. The comparison below is specifically about the UI-triggered test, not about pure API contract testing.
| Feature | Typical open source API tool | Assrt |
|---|---|---|
| How tests are written | Vendor UI, .bru, .hurl, .feature, .java, or .yaml spec | Plain English in /tmp/assrt/scenario.md |
| UI interactions in the same test | Not supported. Separate tool (Selenium, Cypress) handles UI. | click, type_text, navigate sit in the same TOOLS array as http_request |
| Session cookie carryover | Manual. Extract the cookie from the login response, pass it on every request. | The agent clicks Login. The cookie is set in the real browser. http_request inherits the session. |
| Setup cost for a UI → API test | Two codebases, two runners, two CI jobs, one duct-tape handoff | One scenario.md. One agent loop. One video recording of both halves. |
| Third-party API verification | You write a separate suite that hits the third-party API with static fixtures. | SYSTEM_PROMPT line 247 trains the model: 'after connecting Telegram, poll /getUpdates to verify' |
| Runtime engine | Proprietary runner or wrapped HTTP client | @playwright/mcp for UI, native fetch for HTTP. Both open source, both real browsers and real requests. |
| License | Mix of MIT, Apache, GPL, plus commercial-only premium tiers | MIT for the MCP server, Apache 2.0 for the browser stack. One version. No community-vs-enterprise split. |
| Cost at team scale | Free for the OSS core, then $7.5K+/month for the closed AI QA platforms doing similar work | $0 runner + your Anthropic token spend. Self-hosted, BYO key. |
How to split the work in a real 2026 stack
The answer is not pick one, the answer is layer them. A realistic setup I have seen work in production this quarter.
Schemathesis reads your OpenAPI spec and generates cases that try to break you. Hurl runs a fast smoke suite on every PR. Cheap, deterministic, finds the 500s and the schema drift.
For poking at an endpoint while debugging. Bruno commits to Git so requests live with the code. Hoppscotch if you want a browser tab instead of a desktop app.
Connect-an-integration flows. Signup-then-verify-welcome-email flows. Webhook round-trip flows. Anything where the assertion needs to look at both the rendered page and an external service state.
When the external service is rate-limited, slow, or down, stub it. Point the scenario at the mock URL via a variable and reuse the same scenario.md file.
Have a UI-triggered flow that nothing else tests? Let us look at it together.
Bring one real scenario where a button click fires an API call, and we will write the #Case live and run it in 10 minutes.
Book a call →Frequently asked questions
Which open source API testing tools actually matter in 2026, and what are they best at?
Bruno is the fast desktop client with a Git-friendly .bru spec format, best for teams escaping Postman lock-in. Hurl is a single-binary CLI that runs plain-text files with chained requests and JSONPath assertions, best for CI smoke suites. Hoppscotch is the browser-native Postman alternative with a clean UI and a self-hostable stack. SoapUI is the veteran Java desktop app that still has the deepest SOAP and WSDL support. Karate DSL is a Gherkin-style BDD framework that collapses HTTP, WebSocket, and GraphQL testing into the same feature file. REST Assured is the Java library you drop into JUnit to assert on responses. Schemathesis is a property-based fuzzer that generates cases from an OpenAPI or GraphQL schema. EvoMaster does something similar but with a white-box evolutionary approach. Each is sharp at one slice of the category. The piece none of them own is UI-triggered API verification, where the browser session has already set the auth cookie and your test needs to fire an HTTP request inside that same context.
Why is testing APIs in isolation not enough anymore?
Because the #1 consumer of most product APIs is the product's own UI, not a partner. When a user clicks Connect Slack, the browser does a redirect, the backend exchanges a code for a token, writes a session cookie, fires a webhook, and renders a success state. A pure API test hits /api/integrations and asserts on the response shape. It never exercises the redirect chain, the cookie handoff, the CSRF state, or the webhook delivery. When the flow breaks in production, the API test is green and the UI test is green, because each tested its own half. The bug lives in the seam.
How does Assrt interleave HTTP calls with browser actions without a second test file?
The http_request tool is registered in the same TOOLS array as navigate, click, and type_text. See /Users/matthewdi/assrt-mcp/src/core/agent.ts lines 171 to 184 for the tool definition, and lines 243 to 247 of SYSTEM_PROMPT for the pattern the agent is trained on. Inside a single #Case, the agent can click a Connect button, then call an external API to verify the side effect, then click another UI element to confirm the rendered state. There is no second runner, no handoff script, no token extraction step. The browser session, the fetch call, and the assertion all run in the same tool loop.
Do session cookies from a UI login carry into the http_request call?
When you need them to, yes. The http_request tool accepts a headers object, so if you need to authenticate an internal endpoint, you pass the cookie or bearer token as a header. For external third-party APIs like Telegram, Slack, or GitHub, the typical pattern is to use the API's own token (bot token, webhook secret, personal access token) which is already the right scope for verification. The agent can pull that token from a test variable (the variables map at server.ts line 344 interpolates {{KEY}} placeholders) or from the page after the UI flow exposes it.
Can I run Bruno, Hurl, or Hoppscotch next to Assrt instead of replacing them?
Yes. Bruno or Hurl are excellent for contract-level tests that belong in a pipeline gate before any browser boots. Schemathesis is the right fuzzer for catching 500s on malformed payloads. Assrt is the tool you reach for when the question is 'does this whole user journey work', and the journey includes both a button click and an HTTP side effect. A reasonable 2026 stack is Schemathesis for fuzzing, Hurl in CI for smoke, Bruno for local exploration, and Assrt for end-to-end scenarios that cross the UI-API line. They do not overlap enough to be competitive.
Is Assrt actually open source or is this a hosted-only trick?
The Assrt MCP server is MIT-licensed at /Users/matthewdi/assrt-mcp/LICENSE. Playwright and @playwright/mcp are Apache 2.0. The Anthropic SDK is MIT. Cloud sync to app.assrt.ai is opt-in and gated by a UUIDv4 that acts as the access token (see scenario-store.ts line 8); run with an inline --plan and nothing leaves your machine beyond the LLM call. Compared to commercial AI QA platforms at $7.5K per month per seat that require uploading your DOM to their cloud, the bytes-out budget is the Anthropic token bill plus zero.
What does the plain-English scenario look like when it mixes UI steps and HTTP calls?
A single #Case in /tmp/assrt/scenario.md with numbered steps. For example: '1. Navigate to /settings/integrations. 2. Click Connect Telegram. 3. Copy the bot token from the confirmation dialog. 4. http_request GET https://api.telegram.org/bot<token>/getUpdates. 5. Assert the response contains a message from this workspace in the last 60 seconds.' The agent reads this like a QA engineer would. It picks element refs from the live accessibility snapshot for steps 1 through 3, fires a real fetch for step 4, and makes a structured assertion for step 5. All five steps run in one agent loop.
How does this compare to Cypress's cy.request or Playwright's APIRequestContext?
Both Cypress and Playwright let you make HTTP calls from inside a browser test, which is already closer to the right shape than a pure API tool. The difference is where the test lives and who writes it. With cy.request and APIRequestContext, the test is a TypeScript file with selectors, locators, and a compile step; a developer writes it and owns its decay. With Assrt, the test is a Markdown case with prose, no selectors exist in the file, and the agent picks elements at run time from the live page. Cypress and Playwright are right when you need deterministic per-action speed (about 1 second per action) and you have engineers who want to own .spec.ts files. Assrt is right when you want a scenario.md that a PM or QA lead can edit, run in 4 to 6 seconds per action, and still exercise the UI-API seam.
What happens when the external API I am verifying has rate limits?
The agent makes one request per http_request tool call, with a 30-second AbortController timeout (agent.ts line 855). If the external API returns 429, the tool result is 'HTTP 429 Too Many Requests' plus the response body, and the agent sees that in its next turn. The usual pattern is either to back off in the scenario with a wait step before the API call, or to split the scenario into smaller cases that exercise one integration at a time. For high-volume verification, run the scenario against a staging endpoint or a mock server (WireMock, Mockoon, Prism) and let your contract tests handle the real rate-limited service.
If the agent is making real HTTP calls, how do I stop it from hitting production by mistake?
Three layers. First, the agent only knows the URLs that are written into the scenario.md, so a scenario that only references localhost cannot accidentally fire at a production hostname. Second, the variables map at server.ts line 344 lets you pass an environment-specific base URL as a {{BASE_URL}} placeholder and keep the prose the same across dev and staging. Third, in CI you pass a scenarioId that was vetted in the web UI, and anything new needs a code-review approval before it reaches main. Nothing inside the agent will invent a URL you did not provide.
How did this page land for you?
React to reveal totals
Comments (••)
Leave a comment to see what others are saying.Public and anonymous. No signup.