Testing Guide · April 2026

Testing new Hugging Face models in April 2026: drive the Space, not the API

Almost every model that lands on Hugging Face this month ships with a Space demo, and the Space is what most developers click on before they trust the weights. Driving that Space end-to-end takes one Markdown file, one real Chrome session, and zero.spec.tsfiles. Here is the recipe with the actual selectors and the failure modes you will hit when a Gradio rebuild changes every class hash on you.

Matthew Diakonov, Written with AI

Published April 29, 20269 min read

Direct answer · verified 2026-04-29

How do you actually test a new Hugging Face model in April 2026?

Open the model on huggingface.co/models?sort=created, click through to the linked Space, write a 5 to 8 step Markdown plan that types your prompt and asserts on the output, and run the plan with a Playwright MCP agent. The plan reads the live DOM each run, so a Gradio version bump does not break the test. The Hub's created-at index is the authoritative source for what is genuinely new this month.

Why most guides on this stop at curl

Look at the top results for this question and they all do the same thing: install the huggingface_hub client, call the Inference API with a curl example, print the response, done. That gets you a single token stream from a single endpoint. It does not get you the actual user experience of the demo, where the model author has wired up a system prompt, an image preprocessor, a safety filter, and a specific decoding config.

The Space is the integration the author claims produces the screenshots. The Inference API endpoint is a different integration with different defaults. If you are trying to decide whether the model is good, you want the author's integration. That means a browser, the demo URL, and a way to click the Submit button reliably across rebuilds.

The brittle way and the resilient way, side by side

On the left, a hand-written Playwright spec that looks fine today. On the right, the equivalent Markdown plan an MCP agent executes. The Gradio class hashes (.svelte-1ipelgc) regenerate on every Space rebuild, so the spec on the left dies at the next push. The plan on the right does not name a class once.

Same Hugging Face Space test, two different shelf lives

import { test, expect } from "@playwright/test";

test("Space generates text", async ({ page }) => {
  await page.goto("https://huggingface.co/spaces/owner/new-model");
  await page.locator(
    ".gradio-container .prose textarea"
  ).fill("Summarize Crime and Punishment in 80 words.");
  await page.locator("button.gr-button-primary.svelte-1ipelgc").click();
  await page.waitForTimeout(20000);
  await expect(
    page.locator(".output_text.svelte-1ipelgc")
  ).toContainText("Raskolnikov");
});

31% fewer lines

Six selectors that survive a Gradio rebuild

These are the role-based hooks the agent uses when walking a Space. None of them depend on the build hash that changes when an author redeploys. The agent reads the accessibility tree fresh on every step, so when a Space upgrades from Gradio 4 to Gradio 5 and every Svelte class shifts, the test still passes.

Prompt textarea (Gradio v4 and v5)

Found by role 'textbox' with no name. The agent ignores the wrapper class hash. The cl ass changes on rebuild; the role does not.

Submit button

Found by role 'button' with name matching /^Submit|Generate|Run$/i. Gradio names the primary button differently per Space, so a name regex covers the variations.

Output region for text models

Found by role 'region' or 'paragraph' under the output Block. Width does not matter; the agent reads the inner text and asserts on substring presence.

Output image for diffusion models

Found by role 'img' inside the output Image block. The agent waits until the src attribute changes from the placeholder data URL to a real /file= URL, then asserts the HEAD request returns 200.

Example button strip

Found by role 'button' inside the .gr-examples region. Each example carries a label that matches the prompt it inserts, so the agent picks an example by label rather than index.

Building/Sleeping banner

When a Space is paused, Gradio renders text 'Building', 'Sleeping', or 'This Space is restarting'. The agent's wait_for tool checks for those strings before any other step runs.

Walk-through, six steps, no Node project required

Find the model on the Hub's created-at index

Open https://huggingface.co/models?sort=created. Filter by task if you only care about a modality. The top rows are the most-recent uploads; in April 2026 you will see Llama 4 fine-tunes, new Qwen3 GGUFs, fresh diffusion checkpoints, and a long tail of specialist models pushed by individual researchers.

Open the linked Space, not just the model card

On the model card, scroll to the 'Spaces using this model' strip. Pick the Space whose author is the model author when possible; their demo carries the prompt scaffolding the model was trained for. If the author did not ship a Space, pick a community Space that has clear example prompts, because those become your test fixtures.

Write the plan in /tmp/assrt/scenario.md

One #Case per behavior you want to verify. Five to eight cases is the sweet spot before plans get hard to reason about. Write each step in plain English. The agent reads role-based DOM, so you say 'click the Submit button' rather than 'click button.svelte-9b8c2d.gr-button-primary'.

Run assrt_test pointed at the Space URL

The MCP server spawns a real Chrome via Playwright's MCP, attaches to your already-logged-in profile, and walks the plan. Snapshots, clicks, asserts, and waits go through the 14-tool MCP schema. There is no .spec.ts file. There is no playwright.config.ts. There is the plan, the agent, and your real Chrome.

Review the WebM and the JSON

Per run, the agent saves a video recording, per-step screenshots, and a structured JSON report. When a case fails, you copy the actual generated text from latest.json, paste it into the model's Hub discussion or a GitHub issue, and you have evidence rather than a recollection. Same evidence shape across every Space you test.

Commit the Markdown next to the model name

tests/scenarios/<owner>-<model>.md is a clean home. The plan is portable: a teammate who has never touched Assrt can read it in 30 seconds, run the same MCP server against the same URL, and reproduce the result. There is no codebase to onboard onto.

What a real run looks like

This is the terminal session for one #Case against a hypothetical Space at huggingface.co/spaces/<owner>/<name>. The agent boots a real Chromium via the Playwright MCP server, attaches to the persistent profile (so HF cookies are already loaded), walks the Markdown plan, and writes a recording plus a JSON report. Inference time dominates; agent overhead is around 800ms per step.

zsh — assrt_test against a Hugging Face Space

Plan executor vs hand-written spec for HF Spaces

The choice is not really about Assrt vs Codegen. It is about whether the test should be a piece of TypeScript that names classes, or a piece of Markdown that names roles. The trade-off shows up the day the Space rebuilds.

Feature	Hand-written .spec.ts	Markdown plan
Selector survives a Space rebuild	No, class hashes change on rebuild	Yes, role-based and re-read each run
Works against gated models	Needs HF_TOKEN env or login script	Yes, reuses your real Chrome HF cookies
Files added to your repo	playwright.config.ts plus tests/*.spec.ts	One Markdown file (the plan)
Edits at runtime	Re-run codegen, diff, re-commit	Save the .md, agent picks it up in 2 seconds
Reading the test for review	TypeScript with selector strings, dev-only	English steps, anyone on the team can read
Vendor lock-in	Whatever shape your closed SaaS chose	Markdown plus standard Playwright MCP

“Lines of TypeScript needed to test a new Hugging Face Space”

The plan lives in /tmp/assrt/scenario.md as plain Markdown

What April 2026 specifically changes

Two things matter this month. First, Gradio 5 has rolled out broadly enough that you cannot assume a Space is still on 4.x; class hashes are different even within Gradio 5 minor versions because the Svelte build cache invalidates per push. Hand-written CSS-class selectors written before April are largely dead. Second, the rate of new specialist models on the Hub has gone up: small fine-tunes of Llama 4, audio diffusion models, video understanding models, all with a Space and an example button. That makes the per-Space test cost the bottleneck. If each evaluation requires writing a fresh Playwright file, you will not bother. If each evaluation is a 5-line Markdown plan you can produce in 30 seconds, you will.

0sagent start-to-first-output for a text Space

0.spec.ts files added to your repo

0 casessweet spot per Space (5 to 8)

0sfs.watch debounce on plan edits

What I actually do when I want to test five new models in an hour

Open https://huggingface.co/models?sort=created and copy five model names off the top of the list that match the task I care about.
For each model, click into the model card, find the "Spaces using this model" strip, and grab the Space URL.
Write one /tmp/assrt/scenario.md with five #Case blocks, one per Space, all running the same prompt. The prompt is my apples-to-apples test sentence.
Run assrt_test once, walk away, come back in five minutes.
Open /tmp/assrt/results/latest.json and read the actual outputs side by side. The model that did best on my prompt is the one I integrate next.
Commit the .md to a tests/eval/ folder so next month I rerun the same prompt against the next batch.

Honest limits

A browser-driven Space test is not the right tool when you need quantitative benchmarking on thousands of prompts (use the Hub's evaluation harness or a direct Inference API call for that), when the model is gated behind a paid endpoint without a free Space, or when you want millisecond-level timing measurements. Use it for what it is: a fast, honest, human-readable way to decide whether a freshly-released model does the thing the author claims, before you wire any production code to it.

Want a hand wiring this against the model you are evaluating?

Book a 20-minute call. We can write the plan together against the actual Space, run it, and look at the JSON output.

Frequently asked questions

What is a Hugging Face Space and why does it matter when a new model drops in April 2026?

A Space is a small web app, usually built on Gradio or Streamlit, that the model author publishes alongside the weights so you can try the model in a browser without setting up an inference server. When a new model lands on Hugging Face in April 2026 (Llama 4 fine-tunes, new Qwen3 quantizations, the wave of small specialist models on the Models tab), the linked Space is almost always the first thing a developer touches. The Space's URL pattern is huggingface.co/spaces/<owner>/<name>; the Gradio runtime serves a single-page app at that URL with the prompt input, the output panel, and a few example buttons. If you want to verify a model behaves before pulling it into your product, you click through that Space, and that clicking is exactly what a browser test automates.

Why not just call the Inference API and skip the browser test?

Two reasons. First, the Space frequently does work the raw model does not: pre-prompt scaffolding, system prompts, post-processing, image preprocessing, safety filters. The Space is the integration that the model author claims gets the result you see in screenshots; the API endpoint may not include any of it. Second, when you are evaluating a new model in April 2026 you usually want to compare three or four candidates side by side, and the example prompts wired into the Space are the apples-to-apples test set the author already tuned for. Driving the Space gives you the comparison the author intended, with the prompts they already verified work.

What breaks when you write Playwright code against a Gradio Space and the Space upgrades Gradio?

The selectors. Gradio auto-generates the DOM from a Python definition, and the class names change between major versions. A test that locates the prompt textarea via .gradio-container .prose textarea works on Gradio 4.x and silently fails on Gradio 5.x because the wrapper class moved. The output panel selector .output_text.svelte-1ipelgc shifts when the build hash regenerates, which it does on every Space rebuild. If you wrote a .spec.ts in March and the Space rebuilt last week, the file is dead until someone runs codegen again. The plan-executor approach reads the accessibility tree fresh on every run and finds the textarea by role, so a wrapper class change does not break it.

What does the actual Markdown plan look like for a Hugging Face Space test?

Five to seven lines, written like an English instruction sheet. #Case 1: Generate text from a prompt. 1. Navigate to https://huggingface.co/spaces/<owner>/<model>. 2. Wait for the prompt textarea to appear. 3. Type "Summarize the plot of Crime and Punishment in 80 words." 4. Click the Submit button. 5. Wait for the output panel to contain at least 50 characters. 6. Assert: the output mentions Raskolnikov. The plan lives in /tmp/assrt/scenario.md, the agent loads it at run time, and the same plan keeps working when the Space rebuilds because the agent re-reads the DOM each step.

How long does the agent take to run one #Case against a Hugging Face Space?

12 to 45 seconds for a text model Space, depending on the model's inference latency. The bulk of the time is the model itself generating tokens, not the browser automation. The agent's own overhead per step is around 800 milliseconds: snapshot, choose a tool call, execute, verify. Image-generation Spaces (Stable Diffusion 3.5 Turbo, FLUX.1 schnell, the new April releases) take longer because the actual inference is 4 to 30 seconds per image. The agent waits using a wait_for_stable tool that polls the DOM until output content stops growing, so you do not have to hardcode a sleep.

Can I run this against a Space that requires a Hugging Face login?

Yes. The agent attaches to a real Chrome instance via Playwright's --extension flag (browser.ts spawns @playwright/mcp with --browser chromium and --extension when the EXTENSION env var is set). That Chrome carries your existing huggingface.co cookies; the Space loads logged in, the rate limit on the demo is your account's rate limit instead of the anonymous quota, and gated models that require an HF token in the Space settings work because the Space already has them. You log in once in your real Chrome, then every subsequent run reuses that session.

What is the lookup URL where I check that a model is actually new in April 2026?

https://huggingface.co/models?sort=created with the date filter set to your window. The Hub's models index sorts by upload date, which matches what most developers mean by released. The trending tab (https://huggingface.co/models?sort=trending) is the social signal but it conflates a model from January that is suddenly popular with one that shipped this week. For a true newness check, use sort=created and filter by task or library. This is the authoritative source; the page renders directly from the Hub's database.

What evidence does the agent leave behind after a run, so I can defend the result in a review?

Three things. A WebM recording of the browser session at /tmp/assrt/results/<scenario-id>/recording.webm, a JSON pass/fail report at /tmp/assrt/results/latest.json with per-case assertions and the actual generated text from the Space, and a sequence of screenshots at /tmp/assrt/results/<scenario-id>/screenshot-<n>.png taken after each major action. When a model under-performs the author's claims, you reach into latest.json, copy the prompt and the failed output verbatim, and paste them into a GitHub issue. No retyping the prompt from memory.

What if the Space is down or stuck on the build queue?

Hugging Face Spaces hibernate after about 48 hours of inactivity on the free tier. The first request after that wakes the Space, and the Gradio frontend renders a 'Building' or 'Sleeping' state instead of the model UI. The agent will see those words in the accessibility tree and the case will fail with an evidence string like 'expected output panel, found Building text.' The right move is to add a step at the top of every #Case: wait until the prompt textarea is interactive. The agent's wait tool plus the role-based selector handle this transparently; on a closed-source generator you would write a custom waitForFunction. Markdown plans cost nothing.

Does this approach work for non-text models, like the new April 2026 audio and image releases?

Yes. The selector strategy is identical because Gradio uses the same component primitives across modalities; only the input/output widgets differ. For an image model, the input becomes a textarea or a file upload component (Gradio's Image() block exposes a drop-and-upload region with a file input you can hit via the upload_file MCP tool), and the output becomes an <img> with a src that swaps when generation finishes. The case becomes: type the prompt, click submit, wait for the output image src to change, assert the response code on the new image URL is 200. The agent's tool surface includes upload_file and screenshot, so the same Markdown vocabulary covers the modality.