Testing AI-Generated Apps: Edge Cases and the 80/20 Validation Trap

1. The Movie Set Problem

A movie set looks like a real building from the front. Walk behind it and you find plywood and scaffolding. Many AI-generated applications have the same architecture. The landing page is beautiful. The signup flow works. The main dashboard renders correctly. But try to cancel a subscription, handle a failed payment, recover from a network timeout, or process a form submission with Unicode characters, and the facade collapses.

This is not a criticism of AI coding tools. They are remarkable at generating the visible surface of an application. The problem is that developers mistake the visible surface for the complete product. The 80% that AI generates brilliantly is the 80% your users will interact with under normal conditions. The 20% it skips is the 20% that determines whether your application survives its first real-world deployment.

The gap between prototype and production is not about features. It is about resilience. Production applications encounter every edge case eventually. Network connections drop. Users paste unexpected data. Browsers crash mid-session. External APIs return malformed responses. Disk space fills up. The question is whether your application handles these situations gracefully or silently corrupts data.

2. What AI Generates Well

Understanding what AI tools handle well helps you identify where the gaps are. Modern AI coding assistants consistently produce high-quality output for UI components, CRUD operations, standard authentication flows, basic routing, and common API integrations. These are well-represented in training data, and the patterns are predictable.

Happy path implementations

AI excels at the golden path: user enters valid data, submits the form, receives a success response. This is the scenario most commonly demonstrated in tutorials, documentation, and code examples, which means it is heavily represented in training data. The AI generates clean, functional code for these scenarios.

UI scaffolding

Layout components, navigation structures, form rendering, and responsive design are all areas where AI produces reliable output. The visual layer of your application can be generated quickly and usually works correctly because UI patterns are well-established and consistent across projects.

3. The Invisible 20%

The invisible 20% encompasses everything that does not appear in a demo but matters in production. This includes error handling for every possible failure mode, input validation beyond basic type checking, concurrent access patterns, data migration and backward compatibility, graceful degradation when dependencies fail, and security hardening against malicious input.

AI tools tend to skip this layer for a simple reason: edge cases are context-specific. The edge cases for a medical records system are completely different from the edge cases for an e-commerce platform. AI cannot infer your specific failure modes from a prompt. It needs to be told, and most developers do not think to ask for comprehensive error handling during initial development.

This is exactly where automated testing provides the most value. Tests force you to think about what can go wrong. Writing a test for a network timeout requires you to implement timeout handling. Writing a test for invalid input requires you to implement input validation. The act of testing the invisible 20% is what transforms a prototype into a production application.

4. Edge Case Categories to Test

A systematic approach to edge case testing starts with categorizing the types of failures your application might encounter. Each category represents a class of bugs that AI-generated code commonly misses.

Empty and null states

What happens when a list is empty? When a user has no profile photo? When an API returns an empty array instead of the expected data? When a database field is null because it was added after the record was created? Empty states are the most common source of production crashes in AI-generated applications because AI tends to assume data always exists.

Boundary values

Zero, negative numbers, maximum integer values, extremely long strings, dates at the boundaries of ranges, and single-character inputs all live at the edges of expected input. AI-generated validation often handles the common cases but misses these boundaries. A quantity field that accepts negative numbers or a text field that crashes on emoji input are classic boundary value failures.

Timing and concurrency

What happens when two users edit the same record simultaneously? When a webhook fires before the associated record is created? When a user double-clicks a submit button? When a long-running API request completes after the user has navigated away? Concurrency bugs are nearly impossible for AI to anticipate because they depend on runtime timing that varies between environments.

State transitions

Applications have complex state machines: orders move from pending to paid to shipped to delivered. What happens when a refund is requested for a shipment already in transit? When a user tries to edit a record that another user just deleted? State transition bugs often cause data corruption because the application assumes transitions only happen in the expected order.

5. Security Validation

Security is the most dangerous gap in AI-generated code. AI tools often produce code that works correctly but is vulnerable to common attacks. SQL injection, cross-site scripting (XSS), cross-site request forgery (CSRF), broken authentication, and insecure direct object references are all vulnerabilities that functional tests will not catch.

Test your API endpoints with unauthorized tokens. Verify that users cannot access other users' data by manipulating URL parameters. Check that input sanitization works against script injection. Verify that file uploads reject executable files. Confirm that rate limiting prevents brute force attacks. These tests do not require specialized security tools; they can be written as standard Playwright or API tests with malicious input data.

The most critical security test is authorization boundary verification: ensuring that every API endpoint and page checks whether the current user has permission to access the requested resource. AI-generated code frequently implements authentication (is the user logged in?) but skips authorization (can this user access this specific resource?). A single missing authorization check can expose your entire database to unauthorized users.

6. Error Handling Depth

AI-generated error handling typically follows a pattern: try the operation, catch the error, show a generic error message. Production-grade error handling is far more nuanced. Different errors require different responses: retrying transient failures, displaying specific guidance for validation errors, logging detailed context for debugging, and gracefully degrading when non-critical services fail.

Network failure resilience

Test what happens when API calls fail at every point in your user flows. A payment submission that fails mid-transaction needs different handling than a search query that times out. Network failures should not lose user input, corrupt state, or leave the UI in an unusable state. These scenarios need explicit tests because they will occur in production.

Graceful degradation

When a non-critical service fails (analytics, recommendation engine, social login provider), the application should continue functioning without it. Test that your application degrades gracefully: the core functionality works even when optional dependencies are unavailable. AI-generated code rarely implements this level of resilience because it treats all dependencies as equally critical.

7. Testing Strategies for AI-Generated Code

The most effective approach is to use AI tools for both code generation and test generation, then review the tests manually for completeness. Start by generating your application with AI, then use a tool like Assrt to auto-discover test scenarios by crawling the running application. This gives you baseline coverage of the happy paths. Then manually add tests for the edge cases, security boundaries, and error conditions that automated discovery misses.

Run npx @m13v/assrt discover https://your-app.com to generate real Playwright tests for every discoverable user flow. These tests are standard Playwright files you can inspect and extend. Once the baseline is established, add targeted tests for the invisible 20%: empty states, boundary values, concurrent access, security boundaries, and error handling.

The key insight is that the invisible 20% is where bugs live. Investing testing effort there produces the highest return on quality. A hundred tests for happy paths catch fewer production bugs than ten tests for edge cases, because the happy paths were already working.

8. From Prototype to Production

The path from AI-generated prototype to production application is paved with tests. Each test you add addresses a specific failure mode and forces you to implement the handling code. This iterative process (test, implement, verify) systematically fills the gaps that AI left behind.

Do not wait until the application is feature-complete to add tests. The earlier you start testing edge cases, the cheaper they are to fix. A null pointer exception caught during development is a five-minute fix. The same bug discovered in production after it has corrupted user data is a multi-day incident with customer trust implications.

Your AI-generated application is not a movie set by default. It becomes one when you skip the invisible 20%. The tools exist to test that 20% efficiently. The question is whether you invest the time to use them before your users discover the gaps for you. The answer should be obvious, but the temptation to ship the demo is real. Resist it. Add the tests. Build the real building, not just the facade.