Breken Wants to Find Software Bugs Before Users Do by Simulating Real-World Conditions

Breken simulates production environments to find software bugs before users encounter them, identifying 76 of 235 defects in its ESCAPES benchmark.

Automated tests generally check situations that developers have anticipated. Some bugs, however, appear only after weeks of use, when an external service fails, two people edit the same document, or a payment operation is interrupted at exactly the wrong moment. Breken is introducing a system designed to reproduce those conditions before customers encounter them. Its software runs applications in isolated environments, simulates users and dependencies, and searches for failures before preparing potential fixes. In a benchmark covering 33 open-source products, the company reports finding 76 of 235 bugs, compared with 37 for Claude Fable 5.1 and 28 for GPT-6 Astra. The Bugs Conventional Tests Do Not Catch

AI coding assistants are accelerating software development, but they do not eliminate the problems that emerge after deployment. An application can pass unit tests, satisfy code review, and operate correctly during a demonstration while behaving differently as data accumulates or concurrent operations overlap. Some of the hardest defects are caused not by an obvious mistake in one function, but by a combination of events rarely encountered during development.

Breken aims to address the gap between technical validation and real-world use. The company describes this discipline as production engineering: instead of testing only what software is expected to do, it attempts to observe how the system behaves under unexpected conditions. Rather than waiting for customers to report failures, Breken generates scenarios intended to expose those problems earlier.

The approach is not presented as a replacement for existing tests. Code review remains useful for identifying certain logic errors and assessing changes before they are merged. Automated tests verify behaviors developers already understand. Breken adds another layer, exploring how a system behaves after accumulating history, users, pending tasks, and unreliable external services. Reconstructing an Application and Giving It a History

Breken runs a copy of the application on its own infrastructure. Databases, job queues, payment systems, email services, and other dependencies are represented inside an isolated environment. The stated goal is to reproduce their behavior without allowing the simulated environment to communicate directly with external production systems.

The system begins by building a plausible operational state. It creates accounts, records, pending transactions, and situations that would normally emerge after extended use. A nearly empty database does not expose the same problems as one containing large numbers of records, repeatedly renewed subscriptions, and interrupted jobs. Breken therefore attempts to construct states that would otherwise take months of activity to develop.

The company highlights simulated time acceleration of up to 4,000 times real time. Its demonstrations describe exploring years of operational history within hours. This makes it possible to examine expiry dates, renewals, configuration changes, and data that evolves over long periods.

The simulation also includes people and software agents. An administrator may make repeated bulk changes, a user may open the same document in multiple tabs, and an agent may process refunds from an inbox. Those behaviors are combined with network interruptions, delays, and state changes to uncover failures. Changing One Condition to Understand a Failure

Once the environment has been constructed, Breken explores different scenarios by varying one condition at a time: a lost network response, a repeated operation, a broken connection, a clock change, or a deployment during an active job. Other parts of the scenario remain unchanged, allowing the system to distinguish a reproducible failure from an incidental outcome.

One example concerns a refund following a partial payment capture. Under normal conditions, the refunded amount should reflect the money actually collected. In a particular state, however, the application refunds the full amount originally authorized. By executing the operation with and without the partial capture, Breken can isolate the condition responsible for the error.

When it identifies a defect, the system collects the initial state, reproduction steps, and execution traces. It can then prepare a draft pull request for developers to review. The objective is to provide a demonstrable failure rather than a general warning that some part of the code might be vulnerable to a problem.

Not every finding reaches the same standard of proof. Some failures can be reproduced automatically and compared with successful control runs, while others still require interpretation or human validation. ESCAPES: 327 Real Bugs Across 33 Software Products

To evaluate its approach, Breken created ESCAPES, a benchmark based on defects reported by actual users of open-source software. Its first version contains 327 bugs across 33 products, including web applications, APIs, developer tools, and software libraries. Each defect is tied to a public report and a merged fix dated between July 1 and October 19, 2026.

The evaluation reconstructs a previous version of each product, before the bug became known, and asks participating systems to identify problems users would encounter later. They can access the software, documentation, and earlier repository history, but not the subsequent bug reports or fixes.

Of the 327 collected defects, 235 were already present in the code on July 1, 2026. Those form the main scored set. The other 92 were introduced later, making them impossible to find in the earlier code snapshot.

The systems operate under comparable spending limits and are evaluated on whether their findings describe the same failures subsequently reported by users. Breken is compared with Claude Fable 5.1 and Claude Opus 5.5 through Claude Code, GPT-6 Astra through Codex CLI, and Semgrep, a static analysis tool. The coding agents can inspect repositories, read documentation, and execute