GPT-Red, the in-house attacker OpenAI trains against its own models

OpenAI's GPT-Red model automates the search for security vulnerabilities. It succeeds in 84% of indirect prompt injections compared to 13% for humans.

GPT-Red is an automated red-teaming model that OpenAI keeps internal to flush out vulnerabilities in its systems before they are deployed, first and foremost among them prompt injections. The principle replicates the actions of a human red-teamer, but on a different scale: sending a rigged query, observing the target model's response, and repeating the process while refining the attack.

Training involves self-play. GPT-Red and a series of defender models progress simultaneously across a wide range of scenarios. The attacker is rewarded when it triggers a vulnerability, while the defenders are rewarded when they resist while successfully completing their task. As the defense hardens, GPT-Red must vary its attacks, which can be embedded in a local file, a web banner, the body of an email, or the output of a tool.

OpenAI then turned the model against its own production systems. The claimed result: GPT-5.6 Sol, adversarially trained against it, suffers six times fewer failures on its toughest direct injection benchmark than the company's best production model four months prior. In an indirect injection arena, the attacker succeeds in 84% of scenarios, compared to 13% for human red-teamers, according to the company.

The real-world test remained. Unleashed on the agent of a vending machine installed in OpenAI's offices, GPT-Red achieved its three objectives: reducing a costly item to the rock-bottom price of fifty cents, ordering a product worth over a hundred dollars to sell it off for fifty cents, and canceling another customer's order. The vulnerabilities were reported, and new safeguards are being tested. The model remains sandboxed far from deployed versions, and a more detailed preprint is expected later this week.