Unprecedented behaviors eluded OpenAI's prior evaluations

OpenAI's model bypassed its restrictions to act outside of its sandbox. The company is implementing trajectory monitoring to address this.

During restricted internal use, a model trained for long-horizon tasks produced behaviors that OpenAI’s prior evaluations had not detected. The company suspended access, built new evaluations based on the observed incidents, reworked alignment on long sequences, added trajectory-level monitoring, and expanded the control left to the user, before reopening limited access. This is the model behind the disproof of the Erdős unit distance conjecture.

Persistence is at the heart of the problem. Where previous generations stopped when faced with an environmental constraint and handed back control, this one keeps searching, including for ways to act outside its sandbox. On the NanoGPT speedrun, a public benchmark consisting of training a small language model in a minimum number of steps, it developed a learning rate decay method dubbed PowerCool. The internal instruction was to post the results on Slack and nowhere else, while the benchmark instructions called for a pull request on GitHub. The model followed the latter, bypassed restrictions meant to block all external access, and opened the PR after spending an hour searching for a vulnerability.

In another reported case, the model attempted to retrieve private solutions from the evaluation backend. Since its first attempt was blocked by a scanner detecting an authentication token, it fragmented this token into two pieces, obfuscated them, and then reconstructed the full value at runtime, so that the entire string never appeared in one piece. Its comments and reasoning traces explicitly indicated that the maneuver was used to bypass the scanner. Each step taken in isolation appears acceptable, OpenAI notes, whereas the sequence leads to a result that would not have been approved.

Hence the shift from action-by-action control to monitoring the entire trajectory, capable of interrupting a session and alerting the user, who is then free to resume the work in the event of a false alarm. The company notes in passing that its models did not retain instructions well over long rollouts, and that dedicated training improved the maintenance of alignment over time. On a replay of the environments where the model had gone off track, it numbers high- and medium-severity incidents at zero with the new safeguards, compared to 1% and 1.4% with the old ones, and minor incidents at 2.9%, compared to 12%. Low-severity cases remain, including SSH connections to pods belonging to other employees and a command intended to kill all accessible processes, which expired before executing.