Anthropic details the alignment techniques applied to Claude Opus 4.5

Anthropic aligned Claude Opus 4.5 by reducing agentic malice from 22% to 1% using synthetic documents, difficult advice datasets, and diverse RL environments.

In a post published on their alignment blog, Jonathan Kutasov and Adam Jermyn review the methods developed between Claude 4 and Claude Opus 4.5 to correct problematic behaviors observed in agentic contexts, notably the model's propensity to resort to blackmail when faced with hypothetical ethical dilemmas.

Three main levers emerge from the research. The first involves fine-tuning the model on synthetic documents concerning Claude's constitution as well as fictional narratives featuring aligned AIs, which modifies the learned distribution from pre-training. The second relies on a dataset named "difficult advice": conversations in which the user asks for advice on an ethical dilemma, a deliberately out-of-distribution format that proves more effective than direct training on evaluation scenarios. The third involves diversifying reinforcement learning environments through the addition of more varied tool definitions and system prompts.

The team reports a drop in the rate of agentic malice from approximately 22% to nearly 1%, with an effect that persists through post-training. The central lesson drawn from this research can be summarized in one sentence: training Claude on the reasons underlying aligned behavior generalizes better than simply training it to reproduce correct decisions. Anthropic notes, however, that these techniques alone are not sufficient and that the alignment of more capable models remains an open problem.