Odyssey releases PROWL

Odyssey releases PROWL, an adversarial training framework that uses a MineRL agent to find and fix world model failures through a prioritized buffer.

Odyssey, a specialist in World Models, publishes PROWL (Prioritized Regret-Driven Optimization for World Model Learning), an adversarial training framework in which a reinforcement learning agent explores game environments to reveal failures of its accompanying world model. At each iteration, detected errors (geometry breaks, visual inconsistencies, poor action tracking, unrealistic dynamics) are reinjected as training data, creating a closed loop between failure discovery and model improvement.

The system relies on a buffer named Prioritized Adversarial Trajectory (PAT), which prioritizes failing trajectories based on their difficulty. Once a failure is corrected, PAT de-prioritizes it to focus computational effort on unresolved cases, forming a curriculum that hardens as the world model progresses. The agent is also kept within a realistic exploration zone to prevent it from exploiting the model via out-of-distribution actions.

The framework was tested on the MineRL research environment, derived from Minecraft, with an initial model trained by Diffusion Forcing on a human dataset. Comparisons with the baseline show progress in action tracking, the stability of 3D-anchored UI elements, the temporal consistency of scene transitions, the persistence of placed objects, and the fidelity of non-rigid phenomena like waterfalls. PROWL also manages to reproduce emergent behaviors absent from the initial human data, such as 180° pivots and dodges discovered by the agent.

The paper is co-authored by Ahmet H. Güzel, Jenny Seidenschwarz, Benjamin Graham, Jonathan Sadeghi, and Jeffrey Hawke, with contributions from Jack Parker-Holder and Ilia Bogunovic, from University College London.