EdgeBench: ByteDance Measures How Agents Learn in Real-World Environments

ByteDance Seed introduces EdgeBench, an open-source benchmark measuring how AI agents like GPT-5.5 learn and improve over time in real-world environments.

A classic benchmark evaluates what a model already knows. EdgeBench, published by ByteDance Seed, focuses on a different aspect: how an agent progresses when given time, feedback, and room to improve. The benchmark brings together 134 tasks drawn from real-world practices, divided into six families (scientific and ML problems, software engineering, optimization, knowledge work, formal mathematics, and games), most of which were built from scratch. Each task runs continuously for 12 to 72 hours, allowing experience to accumulate; expert human effort averages 57.2 hours, and up to 320 hours for the most demanding cases. The agent operates in a loop: it attempts a solution, observes environmental feedback, analyzes it, and then converts that experience into better plans. ByteDance is open-sourcing 51 of the 134 tasks and the entire evaluation framework.

After 38,000 hours of runs, a regularity emerges. By averaging the learning curves point by point, the noisy trajectories converge to a single log-sigmoid function, with an average R² of 0.998. ByteDance proposes an interpretation through graph exploration: a task's score is assembled from a multitude of small units; unlocking one makes its neighbors more accessible, and progress advances like a front gaining ground. Comparing models released between September 2025 and May 2026, the team also notes that the learning speed in an environment would approximately double every three months.

The study details one case: on a gravitational wave reconstruction task, a GPT-5.5 agent performs 247 graded attempts and increases its score from 42.8 to 67.0 in twelve hours, with seven shifts where it reformulates the problem rather than merely adjusting its settings.