Stanford and Google DeepMind build TURTLE to make AI benchmarks evolve

The system uses AlphaEvolve to make tool-use tests harder while keeping them solvable, so benchmarks can keep pace with the models they measure AI models keep acing the tests built to measure them. Researchers at Stanford University and Google DeepMind think the fix is to let the tests fight back. Their new system, TURTLE, takes existing tool-use benchmarks and evolves them into harder versions. The key constraint is that every task must stay solvable, so the result is a tougher exam rather than a broken one. How TURTLE turns up the difficulty TURTLE stands for Tool Use Related Tasks with Long-Horizon Evolution. The work is described in an arXiv paper dated October 19, 2026, listed as arXiv:2610.27307. It focuses on tool use, meaning how well an AI agent can operate software to finish real tasks across many steps. Under the hood, TURTLE relies on three cooperating agents. A proposer, built on DeepMind’s AlphaEvolve evolutionary coding agent, suggests changes to a task. A solver model then attempts the modified version. A feedback generator reviews what happened and steers the next round of edits. The mutations take several forms. TURTLE can add new filters, insert extra steps, or b...