AIRA₂ tops ML benchmark, outperforms prior agents by 9 percentage points

An AI agent just beat roughly 3,992 human teams at their own game. AIRA₂, an autonomous AI research agent developed by researchers at Meta’s FAIR lab, University College London, and the University of Oxford, placed 8th out of 4,000 teams in a Kaggle competition focused on AI reasoning, earning a gold medal in the process.

What AIRA actually does

The AIRA line of agents (the name stands for AI Research Agent) is designed to autonomously tackle complex machine learning engineering problems. Rather than just running a single model and hoping for the best, AIRA uses an asynchronous multi-GPU execution strategy running on 8 Nvidia H200 GPUs.

The system employs what the research team calls a “Hidden Consistent Evaluation” protocol, a method to prevent the agent from gaming its own test scores. The agent also uses dynamic ReAct-style operators, meaning it can reason through problems step by step, adjust its approach on the fly, and execute code across multiple processors simultaneously.

On MLE-bench-30, a structured benchmark built from 30 real Kaggle competitions, AIRA₂ achieved a mean percentile rank of 81.5% after 24 hours of computation. Give it 72 hours, and that number climbed to 83.1%. The best prior baseline from other agents sat at 72.7%, making AIRA₂’s improvement roughly 9 percentage points above the competition.

Why this matters beyond bragging rights

AIRA₂ exceeded human state-of-the-art performance on 6 out of 20 tasks in the AIRS-Bench evaluation. It also won gold medals on individual Kaggle tasks where earlier versions of the agent failed to secure any awards at all.

The framework builds on what the team calls the AIRA-dojo approach, designed specifically to address limitations of earlier agents. Those limitations included limited computational throughput, evaluation overfitting, generalization gaps across different problem types, and static operational constraints that prevented the agent from adapting its strategy mid-task.

The competitive AI research landscape

The collaboration between Meta FAIR, UCL, and Oxford is notable for its institutional weight. The research team disseminated its findings through arXiv preprints, the standard channel for cutting-edge AI research.

Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our Editorial Policy.