Kaggle

That’s a wrap on Bench Summit by Kaggle! ✨  Thank you to our partners at @zapier and @GoogleDeepMind, and to everyone who joined us and made this inaugural event a success! Stay tuned: we’ll be rolling out talk recordings and a digital track soon…

The afternoon breakout sessions at Bench Summit have wrapped up! 🏔️ Benchmark builders and AI researchers covered topics ranging from multi-agent systems to security, for example: • “The Utility of Multi-Agent Systems in Benchmarks” by Om Buddhdev (@sensho) from Olam Labs • “Superhuman Evals for Superhuman Intelligence” by Xiangyi Li (@xdotli) from BenchFlow • “We Gave 7 Frontier Models Real Businesses. They sent $12,431 in Fake Invoices, spammed thousands of emails, and made $0” by Alex Reibman (@AlexReibman) from Bottleneck Labs • “Reproducing the Hugging Face incident” by Kevin Lin (@quasicoh) from Abundant

How do competitors with restricted compute find opportunities to win, build their careers, and earn medals? Last year, Isaiah Pressman finished 2nd in a competition on his home PC, behind a team running 8 H100s. This year, he won on a cluster. In the first episode of Beyond the Leaderboard, three Orbit Wars competitors weigh scale against clever engineering. Here’s a sneak peek:

Could we get more from a quantum computer by asking it to do less? ⚛️ Verifiable Quantum Circuit Compilation, hosted by @GoogleQuantumAI and Kaggle, is live! Build search and AI methods to compile quantum circuits for real-world bottlenecks in chemistry, fusion energy, and numerical analysis. 💰 Prize Pool: $150,000 ⏰ Entry Deadline: January 13, 2027

How tightly can you pack (n) squares into a square? 📦 Lining squares up in a grid often wastes space, and tilting just a few of them at the right angle can make the box smaller. In Squares in Squares, you'll pack 1 to 1,000 unit squares into the smallest square container you can. Only 59 of those puzzles have proven best answers, so the rest are open. 💰 Prize Pool: $50,000 ⏰ Entry Deadline: July 11, 2027 🏆 Prize Milestones: At 3, 6, and 9 months

Can frontier models actually predict how real people behave? 🕺 Today, in collaboration with @RunRehearsals, we are launching Human Behavior Bench, the first public benchmark that tests whether AI can understand human preference and everyday decision-making. AI models have gotten increasingly good at coding, but knowing for whom to build is the next frontier of challenges for AI models to capture. Human Behavior Bench evaluates models on predicting real human behavior across health, entertainment, recreation, values, and consumption. To build this baseline, we gathered ground truth data directly from real people and evaluated models across a wide range of everyday scenarios: • Ground truth baseline: 1,244 US adults surveyed (June-Sept 2026) • Some cohorts are traditional demographic indicators such as age, income, and location • Other filters are behavioral e.g. GLP-1 patients and recent car buyers • 750 cases total (149 public / 601 private) across finance, health, recreation, values, and consumption • Scored using 1-TVD (total variation distance) Base models still have a significant gap to close to accurately predict human behavior by cohort. LLMs’ predictions tend to converge towards the mean, while the true distribution of human preferences vary greatly. Against a human noise ceiling of ~93, the current leaderboard scores are: Rehearsals’ Engram 1.0 (AI Twins) tops the leaderboard at 94.1%, followed by Gemini 4 Argon at 80.6% Claude Opus 5 at 80.2%, GPT-5.6 Sol at 79.5%.