Mark Randall

Today we're announcing the winners of ARC Prize 2024. We're also publishing an extensive technical report on what we learned from the competition (link in the next tweet). The state-of-the-art went from 33% to 55.5%, the largest single-year increase we've seen since 2020. The benchmark remains unbeaten, but we're happy to see that research progress on the key bottleneck to AGIs (in particular on-the-fly adaptation to novel tasks) has been reignited in 2024 -- in part thanks to ARC Prize. In particular the competition has popularized Test Time Training (TTT), originally pioneered for ARC-AGI by Jack Cole last year. I believe TTT represents the largest jump in LLM generalization capabilities since the initial findings regarding in-context-learning circa 2019-2020. ARC Prize has also led to a considerable surge of research interest towards program synthesis. Competition winners: 🥇 the ARChitects (Daniel Franzen, Jan Disselhoff) 🥈 @guille_bar 🥉 alijs (Agnis Liukis) Paper Award winners: 🥇 "Combining Induction and Transduction For Abstract Reasoning" by @xu3kev et al. 🥈 "The Surprising Effectiveness of Test-Time Training for Abstract Reasoning" by @akyurekekin et al. 🥉 "Searching Latent Program Spaces" by @ClementBonnet16 & @MattVMacfarlane ARC-AGI-Pub Leaderboard (solutions using commercial APIs): 🥇 @jeremyberman 🥈 @ellisk_kellis & @akyurekekin 🥉 @ryangreenblatt

Within 10-20 years, nearly every branch of science will be, for all intents and purposes, a branch of computer science. Computational physics, comp chemistry, comp biology, comp medicine... Even comp archeology. Realistic simulations, big data analysis, and ML everywhere