Pablo A. Iturralde (@pablo_itu) profile picture

EEng, BioEng PhD Assoc Prof @ucuoficial Movement Control & Rehab | Machine Learning Tsundoku-er. Se habla español.

Montevideo, Uruguay
Joined October 2011
Photo by Pablo A. Iturralde (@pablo_itu)Photo by Pablo A. Iturralde (@pablo_itu)Photo by Pablo A. Iturralde (@pablo_itu)Photo by Pablo A. Iturralde (@pablo_itu)Photo by Pablo A. Iturralde (@pablo_itu)Photo by Pablo A. Iturralde (@pablo_itu)Photo by Pablo A. Iturralde (@pablo_itu)Photo by Pablo A. Iturralde (@pablo_itu)Photo by Pablo A. Iturralde (@pablo_itu)Photo by Pablo A. Iturralde (@pablo_itu)
Pablo A. Iturralde retweeted
What it's like to watch AI fix a bug
423
2,451
407
20,367
943,490
Pablo A. Iturralde retweeted
The prevailing wisdom is that compute is the most important factor for frontier AI training. We think this is wrong: data is the most costly and important component of AI training. We collected estimates of revenue for major data labeling companies and compared them with the marginal compute cost for training top models in 2024. Our estimates show that data labeling is ~3x higher than the marginal training compute. 1/8
51
156
31
1,121
154,572
"What comes next is not the next spectacular demo but the quiet absorption of today’s tools into the 80 percent of the economy that still runs on Excel and email. Code will ship with more AI-authored lines, but senior devs will still sign the diffs."
I’ve got bad news. The AI cycle is over—for now. I’ve been an unapologetic AI maximalist since the first time I tricked GPT-4 into writing a working Python back-test for a volatility strategy back in early 2023. I’m still convinced it will take the wider economy years—maybe decades—to fully digest the productivity shock we’ve already uncorked. But the curve we’ve been riding just flattened into a long plateau. The problem isn’t that the models stopped improving. It’s that the improvements we need are measured in orders of magnitude, not percentage points. Every step up the scaling laws now demands a city’s worth of electricity and a sovereign wealth fund’s worth of GPUs. You can still squeeze clever tricks out of mixture-of-experts or chain tiny specialists into something that looks like agency; that keeps the demo videos cinematic. It just doesn’t get us to super-intelligence. For that we need either an architectural miracle (unforecastable by definition) or a civil-engineering miracle (a decade-long sprint to build nuclear plants and 2-nanometer fabs). The first is luck. The second is politics. Both are scarce. Meanwhile the models we have remain, at their core, next-token roulette wheels. Chain enough spins together and tiny error probabilities compound into existential glitches. In domains where you can automatically verify an answer—unit tests pass, the protein binds—those glitches are an acceptable tax. You iterate until it works. In domains where judgment is qualitative, the tax becomes fatal. My portfolio example still stings: I asked Claude Opus to locate the optimal weight for a new asset in an already-levered book. Opus quietly renormalized all weights to 1.0, vaporizing the leverage assumption. A layperson would see clean numbers and move on. Scale that failure mode to law, medicine, or national security and you understand why “human-in-the-loop” isn’t a slogan it’s a ceiling. What comes next is not the next spectacular demo but the quiet absorption of today’s tools into the 80 percent of the economy that still runs on Excel and email. Code will ship with more AI-authored lines, but senior devs will still sign the diffs. Customer-support bots will escalate the five percent of tickets that matter; the other ninety-five percent will vanish so smoothly customers won’t notice. Radiologists will still stare at scans, except now a model will have pre-read every slide and flagged the one in a thousand the tired resident would have missed. And yes, you’ll still need a PhD to notice that the portfolio weights got renormalized but you’ll build the new clustering module in five minutes instead of an afternoon. The productivity gains are real; they’re just not cinematic. For founders this means stop chasing the next 0.3 percent on MMLU. Find a vertical where verification is cheap and margins fat, then build the scaffolding that lets domain experts ride the model instead of babysit it. For investors, treat “AI” the way we treated “mobile” circa 2011: infrastructure bets can still clear the hurdle rate if you triple the time discount, but the application layer is a graveyard of demos wearing revenue costumes. For policymakers, forget the AGI manifestos and write zoning rules that let utilities run high-voltage lines to data centers. To my fellow zealots: we are not going back to the pre-2022 world. The ceiling just got higher, but the ladder is longer than we thought. That isn’t failure; it’s physics. The next breakthrough will arrive; maybe from a grad student with a sparse attention kernel, maybe from a national lab running a ten-gigawatt reactor. Until then the boring work of integration is the only game in town. So breathe. Ship the eval harness. Close the ticket. And remember: exponential curves always look flat when you zoom in too close.
1
87
Pablo A. Iturralde retweeted
I’ve got bad news. The AI cycle is over—for now. I’ve been an unapologetic AI maximalist since the first time I tricked GPT-4 into writing a working Python back-test for a volatility strategy back in early 2023. I’m still convinced it will take the wider economy years—maybe decades—to fully digest the productivity shock we’ve already uncorked. But the curve we’ve been riding just flattened into a long plateau. The problem isn’t that the models stopped improving. It’s that the improvements we need are measured in orders of magnitude, not percentage points. Every step up the scaling laws now demands a city’s worth of electricity and a sovereign wealth fund’s worth of GPUs. You can still squeeze clever tricks out of mixture-of-experts or chain tiny specialists into something that looks like agency; that keeps the demo videos cinematic. It just doesn’t get us to super-intelligence. For that we need either an architectural miracle (unforecastable by definition) or a civil-engineering miracle (a decade-long sprint to build nuclear plants and 2-nanometer fabs). The first is luck. The second is politics. Both are scarce. Meanwhile the models we have remain, at their core, next-token roulette wheels. Chain enough spins together and tiny error probabilities compound into existential glitches. In domains where you can automatically verify an answer—unit tests pass, the protein binds—those glitches are an acceptable tax. You iterate until it works. In domains where judgment is qualitative, the tax becomes fatal. My portfolio example still stings: I asked Claude Opus to locate the optimal weight for a new asset in an already-levered book. Opus quietly renormalized all weights to 1.0, vaporizing the leverage assumption. A layperson would see clean numbers and move on. Scale that failure mode to law, medicine, or national security and you understand why “human-in-the-loop” isn’t a slogan it’s a ceiling. What comes next is not the next spectacular demo but the quiet absorption of today’s tools into the 80 percent of the economy that still runs on Excel and email. Code will ship with more AI-authored lines, but senior devs will still sign the diffs. Customer-support bots will escalate the five percent of tickets that matter; the other ninety-five percent will vanish so smoothly customers won’t notice. Radiologists will still stare at scans, except now a model will have pre-read every slide and flagged the one in a thousand the tired resident would have missed. And yes, you’ll still need a PhD to notice that the portfolio weights got renormalized but you’ll build the new clustering module in five minutes instead of an afternoon. The productivity gains are real; they’re just not cinematic. For founders this means stop chasing the next 0.3 percent on MMLU. Find a vertical where verification is cheap and margins fat, then build the scaffolding that lets domain experts ride the model instead of babysit it. For investors, treat “AI” the way we treated “mobile” circa 2011: infrastructure bets can still clear the hurdle rate if you triple the time discount, but the application layer is a graveyard of demos wearing revenue costumes. For policymakers, forget the AGI manifestos and write zoning rules that let utilities run high-voltage lines to data centers. To my fellow zealots: we are not going back to the pre-2022 world. The ceiling just got higher, but the ladder is longer than we thought. That isn’t failure; it’s physics. The next breakthrough will arrive; maybe from a grad student with a sparse attention kernel, maybe from a national lab running a ten-gigawatt reactor. Until then the boring work of integration is the only game in town. So breathe. Ship the eval harness. Close the ticket. And remember: exponential curves always look flat when you zoom in too close.
114
388
156
2,829
621,560
Pablo A. Iturralde retweeted
In order to supervise an automation tool (or another person!) effectively, you need to be able to do the same job yourself. Doesn't matter if you rarely ever do the job yourself (like a manager who no longer codes), you need to be *able* to do it.
what almost every amateur who has been sold bullshit about vibe coding is eventually going to encounter
31
162
26
940
89,781
Pablo A. Iturralde retweeted
The appearance of large language models caused a drastic shift in the vocabulary of academic writing, according to an analysis in @ScienceAdvances of more than 15 million biomedical abstracts published from 2010 to 2024. scim.ag/44IwG8L
13
36
5
126
39,479
“La duda que surge es hasta cuándo este “modelo de convivencia” resiste. No vale defender sistemáticamente a cada corporación que reclama su parte, y luego poner cara de incrédulo y preguntar ¿por qué Uruguay es caro?” Uruguayísima - ⁦@Itu_Agustin⁩ elpais.com.uy/opinion/column…
9
18
1
66
6,383
Pablo A. Iturralde retweeted
More than 50% of the reported reasoning abilities of LLMs might not be true reasoning. How do we evaluate models trained on the entire internet? I.e., what novel questions can we ask of something that has seen all written knowledge? Below: new eval, results, code, and paper. Functional benchmarks are a new way to do reasoning evals. Take a popular benchmark, e.g., MATH, and manually rewrite its reasoning into code, MATH(). Run the code to get a snapshot that asks for the same reasoning but not the same question. A reasoning gap exists if a model’s performance is different on snapshots. Big question: Are current SOTA models closer to gap 0 (proper reasoning) or gap 100 (lots of memorization)? What we find: Gaps in the range of 58% to 80% in a bunch of SOTA models. Motivates us to build Gap 0 models. We’re releasing the paper, code, and 3 snapshots of functional MATH() today. arxiv draft: arxiv.org/abs/2402.19450 github repo: github.com/ConsequentAI/fnev… 1/🧵
42
215
62
1,156
486,714
Pablo A. Iturralde retweeted
Un nuevo proyecto para 2024: Inteligencia Artificial y Ciencia de Datos en @UCUoficial. Un gusto haber trabajado con @VazquezGust y @pablo_itu en este desafío. Una carrera específica en #IA y #Datos. youtu.be/wihFQ0LamOk?si=yCHp… a través de @YouTube
4
19
633
Pablo A. Iturralde retweeted
I like having no strong opinion on most things. Having an opinion about the topic of the day would require me to know a decent amount about it, and given the endless ocean of things there are to opine about, that would represent an immense loss of focus and waste of energy.
19
86
13
777
111,550
Pablo A. Iturralde retweeted
☺️La Academia Nacional de Ingeniería de Uruguay 🇺🇾 entregó sus premios 🏆anuales a los mejores trabajos de grado y posgrado. Nuestra #AlumniUCU Natalia Martínez resultó premiada 🥇por su trabajo en la categoría trabajo de grado en Ingeniería en Electrónica 👏 ¡Felicitaciones 🥳!
4
11
Pablo A. Iturralde retweeted
Finalizó la 9a Olimpíada de Robótica, Programación y Videojuegos con más de 2.300 estudiantes y 308 docentes que participaron, investigaron e integraron conocimientos a problemáticas cercanas. #OlimpiadaCeibal 🏅

7
44
4
205
Pablo A. Iturralde retweeted
This is a cyclist using his knowledge of physics and aerodynamics to gain a competitive advantage Michael Guerra unclips from his pedals, lies down on the seat and stretches out his legs to achieve ultimate aerodynamic efficiency.
1,962
11,784
4,180
105,682
Pablo A. Iturralde retweeted
All of linear algebra in one diagram:
60
1,171
91
6,173
Pablo A. Iturralde retweeted
Gran mesa con tremendos expositores en el 🚀 lanzamiento de la Maestria en Ciencia de Datos de @UCUoficial
1
5
29
Pablo A. Iturralde retweeted
Proxy Variable xkcd.com/2652
37
703
175
7,181
Pablo A. Iturralde retweeted
The public are excited by the images, but astronomers are as excited by the spectra, like this one. I want to talk about why
219
5,235
569
31,817
Pablo A. Iturralde retweeted
YYYY-MM-DD is the only correct date format. * It's an ISO standard: en.wikipedia.org/wiki/ISO_86… * It's unambiguous (is 03/05/22 March 5 or May 3? No such problem here!) * General-purpose sorting sorts dates correctly * East Asian countries like Japan 🇯🇵🇯🇵🇯🇵 use it ("2022年5月3日")
942
738
240
5,818
"The crucial lesson was that the scope of things I didn't know wasn't merely vast; it was, for all practical purposes, infinite. That realization, instead of being discouraging, was liberating. If our ignorance is infinite, the only possible course of action is to muddle through"
I remember breaking down to my Ph.D. advisor about how stupid I felt while troubleshooting a problem in my project when she handed me this paper. Years later it is still relevant to young students starting off in Science. It's ok to feel stupid, we all do on a regular basis.
2
Pablo A. Iturralde retweeted
This new absolutely killer feature is now in #overleaf!
Hey everybody! 🤩 We are very excited to announce a brand new feature to rock your workflow: the PDF detach! 🛠️ 🎉🎈🎸 You can now see the PDF preview of your project in a separate tab! 📑 💻 Check out our blog post: overleaf.com/blog/new-featur… #TeXLaTeX #Overleaf #OverleafBlog
1
8
4
75