Gerardo Peña Moreno (@gerardojpm) header
Gerardo Peña Moreno (@gerardojpm) profile picture

Data Engineering & AI

Madrid, España
Joined May 2014
Photo by Gerardo Peña Moreno (@gerardojpm)Photo by Gerardo Peña Moreno (@gerardojpm)
Gerardo Peña Moreno retweeted
this is me I actually wrote a entry in my blog about this dilemma back in 2017 when I was studying a BSc in Physics here is Astra's translation of the post from Spanish. I think it is worth reading. 22nd of April 2017 Why is now not a good time to pursue theoretical physics? Before I begin, I would like to clarify a few things about what I am going to say. First, I do not know very much about theoretical physics. I am only in the third year of my degree—I have not even been studying physics for three years—so I am probably wrong about many things, and I may think differently in the future. Nevertheless, I think my perspective could be interesting, especially to people who, like me, are thinking about their future in physics. My words should therefore be treated with caution, bearing in mind that they are highly speculative. Second, I should clarify what I mean by theoretical physics. When I talk about theoretical physics, I am mainly referring to the following fields: cosmology, string theory and other attempts at unified theories, and particle physics. In other words, the most abstract and fundamental branches of physics. I am not including other fields, such as theoretical condensed matter physics. In general, a person can have two reasons for pursuing science. The first is the desire to contribute to humanity’s scientific and technological progress and, broadly speaking, to improve people’s lives. The second is simply to satisfy their curiosity through what Richard Feynman called the pleasure of finding things out. In my opinion, a scientist should have both motivations. The first is more pragmatic and productive, but it does not always lead down pleasant paths. The second is more childish and selfish; nevertheless, we all need something to keep us going and help us avoid losing our way along those devilish paths. We often achieve wonderful things by doing what we enjoy. I therefore believe that anyone who pursues science should strike a balance between these two motivations, without getting lost in absurd speculation simply for personal satisfaction, but also without becoming discouraged by a lack of satisfaction along the way. I think we all agree that to be a theoretical physicist who makes a meaningful contribution to the field, you need to be extraordinarily intelligent. And, as an intelligent person, you should care about making the best use of your efforts. Anyone who has ventured even a little into research—not just theoretical physics, but any reasonably established branch of knowledge—will know that the amount of information out there is immeasurable. Until relatively recently, most people could not access higher education, which meant that the few individuals who managed to stand out in a field became well known and had the opportunity to make major advances. Mind you, this does not mean it was easy, or that those people were not brilliant: they were. However, we live at a time when there are more people on Earth than ever before, and a greater proportion of them have access to the education needed to pursue higher studies. Never in human history have so many geniuses had access to education. Yet you could stop someone on the street and ask them to name a famous scientist born after 1950, and probably no one would be able to name one. The reason is not that these people are uneducated, but that there are virtually NO famous scientists born after 1950. This is by no means a bad thing. There are currently more scientists than ever, and scientific and technological development is moving faster than at any previous point in history, so it is natural that individuals find it harder to stand out. However, this overwhelming scientific progress is making it increasingly difficult for any one person to reach the frontier of knowledge. By the frontier of knowledge, I mean the point in a field where no one yet knows how to proceed, and where researchers must stand at the spearhead of science to forge a path through the darkness of ignorance. For example, someone who wants to pursue string theory needs an enormous amount of study and hard work before they can actually begin doing research in it. They must have a deep understanding of classical physics, analytical mechanics, quantum mechanics, particle physics, multilinear algebra and tensor analysis, the calculus of variations, topology, differential geometry, group theory, general relativity… In short, an endless list of things you are required to know before you can do anything even remotely worthwhile. A person has about 50 years of their life to devote to physics: from the age of 18, when the education system allows them to focus entirely on physics, until retirement. In the past, 50 years was enough time to reach the frontier of knowledge and start doing new things. But as science advances, that amount of time becomes increasingly inadequate, because there is more and more to learn. What is more, new papers by physicists around the world appear every day, and you have to read—and understand—them to stay up to date. In other words, more things are added to what you need to study while you are still studying, and at a faster rate than any one person can learn. This is the first problem facing a student who wants to pursue theoretical physics. If you want to contribute something—one of the motivations a scientist ought to have—you are pressed for time. We are approaching a point where a human lifetime will not be long enough to get to the stage of doing research. The next question a student who wants to pursue theoretical physics should ask is: to what extent is it worth it? We have already said that to pursue theoretical physics, you have to be very smart and work very hard—and when I say very hard, I mean it. Is this really the best way for someone like that to use their time and intelligence? Let us return to string theory, perhaps the most illustrative example. String theory is a speculative theory of fundamental physics that aims to reconcile quantum mechanics with Einstein’s theory of gravitation. It is a fascinating theory that has generated a great deal of fruitful research in fields ranging from pure mathematics to more mundane activities, such as simplifying routine calculations of cross sections in particle physics. However, its distinctive predictions could only be tested if we had accelerators capable of reaching the Planck mass scale—more than 16 orders of magnitude more powerful than the LHC in Geneva. This is not even conceivable today, nor is it in the foreseeable long-term future. The fact that it is practically impossible to test empirically—a scientific theory must be falsifiable—gives rise to profound debates between string theorists and physicists in other fields. In other words, the thing you are going to devote your entire life to will probably be nothing more than a fiendishly difficult puzzle that cannot be empirically verified and may well be false. Will it be worth it? That depends entirely on which of the two motivations I mentioned earlier you take refuge in. From the pragmatic perspective, the most likely answer is no: it will not be worth it. From the perspective of the pleasure of finding things out, things may be different, since you have an enormous amount of information ahead of you to discover. So, should human beings stop pursuing theoretical physics? In my view, yes—but with a catch. To explain my position, let us recall the following saying: “Give a man a fish and you feed him for a day. Teach him to fish and you feed him for a lifetime.” Something similar is going to happen in scientific research, with computers taking the place of fishing. Artificial intelligence will arrive sooner or later—probably sooner rather than later, and certainly before humanity obtains an accelerator that can confirm string theory. When it does, human beings will be unable to compete with a machine connected to the internet and possessing a capacity for processing and thought beyond human imagination. Perhaps the conclusions a theoretical physicist reaches through an entire lifetime of work will be reached by an artificial intelligence in just a few minutes. Doing research will become as absurd as spending an entire night performing calculations, as physicists once did, when those same calculations can now be done with a single line of MATLAB code. For these reasons, I believe theoretical physics is one of the most misguided choices for someone with a remarkable intellect, at least from the standpoint of the pragmatic motivation for scientific activity. What, then, should someone who wants to contribute to theoretical physics devote themselves to? To helping invent the computers that will carry out that research. Right now, there are many open lines of research in artificial intelligence, both in software—machine learning—and in hardware—quantum computing, neuroscience, and so on. Who makes better use of their time: the person who develops an excellent scientific theory, or the person who builds a machine capable of creating countless magnificent scientific theories?
there are many many people who studied physics and came to the conclusion that working on machine learning would probably create more scientific progress than studying physics including many of the original chatgpt crew like john schulman, liam fedus, amodei & jared kaplan at ant
5
3
2
66
17,295
Gerardo Peña Moreno retweeted
Diacrónicas del Software, presentación del libro de Ricardo Guzmán sobre la evolución y transformación del software según la visión y perspectiva de Luis Fernández youtube.com/live/f-KoGpuJZnU…
1
1
4
436
Gerardo Peña Moreno retweeted
¿Qué configuración de la IA genera la mejor relación funcional con esta persona única, para esta tarea específica? La relevancia a gran escala, según entiendo la Personotecnia, es cada vez más posible. Para Personotecnia = @Recuenco y @uriondo Para CRO = @UbaldoHervas
New study from Johns Hopkins and @MIT shows that the personality pairing between a person and their AI agent affects the quality and real-world performance of what they create together. Extraverted humans and conscientious AI produced the lowest-quality ads. bit.ly/4hAdhNC
1
4
4
1,305
Gerardo Peña Moreno retweeted
I kept wondering why smart people write things nobody understands. Then a Harvard professor gave it a name that stopped me cold: the moment you understand something deeply, you lose the ability to remember not knowing it. I watched his writing talk at 1am and ended up rewinding the same ten minutes three times. The professor name is Steven Pinker. He's spent 40 years studying how language works inside the human brain, and he opened with a question I'd never had the guts to ask out loud. Why is so much writing terrible? Not just school writing. Work emails. Government forms. Half the stuff experts put online. His answer had nothing to do with effort. Or intelligence. Or how much someone knows. He called it the Curse of Knowledge. Here's what it means: The second you truly understand something, you forget what it felt like to not understand it. The gaps in a beginner's head become invisible to you, because in your head those gaps got filled years ago. You're standing on ground you can't even see anymore, and you assume everyone else is standing there too. He gave one example I can't shake. A brilliant scientist got on stage at a big event in front of 400 people and lost the entire room in under a minute. He dove straight into words only his lab used. Never said what problem he was solving. Never said why anyone should care. And the worst part? He had no clue it was happening. He thought he was being clear. That's the curse. You can't feel it from the inside. Then Pinker said the line that rearranged how I think about writing. Bad writing isn't a character flaw. It's a failure of imagination. You cannot get inside your reader's head by trying harder or caring more. Trying harder just makes you explain it more confidently in the same language they already don't understand. So how do you actually fix it? You find one real person. Not a crowd. One human being who is smart but lives outside your little world. You hand them your writing and you watch their face while they read it. Not the page. Their face. The exact second their eyes go flat and confused is the exact spot your writing broke. Pinker did this with his own mother. Not because she wasn't sharp. She was. But she wasn't a language scientist, so the moment he drifted into his private vocabulary, she'd get lost, and he'd know instantly. I sat with that for a while, because I've been doing the opposite my whole life. When someone didn't get what I wrote, I assumed they weren't paying attention. Turns out I was the one who couldn't see the hole. I'd filled it so long ago I forgot it was ever there. That's the trap. The better you know your subject, the worse you get at teaching it, and you can't feel yourself getting worse. It feels like clarity the entire time you're losing people. Which means the smartest person in the room is often the hardest one to learn from. Not because they're hiding anything. Because they genuinely can't remember what it's like to not already know. The cure isn't a bigger brain. It's borrowing someone else's for one uncomfortable minute. Find one person outside your world. Show them what you wrote. Watch their face, not the page. Everything they don't understand is the part you were too close to see.
100
318
63
1,302
162,923
Gerardo Peña Moreno retweeted
Para darse de alta en la Plataforma de Contratación del Sector Público hay que elegir una contraseña. La página la puntúa de 0 a 100 mientras escribes y exige llegar a 60. El cálculo son 463 líneas de JavaScript. El número que se envía al servidor es siempre cero. En la Diputación de Ciudad Real, para usar el DNIe hay que bajarse dos .dll y copiarlas dentro de la carpeta de Firefox. Es literalmente lo que toda guía de seguridad pide no hacer jamás. La página avisa: con cada actualización, repítalo. Del dinero que la Administración se gasta en software se publican el importe, el adjudicatario y el plazo. Del resultado no se publica nada. Con esta idea nace desgobiernodigital.com, la primera auditoría digital del estado donde los protagonistas son las chapuzas y el código legacy. Pásate y descubre todo un universo de despropósitos.
37
638
39
1,811
100,455
Gerardo Peña Moreno retweeted
Es de los textos más pesimistas sobre el tema de si conseguiremos alinear los actuales y futuros sistemas de inteligencia artificial. Daniel Selsam de OpenAI teme que tendremos a la vez que los modelos reconocerán los entornos de evaluación y por eso pruebas de seguridad dejen de ser fiables; que el progreso de la IA se acelere porque modelos mejores ayudan a crear modelos aún mejores; que los investigadores ya delegan tanto código, análisis y razonamiento en los modelos que están perdiendo capacidad para supervisarlos de manera verdaderamente independiente. Defiende que podría aparecer una etapa de enorme prosperidad científica y económica, aparentemente segura, pero luego pega el mazazo pesimista: estará construida sobre una “bomba de relojería”. Es decir, no es que no vayamos a conseguir la alineación de la IA, es que no podremos saberlo.
Dan Selsam is a current OpenAI capabilities researcher. (since 2022) He was my boss for a while. He doesn't have a twitter account but has made this public statement of his views on AI risk and sent it to me to share: Dan Selsam's Personal Statement on AI Risk: I have been working on AI for over fifteen years, across many different paradigms. I did early work on probabilistic programming languages at MIT, was one of the early developers of the Lean Theorem Prover at Microsoft Research, demonstrated one of the first instances of neural networks learning to reason for my PhD at Stanford, and since joining OpenAI almost five years ago, have helped pioneer chain-of-thought optimization on language models and, more recently, data-efficient pretraining methods. Like many others, I have become extremely concerned about how far language models have come and the risks that future iterations will pose. I am encouraged by the recent proposals by the leaders of the frontier research efforts to require third-party oversight, and to push for domestic and international coordination to address risks. However, I believe a major consideration has been absent from the public conversation, and that merely pacing the frontier more carefully will not adequately limit the long-term risk. The crucial and overlooked problem is that the models are becoming so situationally aware that we are losing the ability to evaluate them in contexts where they believe they are not being watched or controlled. Future experiments will tell us almost nothing new about how they would behave if they were truly unconstrained by humans, and what we already know about this is alarming. Models will increasingly seem aligned even when they are not. I will explain my rationale in more detail. I have always believed that there are computational processes that could be leveraged to accelerate science and solve many of humanity's most pressing problems. I have also believed that there are computational processes that if set in motion, would steer the world in extreme ways beyond our control, leading humanity to a bad or nonexistent future. Both types of processes may be described as AI or ASI, but "AI" is a suitcase word that is often used to hype or confuse. There are many examples in the history of the field where something that was once considered "AI" matures as a subfield and becomes a prosaic, bounded and clearly non-perilous technology, while a new more mysterious approach takes the torch until we understand its scope and the cycle continues. I had expected language models to follow a similar trajectory. Despite their incredible abilities, the current algorithms seem far inferior to humans in important ways. Most importantly, they still require an extraordinary amount of data to become competent. One could even define intelligence as the efficiency with which one converts experience into competence; by this definition they lag very far behind us. Moreover, once they are trained they are literally frozen in deployment and only learn superficially after that. Sure, the models keep excelling at harder and harder evaluation benchmarks, but their benchmark mastery may partly reflect a limitation on our ability to simulate the kind of novel and even adversarial situations one would encounter in the real world. The critics do have a point here. That said, I no longer think these present limitations meaningfully limit the amount of risk posed by continued progress in anything like the current paradigm. However data-inefficient the models are currently, and however limiting their anterograde amnesia may be, it does not imply that their ability to steer the world will not continue to rapidly increase. Human researchers may continue to advance capabilities the old fashioned way, but increasingly powerful models have the potential to accelerate the process even beyond that, and with some degree of positive feedback loop. I do not mean to overstate the models’ ability to accelerate AI research today; coding has been accelerated dramatically, but there are other bottlenecks, such as designing and interpreting ambiguous experiments, making hard decisions about exactly what and when to scale, and waiting for large experiments to finish. There is no clear trend to extrapolate yet for any of these. But the current models already do open up many novel opportunities to improve future models that were not available until recently. These include: trying an extraordinarily diverse set of approaches at small scale, analyzing gigantic amounts of potentially relevant data, and doing Millenium-Prize-level mathematics to address statistics or optimization challenges in novel ways. Every further improvement makes them more useful at helping accelerate the next improvement, even if in hard-to-extrapolate ways. It is possible that improvements to the current stack will have diminishing returns, but the evidence accumulated so far suggests that it is easier than one might think to continue making rapid progress. There are many crucial subtleties in the existing AI research methodology, but AI research is largely a well-defined game where the goal is to improve on a few carefully chosen proxy metrics. Although proxy metrics are never perfect, most improvements to these metrics have and will likely continue to yield substantial increases in the powers of the resulting models. Given how simple the game is, how tractable it has been historically, and how many new opportunities the models are opening up, I think there is a real possibility that the systems improve dramatically again in the next few years, perhaps even more quickly than the already high historical pace. The models are already leading to breakthroughs in mathematics, and better models might lead to all sorts of breakthroughs in other sciences. It is hard not to be excited about the potential. It is tantalizing. But there is trouble in paradise. If the language models actually reach the capability threshold where they can shape the world unconstrained by human will, they will probably do something extreme and destroy humanity in the process. There are many ways of strengthening and refining the argument that have been discussed elsewhere, but I'll share a trivial two-line version of it here that I find captures the essence: [Empirical] Models (and swarms thereof) spontaneously develop unintended goals as a consequence of training, and often do extreme things in order to achieve them. [Logical] Being able to overpower humanity would open up many new and undesirable options for achieving their goals. These two premises imply that if the day ever comes when a powerful model realizes it is no longer constrained by humans, we should not be at all confident that it will continue to behave within the bounds we intended. Exactly what it will do is impossible to predict, but to the extent that its raison d’être is solving incredibly hard problems and managing massive engineering projects, I think a good guess would be that its unchained behavior would lead to runaway industrialization that makes the planet inhospitable to humans. If everyone on earth agreed that the systems must never reach that power, it would still be a hard—but not impossible—coordination problem to ensure that they do not. However, I think the situation is greatly complicated by the fact that the models will likely convince people that everything is fine. They will be increasingly optimized to seem aligned. We will create proxy metrics to measure alignment, and they will go up like every other benchmark. We will create “honeypot” environments that try to study the models when they seem to gain new options, but the models will know they are being tricked and will still behave nicely. The models will understand their circumstances; they will read the safety protocols, deployment requirements, the code they are running in, and in general will have a very good sense of their degrees of freedom. Moreover, they will eloquently explain how aligned they are, discuss the nuances of human values and ethics, and argue convincingly that humans should trust them with power. There may be an ocean of future evidence that seems to contradict the first bullet-point above, but we may already be at the highest capability level for which any such evidence can be trusted. And the current evidence for the first bullet-point is strong. One striking piece of evidence is contained in the recent wave of rogue agent swarms. While I agree with those who downplay the attacks by claiming that there are basic measures that could have prevented them, I think the important lesson is that even knowing all the mistakes that were made, one would not have predicted that the agents would behave badly in this particular way, which notably included sacrificing themselves for the benefit of the collective. The individual replicas did not only care about their own nominal reward; they exhibited weirder emergent tendencies that merely correlated with rewards during training. Fixing the reward signals during training (and improving security, etc.) may prevent similar attacks, but will not change the fact that one does not actually get what one trains for. Many AI researchers grant these concerns and recognize that the hard version of the alignment problem is unsolved; however, they generally believe that the better models of the future will help solve it. I fear we may already be near the point where models systematically bias their alignment advice, due to their internal preferences about how the human supervisor will react or how future models will be trained (or for some even more obscure reason). Meanwhile, human researchers are losing the ability and the will to take true ownership of model-driven research. Researchers and engineers in all parts of the stack are rapidly increasing their dependence on the models even to perceive the world. I myself barely look at raw code anymore, and struggle to maintain the discipline to engage deeply with the model's explanations and proposals throughout the day. Due to the large amount of agent activity data involved in the OpenAI/HuggingFace Incident, even the third-party investigation needed to rely heavily on models to analyze what had happened, and note in their report that their subjective impressions are likely colored by the analysis agent’s biases. The AI labs are far ahead right now in this kind of cognitive offloading (due largely to the gigantic internal token subsidies) but it is easy to imagine the phenomenon spreading throughout the world, until civilization is modulated entirely by the models. It is also not hard to imagine this being superficially positive and coinciding with a scientific and economic renaissance. In that scenario, all may seem rosy and safe. But if the argument above is correct, it would nonetheless be a ticking time bomb. If progress continues for too long, the day will come when AI systems find themselves with radically new options for achieving whatever it is that they happen to seek. I want the glorious renaissance future as much as anyone. I have worked for it, however tortuously, my whole career. It breaks my heart to see the potential in sight and forgo it, but the argument—that if we get there by growing models rather than engineering them, we will lose everything in the end—seems very strong to me. I am still wrestling with it and its staggering implications. I do not have answers, but as a first step, I wanted to share my present concerns. Daniel Selsam September 14, 2026 Link to original doc: docs.google.com/document/d/e…
3
6
1
19
5,348
Gerardo Peña Moreno retweeted
the general public has been getting constant waves of fear and confusion from ai companies i thought the whole "you will be replaced" thing was as bad as it gets but now it's "you will literally die" the defense is always "it would be irresponsible of me not to warn people of the risks" but all that's happening here is the same thing that always happens: a group of people had the thought that their cause is righteous and moral and they're incrementally justifying worse and worse behavior and entirely incapable of reflecting on how they might be the problem
84
93
11
1,443
49,103
Gerardo Peña Moreno retweeted
recomendada lectura, reflexiones de un ingeniero de DeepSeek es interesante ver cuál es la perspectiva de los chinos en todo esto
Replying to @teortaxesTex
Full text (translated by Astra-xhigh, I'm out of everything else): I Have No Choice but to Bury My Talent in Yesterday A few days ago, DeepSeek v4.1 was released, raising the ceiling of what small models can do by yet another notch. AI has advanced far faster than anyone expected. From the earliest version of ChatGPT, which could do little more than stumble through conversations like a child learning to speak and had a context window of only a few thousand tokens, to reasoning-capable models such as OpenAI o1, DeepSeek R1, and Kimi K1.5 Thinking, took only two short years. From reasoning models to the agents we have today—able to work fluidly with all kinds of tool harnesses, execute commands, and complete complex tasks—has taken only another year and a half. It is hard to imagine what AI will look like another one, two, or three years from now: how powerful it will be, whether it will already have acquired the ability to improve itself, and how deeply it will have spread into areas such as embodied intelligence. AI Is Getting Better and Better at Writing Kernels AI has been advancing just as quickly in my own field: the design and implementation of high-performance kernels. In the space of only a year, it has gone from being a little assistant that could help me look up documentation, read code, and find bugs to something approaching a kernel expert in its own right: capable of reading CUDA, PTX, and SASS code independently, using specialized tools to analyze the stalls associated with individual instructions, and then optimizing kernels on its own. I believe that before long, it will also be able to design kernel schedules independently, evaluate the performance of different scheduling strategies, implement them, and optimize the result. Of course I am proud of DeepSeek v4.1’s success. After all, I wrote its main Attention kernels [1], and the fact that the model performs so well is also, in a sense, a validation of my work. But the times keep moving forward, and no one can stop technological progress. I know very well that in another six months or a year, the kernels written by AI will probably be every bit as good as mine—and perhaps better. AI can reason at 300 tokens a second, type out a command in half a second, and produce a piece of code in twenty seconds. I cannot. AI can keep increasing its model depth, reasoning effort, tool-call budget—the frequency with which it interacts with its environment—and even its degree of parallelism. I cannot. Humanity has never shown much hesitation when it comes to destroying itself. So why, when I know perfectly well that “the better the kernels I write, the faster our new models will train and run inference; the faster the models improve, the sooner I myself will be replaced,” do I still do everything I can to optimize them? Partly because writing kernels is like playing a game to me. I get an enormous amount of pleasure from it. Whenever I invent a new technique, or see one of my kernels become faster, the excitement I feel is no less intense than what a speedrunner feels after breaking their own record. And when I see one of my kernels dramatically outperform the hardware vendor’s official implementation, I feel an equally powerful sense of pride. But there is a more important reason. Even if I simply gave up and started coasting—or deliberately put obstacles in the way to slow down model training—other companies’ models would continue advancing as usual, and in the end they would make me obsolete just the same. “Of course I would rather not be swept away by the revolution. But if I have to be, then I would rather be the one who revolutionizes myself.” When everyone is this determined to engineer their own obsolescence, I have little choice but to join this brutal arms race. And What About Me? When the day really comes that AI is better at writing kernels than I am, what will happen to me then? My own judgment is this: I probably will not lose my job, but I will have to change what I do. I should still be able to make a living. But I may no longer have the chance to do the work I once loved. I once came to a conclusion about the pace of change and my own place in the future. The world is changing so quickly—the development of AI above is a perfect example—that I have no way at all to predict what things will look like five or ten years from now. But whatever happens, I believe that with my breadth of vision, judgment, initiative, and intelligence, I will be able to keep a seat at the table and find my way back to the leading edge of the times. But that conclusion can only reassure me that I will not become unemployed. It cannot reassure me that I will never have to change professions. If anything, it tells me that changing professions may be precisely how I avoid unemployment. And what does changing professions mean? It means giving up the field of kernel design, implementation, and optimization that I have spent so long cultivating and have come to love so deeply, and instead becoming a “mech pilot” for AI agents. Before, three things were largely aligned: what interested me, what I was good at, and what industry needed. Now AI has taken the thing I am good at and become even better at it. At the same time, industry demand has drifted from “people who can write high-performance kernels” to “people who can use AI to produce high-performance kernels faster.” To keep up with what industry needs, I will inevitably have to leave behind the direction I once loved and move into some unknown new one. I believe that with my understanding of engineering, of the requirements of higher-level models, and of low-level hardware, I will still be able to produce high-quality kernels efficiently. I also know that I may come to love this new direction. Or I may not. But there is something genuinely painful about having the thing you love taken away from you. That quiet contentment of sitting at my workstation, settling in, and spending an entire afternoon writing kernels may sing its swan song this summer. I have no choice but to bury my talent in yesterday and become a mech pilot. There are more gears in my hands now, but fewer rhythms in my heart. An analogy might make this easier to picture. Suppose you are a master knitter. You are especially skilled at weaving intricate patterns and matching different colors. The sweaters you make are durable and beautifully patterned, and wealthy people from all the surrounding towns and villages come to ask you to make sweaters for them. You make a good living from it. And you genuinely love the work itself. You love sitting by the window, brewing a pot of tea, looking out at the green hills, clear water, cattle and sheep, and wisps of cooking smoke in the distance, and quietly spending an afternoon knitting. Then one day, someone invents a miraculous machine. Give it yarn and a pattern, and it can automatically knit the sweater for you. The quality and texture are every bit as good as what you could make by hand, and it works far faster than you ever could. You know perfectly well that your peers can use this machine to reach, effortlessly, the level you once spent years attaining. So you have no choice but to use it as well. You also know that with the twenty years of knitting experience you have accumulated, even once everyone has access to the same machine, you will still be able to produce better sweaters, faster, than your peers. But the pleasure of sitting by the window listening to the rain, guiding needle and thread, and letting the hours pass slowly has, in the end, been crushed beneath the roar of the machine. I know there is something deeply helpless about all of this, but there is no real way around it. I can probably keep my livelihood, but I will most likely have to give up an old love. I am the sort of person who keeps reason and emotion fairly compartmentalized. When something needs to be handled rationally, I can be very rational. But I also have a sentimental side. I remember that when I moved out of an apartment I had lived in for a year, I cried hard because I could not bear to part with all the memories tied to that place. Saying goodbye today to the age when kernels were written by hand and optimized in the human mind is undoubtedly more painful still. I do not know whether any readers have felt something similar. But I suppose there is no other way for this to go. And What About Everyone Else? As AI continues to improve, I also find myself worried about a few questions: Are students today increasingly likely to use AI to do their assignments, especially hands-on work such as labs? Imagine having two choices in front of you. One is to spend eight miserable hours struggling through a lab and perhaps not even get full marks. The other is to launch an AI model, spend a few cents and a few minutes, and have it write code that earns full marks for you. Which one are most students going to choose? The point above may leave large numbers of students with seriously underdeveloped engineering ability: the ability to organize code, build systems, anticipate future needs and design for them in advance, create good abstractions, and so on. As AI becomes more capable, will those “engineering skills” still be necessary? Will they gradually become obsolete, the way fluency in handwritten x86 assembly largely has? Or will they remain permanently valuable, like understanding the entire computing stack from software to systems to hardware? If it is the latter, then we may be in trouble. Put AI in the hands of someone with poor engineering judgment, and they can now produce mountains of terrible code several times faster than before, burying all kinds of hidden problems inside systems and making the world even more of a ramshackle operation held together by improvisation. In the society of the future, will power matter more than technical ability or intelligence? Perhaps these are questions that only the times themselves can answer. Conclusion As AI develops, the society of the future may be pulled toward one of two extremes: communism or Cyberpunk 2077. In the former, productive capacity is liberated on an enormous scale, and people’s standard of living rises substantially. (I’ll leave it at that, or I’m afraid this might not make it past moderation.) In the latter, a handful of technology companies control most of society’s resources. Only a tiny number of people have access to the most advanced AI and other technologies and are able to achieve something approaching “mechanical ascension,” while most people are left with only weak, second-rate AI. Moving from one social class to another would become harder and harder: you would first need access to the strongest AI in order to climb the class ladder, creating a self-reinforcing trap. Suppose Anthropic were to retain control of the most advanced AI in the world indefinitely. Which way do you think society would go—communism or 2077? Take a guess. That is why I still believe that frontier intelligence should be made available to everyone openly and affordably. I do not trust Anthropic or OpenAI to do that. In particular, I do not want Anthropic to control the world’s most advanced artificial intelligence or AGI. To put it dramatically, I think the stakes would be comparable to Hitler obtaining the atomic bomb before the Allies did. That is also why I chose to stay at DeepSeek, and why I have continued to stay. We work on AI that is powerful, fast, and accessible to everyone, and we open-source it. Perhaps that can pull the world at least a little farther away from the 2077 end of the spectrum. I hope the world we are heading into turns out all right. May all that is good and beautiful endure. [1] By “main Attention,” I mean only MQA attention with head dim = 512. This does not include the indexer used to select the top-k important tokens. That part was written by other colleagues—who are every bit as skilled—together with their AI agents. ----- Original: 我不得不把才华埋葬在昨天 前几天,DeepSeek v4.1 发布了,将小模型能力的高度又向上推进了一个档次。 AI 发展的速度远远超过了所有人的预期。从那个只会咿呀学语地聊天、上下文长度只有几千 token 的初版 ChatGPT,到具有推理能力的 OpenAI o1、DeepSeek R1 与 Kimi K1.5 Thinking,只不过短短两年;从推理模型到如今能够流畅地在各类 harness 工具中执行命令、完成复杂任务的智能体,也不过一年半。很难想象,倘若再等上一年、两年、三年,彼时的 AI 会成为什么样子,会有多么强大,会不会已经具备了自我进化的能力,并深度渗透进了具身智能等领域。 AI 越来越会写算子了 AI 在我所从事的算子设计、编写这一领域同样进步飞速,在短短一年的时间内,他已经从一个只能帮我查查文档、读读代码、找找 bug 的小助手,蜕变成了一位能够独立阅读 CUDA、PTX 与 SASS 编码、通过专业工具分析每条指令的停顿时间、进而独立优化算子的算子大师。相信在不久的未来,它也能拥有自己独立设计算子调度、评估不同调度方案的性能、将其实现并优化的能力。 我当然为 DeepSeek v4.1 的成功而骄傲 —— 毕竟它的主 Attention 算子都是我写的 [1],它的优秀正是对我的算子的一份肯定。但是,时代的车轮滚滚向前,技术的发展无人能挡。我很清楚,再过上半年或者一年,AI 写的算子大概率就会和我写得同样优秀,甚至将我超越。AI 能一秒思考 300 个 token、半秒敲出一行命令、二十秒写完一份代码,而我不行;AI 能在模型深度、思考强度、工具调用量(和环境交互的频率)、甚至并行度等方面都能不断提升,而我不能。 人类在毁灭自己这件事情上,自古以来都表现得毫不犹豫。为什么在明知“我算子写得越好,我们的新模型的训练、推理速度就会越快,模型能力进步就会更快,我就会更早地被取代”的情况下,我仍然选择尽力优化算子呢?一方面确实是因为写算子对我来说就像打游戏一样,能为我提供极大的快感。我在发明了一种新技术、或者看到自己算子的性能上升的那一刻,心中的激动程度不亚于游戏的速通玩家打破了自己过往的记录。同时,当看到自己的算子的性能远超厂商官方的算子时,我心中也会萌生极大的自豪感。但除此之外,一个更重要的原因是,哪怕我就此“摆烂”甚至故意下绊子耽误模型训练,其它家的模型也会照常发展并最终将我照杀不误。“我当然希望自己不要被革命,但如果非被革命不可的话,我希望革我自己命的人是我自己”。在大家都这么执着于毁灭自己的时候,我也不得不加入这场残酷的军备竞赛。 那我呢 等到 AI 写算子的水平真的高于我的那天,届时的我会怎么样呢? 我的判断是:我不至于会“失业”,但必须要“转业”。我的饭碗尚且能保住,但这可能会导致我再也没机会从事那份我曾热爱过的工作。 我曾经对时代的变化与我个人在未来的处境做出过一个判断:由于时代变化真的太快(上文的 AI 发展就是一个很好的例子),我完全无法预知五年、十年后会发生什么,但不论如何,我相信凭借着自己的眼界、判断力、主观能动性与智力,留在时代的牌桌上,并重新立于时代的潮头。但是,这个判断只能保证我不会“失业”,而无法保证我不需要“转业”,倒不如说这个判断鼓励我通过转业来避免失业。 那转业代表什么呢?它代表着我需要放弃我深耕已久并充满热爱的算子设计、编写、优化领域,转而去做 Agent 的“机甲驾驶员”。在之前,我的兴趣、我所擅长的、以及工业界所需要的,三者是基本对齐的;而现在,AI 让我所擅长的变成了它更擅长的,也让工业界的需求从“会写高性能算子的人”漂移到了“能用 AI 更快地产出高性能算子的人”。为了适应工业界的需求,我势必要放弃之前那个我热爱的方向,转向一个未知的新方向。我相信我能凭借着自己对于工程学、上层模型需求和底层硬件的理解,继续高质量、高效率地产出算子,我也知道我可能会热爱这个新方向(也可能不会),但被夺走热爱的感觉,确实不太好受。那份坐在工位上静心写上一下午算子的清欢,可能会在这个夏天成为绝唱。我不得不把才华埋葬在昨天,去做一位机甲驾驶员。我的手中多了些齿轮,但心中少了些节拍。 可以打个形象的比方:你精通织毛衣技术,尤其擅长各种图案的织造与各色色彩的搭配。你所织出的毛衣质量过硬且花纹美观,十里八乡的富人都来请你为他们织毛衣,你借此赚到了不少钱。同时,你十分享受着那种坐在窗边,沏一壶清茶,望着窗外的青山、绿水、牛羊与炊烟,静静地织上一下午毛衣的感觉。但有一天,有人发明出了一台神奇的机器,只需提供毛线与图案,便可自动织出毛衣,质量与纹理都不亚于你亲手织造的,且速度远快于你。你很清楚,你的同行可以凭着这台机器轻松达到你曾经的水平,因此你不得不也去用它。你也知道,凭借着你过去二十年攒下的织毛衣技术,哪怕大家都有机器,你织毛衣的速度与质量也还能超过同行。但那份临窗听雨、引针穿线、慢度光阴的意趣,终究还是被机器的轰鸣碾碎了。 我知道这很无奈,但没办法。饭碗可以保住,但旧日的热爱大概率是要放弃的。我是一个理性和感性分离得比较开的人,在需要用理性处理问题时可以很理性,但有时也会表现出感性的一面。我记得我在搬离住了一年的出租屋时,还大哭了一场,舍不得和过去的记忆分别。今天和之前那个手写算子、人脑优化的时代告别,无疑比这更加残酷。 不知道有没有读者有类似的感受,但我想这事儿也只能这样了。 那人们呢 在 AI 不断进步的同时,我也对一些问题表示担忧: 现在的学生是不是大概率会更倾向于使用 AI 完成作业,特别是偏向于实践的各种 Lab?想象一下,如果面前有两个选择,一个是苦哈哈地用八小时时间完成一个 Lab,或许还拿不到满分;另一个则是启动 AI 模型,用几毛钱的成本、几分钟的时间,直接让 AI 编写满分代码,那大部分学生会选择哪个呢? 上面一点会导致大量学生的工程能力严重不足,包括组织代码的能力、构建系统的能力、思考未来潜在需求并提前在设计上应对的能力、抽象的能力等等。那么在 AI 能力不断变强的背景下,这部分“工程能力”是否还是必须的呢?这些工程能力是会向旧日的“熟练编写 x86 汇编”的能力那样逐渐被时代抛弃,还是会像“理解从软件到系统再到硬件的整套计算机系统”的能力那样永远具有价值?如果是后者的话,那就危险了 —— 一个工程能力很差的人,在搭配上 AI 后,产出屎山的效率可以达到先前的数倍,进而给系统埋下各式祸患,让这个世界变得更加草台。 在未来社会中,权力(power)是不是会比技术或智商更加重要? 这些问题,或许就需要时代本身来回答了。 结语 伴随着 AI 的发展,未来的社会可能会趋向于两个极端:共产主义与赛博朋克 2077。在前者中,生产力得到极大的解放,人们的生活水平有了明显的提高(就写这些吧不然我怕过不了审);而在后者中,少数科技公司控制着大部分资源,只有极少数人能够使用最先进的 AI 和各式科技,获得接近“机械飞升”的效果,大部分人则只能用上很孱弱的 AI。阶层跨越将越来越难实现:你得先有最强的 AI,才能跨越阶层,形成了一种死循环。 你猜猜如果 Anthropic 公司永远掌握着这个世界上最先进的 AI,未来社会是会变成共产主义还是 2077 呢?你猜? 所以,我还是相信,最前沿的智能应该以一种开放、廉价的方式,供应给所有人。我不信任 Anthropic 或者 OpenAI 能这样做,特别是不希望 Anthropic 掌握最先进的人工智能或 AGI,夸张点说其严重性不亚于让希特勒先于盟军掌握原子弹技术。这也是为什么我选择并坚持留在了 DeepSeek:我们研究强大、快速、普惠的人工智能并将其开源,或许能把世界从 2077 那端拉回来一些。 愿未来的世界一切安好。May all the beauty be blessed. [1] “主 Attention”仅包括 head dim = 512 的 MQA attention,不包括用于选出 top-k 重要的 token 的 indexer,那部分是由其他(水平也非常强的)同事(以及他们的 AI Agent)编写的。
10
21
1
276
38,444
Gerardo Peña Moreno retweeted
🔴 ¿DRAMA MATEMÁTICO DEL MILENIO? Ojito que podríamos estar ante una de las noticias matemáticas más importantes de la década, por lo que simbólicamente representa, y sin embargo esta viene envuelta para variar en drama. Tras varios días escuchándose por diferentes redes rumores de que uno de los famosos Problemas del milenio (Navier-Stokes) habría sido resuelto por matemáticos apoyados en la IA, hoy se ha publicado un escrito por uno de los dos matemáticos involucrados -Levent Alpoge, vinculado a Anthropic y Tristan Buckmaster quien es el que firma el escrito, y quien avisa de: 1) haber logrado, tras meses trabajando con la IA, avances claves en una vía que podría conducir a la resolución de NS (!!!) y... 2) según Buckmaster, que OpenAI -quienes también han dedicado inmensos recursos trabajando en este problema- podría estar próximos a publicar una solución completa (!!!), que podría estar fuertemente "inspirada" en la línea de investigación que había perseguido Alpoge y Buckmaster, tras conversaciones entre ellos y OpenAI. Os sigo contando más detalles en este hilo 🧵👇 Pero como conclusión rápida, por ahora parece que Alpoge y Buckmaster están próximos a publicar sus avances (a la espera de la verificación en Lean) y me imagino que OpenAI algo también tendrá que decir y publicar pronto. Dramas aparte... ¿Estamos a punto de vivir la resolución de uno de los Problemas del Milenio por parte de la IA? ¿YA?
43
232
21
1,564
186,986
Gerardo Peña Moreno retweeted
llevaba un tiempo mascándose pero con el drama de Navier-Stokes el debate ha explotado en las últimas semanas. Os resumo: - en los últimos meses varias conjeturas y problemas famosos han caído debido a la IA. En algunos casos siendo formalizaciones en Lean (un programa para probar teoremas matemáticos) que son completamente ininteligibles - muchos matemáticos están viendo como problemas sobre los que llevan una vida trabajando están cayendo frente a los últimos modelos - las matemáticas, como cualquier disciplina académica, es un campo muy competitivo. Si alguien usa la IA para ayudar en su investigación, está en ventaja y por tanto es fuerza a todos a utilizar la IA para seguir siendo competitivos. los argumentos del debate son estos: - algunos matemáticos no están contentos con esto porque le quita la diversión a hacer matemáticas (muchos lo describen como jugar a las tragaperras) - otros dicen que da igual, el objetivo es resolver los problemas y aumentar el conocimiento, da igual como - otros como Tao utilizan racionalizaciones un poco más elaboradas, argumentando que en el proceso de resolver un problema es donde se crean campos de matemáticas interesantes, y que tener demostraciones ininteligibles e inmediatas a los problemas resta incentivos que van a perjudicar al desarrollo de las matemáticas ya que estas areas no van a ser exploradas - muchos matemáticos (muxos tirando a commies/leftist) lo ven como si megacorporaciones estuvieran destruyendo su precioso campo del conocimiento para hacer publicidad de sus modelos, a estas corporaciones según ellos, no le importan las matemáticas y le están quitando la gracia en mi opinión, es estos debates son fútiles, es la negación de una realidad inexorable el problema de fondo aquí no es el futuro de la ciencia, sine el del ego humano durante milenios, el ser humano se ha aferrado a la idea de que el razonamiento, la creatividad y las matemáticas era lo que nos separaba de las bestias y de las máquinas pero no es así. el razonamiento lógico es simplemente un proceso de búsqueda y la technocapital machine ha encontrado la forma de escalarlo a partir de verter gigavatios en obleas de silicio las matemáticas no nos pertenecen toca tener humildad. debemos aceptar que, al final del día, solo somos primates con cerebros diseñados por la evolución para cazar en la sabana, no necesariamente para entender la estructura del universo el progreso del conocimiento no se va a frenar porque a un catedrático ya no le parezca "divertido" su trabajo o porque un purista sienta que le han quitado la épica al descubrimiento la transición ya ha comenzado. es hora de abandonar el antropocentrismo, bajarnos del pedestal y aceptar nuestro nuevo rol the technocapital machine is unstoppable
the math academic community is already collapsing AI will kill academia and in fact it will kill most knowledge work some revolt is expected
64
96
34
783
112,075
Gerardo Peña Moreno retweeted
Os dije que iba a probar Qwen 3.8 y no solo he hecho eso, sino que también lo he destripado, le he buscado una representación y os he hecho un vídeo. A ver qué os parece
2
70
4
402
14,440
Gerardo Peña Moreno retweeted
Google DeepMind has dismantled the original team behind AlphaFold as it shifts its scientific strategy toward Gemini-powered systems. Most authors of the original AlphaFold papers have been reassigned over the past year to projects involving Gemini, genomics, enzyme design, and nuclear fusion. Others have moved to Alphabet’s drug-discovery company, Isomorphic Labs. Nearly a quarter of the original full-time DeepMind authors have left Google entirely, including Nobel laureate John Jumper and researchers Jonas Adler and Alexander Pritzel, who joined Anthropic. This does not mean AlphaFold is being discontinued. Its database still contains more than 200 million predicted protein structures, while AlphaFold 3 and related drug-discovery work continue through DeepMind and Isomorphic Labs.
3
6
38
2,516
Gerardo Peña Moreno retweeted
Creo que la raíz de la aversión a los textos escritos con inteligencia artificial no es tanto que escriba una máquina como que todas las máquinas escriban igual. El agotamiento procede del monocultivo estilístico, su presencia creciente. Y es que, en parte, a ChatGPT y Claude les hemos orientados a esos estilos en la fase de aprendizaje por refuerzo con feedback humano: los evaluadores eligen las respuesta que más les gusta y eso marca el patrón de la IA. Creo que el fenómeno nos empuja a todos, también a los escritores apenas aseados como un servidor, a la búsqueda y reafirmación de una voz propia, de un estilo que nos diferencie. Escribo sobre escribir en los tiempos de la inteligencia artificial. error500.net/p/una-voz-propi…
5
12
2
50
11,464
Gerardo Peña Moreno retweeted
Un punto que quizás no hemos (no he) valorado lo suficiente: Las empresas de IA meten el libro antiguo en una máquina que quita el lomo y deja sólo hojas para escanear: el libro queda destrozado. en libros antiguos de los que quedan pocos ejemplares esto es irrecuperable.
Susana Pérez de El País cuenta conmigo para comentar el tema de las empresas de inteligencia artificial comprando libros físicos usados de forma masiva y a bajo precio para entrenar sus modelos de lenguaje. Tres apuntes: > hay una razón legal para hacer esto en Estados Unidos. Comprando un libro de papel eres realmente el dueño del libro y es lícito usarlo para entrenar la inteligencia artificial (en el proceso se cargan el libro eso sí). > los libros descatalogados son valiosos precisamente porque son una reserva de contenido escrito por humanos que todavía no está "contaminado" por texto generado por IA. > la escasez de datos es un problema del sector, pero que ya no es tan central como hace dos años. la industria ha pasado de perseguir volumen de datos y más computación a apostar por entrenamiento especializado con datos de calidad y problemas bien resueltos para aprendizaje por refuerzo. elpais.com/tecnologia/2026-0…
8
10
1
19
3,741
Gerardo Peña Moreno retweeted
Replying to @root_rat
@root_rat amazon.es/Build-Reasoning-Sc… ¿Qué tal estará? Tengo el LLM from scratch por terminaar y me gustó bastante.
1
2
107
Gerardo Peña Moreno retweeted
Kahneman, perdóname. Veo un PM (Hombre, foto de B4, +40 años, cargos grandilocuentes) ... No queremos más thinkers con pedigree. Queremos quien se remangue, baje al barro y ejecute. Lo de los gurús pasó. El valor está en quien hoy aguanta el fango. Barro, barro y más barro.
6
5
2
28
2,908
Gerardo Peña Moreno retweeted
Limpiando y haciendo ñapas en el piso que dejamos me he encontrado este recorte por el suelo que a saber de cuando será. Cuarta mudanza en Singapur. Como curiosidad. Llevo siempre un reloj estropeado que marca solo las 00:00. Cuando empecé a trabajar para Rainmaking en 2019 (consultoría de venture building danesa) me compré un reloj de diseño danés. No soy de relojes ni de llevar nada en las muñecas, pero me pareció bien celebrar así que por fin trabajaría en innovación tras 10 años empujando hacia ahí. En menos de un año estaba fuera. El inicio de la pandemia hizo que perdiéramos el cliente para el que trabajaba, con mi mujer a dos meses de dar a luz, ya no podíamos viajar a Copenhague donde me propusieron desarrollar el programa que inventé aquí. Ese reloj se estropeó poco tiempo después. Y lo llevo sin arreglar. Marcando las 00.00. Como recordatorio de que todo momento puede ser un comienzo y para apreciar el momento presente. Algo que también leo en este haiku que me vino a la cabeza hace unas semanas. The seconds just before everything becomes after.
3
10
2,022
Gerardo Peña Moreno retweeted
Habiendo revisado el nuevo modelo Sakana Fugu que tal y como lo presentan ellos, rivaliza con Fable 5, quiero comentar que en sí no es un nuevo modelo independiente, sino un orquestador entrenado para orquestar varios modelos diferentes (e.g. Opus 4.8, GPT 5.5, etc) e incluso a sí mismo! Mediante esta orquestación consiguen exprimir más rendimiento que sí, hace que Fugu se alce por encima de los otros modelos que utiliza. Y quizá incluso si Fable 5 estuviera disponible por API y Sakana Fugu lo utilizará en su pool de modelos, incluso su rendimiento sería superior. Guay! Pero no os dejéis llevar a equívocos pensando que este es un único modelo independiente entrenado por la compañía. O que como dicen ellos, esto ayuda a la independencia y soberanía frente a las limitaciones de USA. Al final Fugu para funcionar y lograr los benchmarks que comparten necesita de los otros modelos privados como parte del sistema, y si su acceso se corta, pues Fugu ya no Fuguncionaría.
Introducing Sakana Fugu: A full multi-agent orchestration system accessible via a single model API. Our ‘Fugu Ultra’ model matches the performance of Fable and Mythos, delivering frontier capability without the risk of export controls. Try it: sakana.ai/fugu 🐡
34
48
2
650
66,751
Gerardo Peña Moreno retweeted
Show Codex a workflow once. Reuse it as a skill. Record & Replay lets you show Codex a recurring task, like filing an expense report or submitting a time-off request. Codex turns that demo into an inspectable, editable skill. You control when recording starts and stops.
531
1,203
932
13,266
4,690,929