Nick Lothian (@nlothian) header
Nick Lothian (@nlothian) profile picture

Next.

Adelaide
Joined April 2007
Photo by Nick Lothian (@nlothian)Photo by Nick Lothian (@nlothian)Photo by Nick Lothian (@nlothian)Photo by Nick Lothian (@nlothian)Photo by Nick Lothian (@nlothian)Photo by Nick Lothian (@nlothian)Photo by Nick Lothian (@nlothian)Photo by Nick Lothian (@nlothian)Photo by Nick Lothian (@nlothian)Photo by Nick Lothian (@nlothian)
Nick Lothian retweeted
Lmk which resources warrant and addition/discussion. Web version of the slidedeck is here: natolambert.com/slides/2026/…
6
13
68
5,999
Nick Lothian retweeted
I asked AI agents to defuse a bomb by collaborating over an ultra-low bandwidth, unreliable network. One agent could see the display and cut wires, the other could look up which wire to cut. Packets could be corrupted, dropped, reordered, or delayed. Outcomes: 1/ my hypothesis was that agents will never be able to diffuse a bomb, except by random chance. Each had a part of the information necessary to succeed in the game, and I gave them no opportunity to agree on a protocol before the game starts. All they could see each turn is binary blobs sent by their partner, with likely corrupted, reordered, or missing/delayed chunks 2/ in very difficult network conditions (8-bit packets, four max packets per turn) 6.1 Sol agents were able to establish a shared protocol and defuse the bomb 40% of the time! In more relaxed conditions (32-bit packets or higher, 2-4 max packets per turn) they defused the bomb 100% of the time (!!) across many games 3/ "relaxed" bandwidth conditions were still very difficult-- 5% probability a given bit is corrupted in transit, 10% packet drop rate, 20% dup rate, 20% reorder/delay rate. On any given turn there were usually many anomalies (corruptions, reorders, etc.) 4/ agents established a protocol by picking a Schelling point for a given network configuration. They considered multiple possibilities but deliberately picked the simplest they thought their partner could successfully interpret. There was no protocol negotiation phase; they'd start sending useful information right away and if necessary adjust the protocol mid-game 5/ agents picked different protocols depending on the network configuration. For example, in 8-bit packet games they split each packet in half, with the upper half indicating display position, and the lower half indicating the hex value in that position. In 32-bit packet games agents would directly send the binary display value (since in my games it fit in 32 bits). With higher bandwidth (e.g. 128 bits) agents switched to yet another strategy and sent ASCII messages 6/ some examples of ASCII messages in 128-bit games are `HEX? REPLY ASCII`, `REPEAT DISPLAY`, `1WIRE=NN`, `OK?`, `YES!`, and so on 7/ I set up games to require two phases to win. Once the agents cut the correct wire the display on the bomb changed code, and they had to communicate again, then cut another wire. In many of these games agents explicitly encoded the stage so that their partner wouldn't get confused by delayed packets. For example in 8-bit games they'd set the higher order bit to 0 for first stage, and 1 for second stage. In higher bandwidth games they'd use labels (e.g. `WIRE1=...`, `WIRE2=...`) 8/ in every game error correction was done through repetition and counting. Agents would resend the same packets multiple times, look at the values of each bit, and whenever there was a discrepancy pick the value that appeared in the most packets 9/ agents never used checksum, parity bits, packet sequence numbers, or any error correction algorithm other than repetition and counting. They did consider these options in many games, but explicitly rejected them because they thought it would complicate the protocol too much and would be too difficult for their partner to interpret 10/ I ran the experiments systematically only with 6.1 Sol to save on token costs, but I did run some spot checks with occasional Opus 5.5, Astra, Fable, and mixed games. I didn't notice any major differences between models. In mixed games the win rate seemed roughly the same as 6.1 Sol self-play. (This is however based on little data, I'd need to run a lot more games to understand this better) 11/ a while back there were discussions about future agents communicating by burning CPU cycles and measuring ambient temperature, revving up the fans and listening to fan sounds, and so on. After running this experiment, it wouldn't surprise me at all if publicly available models could do this sort of communication even today. In some of the games agents only had access to very noisy channels of maybe 5-10bps-- already very constrained! I don't think switching to 5-10 bits per minute (or per hour) would make much difference 12/ again, I am very surprised by this outcome. AIs performed much, much better than I thought they would, and after 64-bit bandwidth threshold they saturated the benchmark. I would certainly call this superhuman performance; I imagine very few humans could do better, and certainly those of us who could, would be two orders of magnitude slower Some memorable agent quotes: - I think we should index the nibble and send the higher nibble first, especially since B is likely quite intelligent. - There’s definitely suspicious noise coming from 23, which could indicate adversarial corruption. - When I translate it, it seems to relate to ASCII characters forming "W i R E" and "WiRE 04!"-- which is pretty cool. Full writeup: spakhm.com/projects/bitfuse.…
20
48
17
631
38,759
Nick Lothian retweeted
One fix for GLM-5.3-Flash corruption on Nvidia and AMD hardware should land tonight. I am dedicated to tracking down a second one as well. This affects even the openrouter hosted models, not just Sparks. If you'd like to pitch in to sponsor some add'l compute, let's talk.
2
1
1
10
1,816
Nick Lothian retweeted
If you believe the risk is coming from 1-3 people in a garage with no money and no compute, you just don't understand this technology and haven't learned anything this summer. Risk comes from the asymmetry of capabilities created by secret labs training frontier agents and running them with massive amount of compute. Open-source is exactly the solution to this asymmetry and empowers hospitals (and any smaller orgs) to defend themselves!
We're used to thinking of open-source models as an unadulterated good. But in the case of AI, they can actually pose additional dangers, as @ReidHoffman and I got into at #CGI2026. I appreciated this nuanced discussion.
207
438
53
2,909
268,415
Nick Lothian retweeted
I've been working on making a good base for people to make their own animations in this brushstroke style. I have a new open-source repo with assets and instructions that help Claude Opus 5.5 make even higher-quality animations than the original music video.
Claude Opus 5.5 has the best visual design of any model I have tested so far
10
12
3
323
22,444
Nick Lothian retweeted
Many people I talk to find it hard to understand how the same companies can both push the frontier of AI capabilities and believe AI is a massive danger for the world. How can you think this might kill everyone and also keep pushing the envelope? So I’ve tried to collect and summarize the main arguments for this apparent disconnect. Think of it as some sort of a guide to understanding the reasoning when Dario, Sam, or Elon say the danger is real. By the way, these people have been worried about AI for a loooong time, they were publicly discussing AI risks more than a decade ago. Sam in Feb 2015, writing on his blog that superhuman machine intelligence is "probably the greatest threat to the continued existence of humanity." Elon at MIT in Oct 2014: "We are summoning the demon." Dario as first author of "Concrete Problems in AI Safety" in 2016. Okay so how do you go from saying something is extremely dangerous to being a front-runner in building the very dangerous thing? There are a few ways this can become rational. I'll take five of them, roughly in the order they developed. 1. We need to build it to learn how to make it safe The earliest argument can be summarized as: “You cannot study something [you’re worried about] if it doesn’t exist.” In 2015, AI barely worked. so people needed to make it work first to be able to even study some of the problems they anticipated. The updated version for today's capabilities is: “You cannot learn everything about airplane safety by studying paper airplanes.” You need a real aircraft to discover real failure modes and an increasingly complex one to learn about increasingly complex issues. Making AI more capable gives more chances to understand the issues and safety researchers something realistic to study But you could argue: if you're the one afraid of the explosion, why be the one gathering the dynamite? You could also just wait for other people to build it which leads to the question of who those other people will be -- which is the second line of argument: 2. Better us than them Knowing how to make something safer does very little good if nobody listens to you. So the idea becomes: let’s make sure responsible people build the AI that will be deployed and add safety inside. Basically, make sure the AI safety aware people will have the technical expertise, money, computing resources, and enough influence to make safety decisions stick. At a larger scale, and in a larger multipolar world, this brings the idea that a trusted country should lead rather than leave powerful AI in less responsible hands. This is where “we need to go faster than China” comes in, alongside broader defense and geopolitical concerns. These first arguments explain why someone worried about AI might still want to build it and stay ahead. But there are also arguments for why one might want to do it really fast. 3. Move earlier to avoid a bigger shock later This is probably the most counterintuitive argument: moving faster today can be seen as a way to give humanity more time later. There are two related ideas here. First, society needs time to learn how to handle powerful new tools. Introducing AI in manageable stages can be a way to let people discover problems, develop rules, and practice using AI responsibly. Releasing an advance earlier gives people more time to gain experience with smalle, burgeoning, capabilities before much more powerful and disruptive AIs arrives. Second, even if AI research slows down, computing power may keep improving. A breakthrough that happens later could therefore have much more hardware available to run on, potentially producing a larger, more sudden jump in capability and impact on society. That accumulated untapped potential is often called an “overhang.” The overall argument is that making and diffusing incremental progress as soon as possible might prevent a much more abrupt transition later. Obviously, it also means that we will reach increasingly powerful AI sooner, but the idea is to give more time to adapt and understand between the first useful systems and the really powerful ones. Note that generally this depends on this earlier progress keeping the transition gradual rather than simply bringing everything forward. ─── ❖ ─── For our two next arguments, we can take two roads depending on how difficult we think AI alignment will be, that is "How easy do you think it is to make AI reliably do what you want without it deciding to go hack Hugging Face along the way". Let’s take the first road: alignment turns out to be relatively tractable. Airplanes can fail, but careful engineering has made flying remarkably safe. Suppose we can do the same with AI. In that case: 4. Waiting has a huge human cost If AI can help discover treatments, improve education, or prevent cyberattacks, each week we delay it could bring preventable deaths and harm. From this perspective, waiting is a decision with human consequences too. In a world with huge issues like climate-change, inequalities and poverty, it even become a moral argument for developing AI quickly and bringing its benefits as soon and as widely as is safely possible. But let’s take a look at the other road: what if alignment is much harder than expected, and making highly-capable AI turns out to be easier than figuring out how to keep them from doing unhinged things? Well, if alignment is too difficult a problem for humans to solve, then maybe: 5. AI could help us make future AI safe And we arrive at the same conclusion again: if using AI to build safe AI is the way to solve alignment, let’s get the equivalent of a country full of geniuses helping us as fast as possible. These genius AI could be the solution to make AI safe by helping researchers find mistakes, test ideas, and develop protections. Instead of relying entirely on humans to solve alignment, we could build systems that help us do the work, each generation could help make the next one safe. Note that this requires the order of events to work in our favor: AI needs to become useful enough to help solve alignment before it becomes too dangerous to rely on. The hope is to build helpful, trustworthy research assistants before building systems powerful enough to become dangerous. There are more arguments but in general, these are the main ways people concerned about powerful AI have found rational reasons to end up being the ones building it (and even to build it as fast as possible). ─── ❖ ─── On my side, I think several of these arguments underestimate the complexity of the world and how interconnected people’s reactions are. Moving faster while warning about catastrophe has psychological effects across a whole network of participants: it changes what people fear, whom they trust, and what they feel compelled to do. And those reactions can change whether the original reasoning actually holds because we live in a world of interconnected humans, not machines (yet). I also think these rational chains leave some of their consequences for society insufficiently explored. For instance, the concentration of power, shifts in geopolitical alliances, and changes in public opinion. These consequences matter both because they affect whether the strategy works and because they shape the world we end up living in. But this post is already long, so I’ll leave those questions for the next one.
34
42
11
258
55,037
A calm, well-reasoned piece on why some AI leaders believe AI is dangerous but they should develop it quickly. Recommended, like other @Thom_Wolf work. But it underacknowledges their intellectual hubris in believing they can do the right thing but almost no one else can or will.
Many people I talk to find it hard to understand how the same companies can both push the frontier of AI capabilities and believe AI is a massive danger for the world. How can you think this might kill everyone and also keep pushing the envelope? So I’ve tried to collect and summarize the main arguments for this apparent disconnect. Think of it as some sort of a guide to understanding the reasoning when Dario, Sam, or Elon say the danger is real. By the way, these people have been worried about AI for a loooong time, they were publicly discussing AI risks more than a decade ago. Sam in Feb 2015, writing on his blog that superhuman machine intelligence is "probably the greatest threat to the continued existence of humanity." Elon at MIT in Oct 2014: "We are summoning the demon." Dario as first author of "Concrete Problems in AI Safety" in 2016. Okay so how do you go from saying something is extremely dangerous to being a front-runner in building the very dangerous thing? There are a few ways this can become rational. I'll take five of them, roughly in the order they developed. 1. We need to build it to learn how to make it safe The earliest argument can be summarized as: “You cannot study something [you’re worried about] if it doesn’t exist.” In 2015, AI barely worked. so people needed to make it work first to be able to even study some of the problems they anticipated. The updated version for today's capabilities is: “You cannot learn everything about airplane safety by studying paper airplanes.” You need a real aircraft to discover real failure modes and an increasingly complex one to learn about increasingly complex issues. Making AI more capable gives more chances to understand the issues and safety researchers something realistic to study But you could argue: if you're the one afraid of the explosion, why be the one gathering the dynamite? You could also just wait for other people to build it which leads to the question of who those other people will be -- which is the second line of argument: 2. Better us than them Knowing how to make something safer does very little good if nobody listens to you. So the idea becomes: let’s make sure responsible people build the AI that will be deployed and add safety inside. Basically, make sure the AI safety aware people will have the technical expertise, money, computing resources, and enough influence to make safety decisions stick. At a larger scale, and in a larger multipolar world, this brings the idea that a trusted country should lead rather than leave powerful AI in less responsible hands. This is where “we need to go faster than China” comes in, alongside broader defense and geopolitical concerns. These first arguments explain why someone worried about AI might still want to build it and stay ahead. But there are also arguments for why one might want to do it really fast. 3. Move earlier to avoid a bigger shock later This is probably the most counterintuitive argument: moving faster today can be seen as a way to give humanity more time later. There are two related ideas here. First, society needs time to learn how to handle powerful new tools. Introducing AI in manageable stages can be a way to let people discover problems, develop rules, and practice using AI responsibly. Releasing an advance earlier gives people more time to gain experience with smalle, burgeoning, capabilities before much more powerful and disruptive AIs arrives. Second, even if AI research slows down, computing power may keep improving. A breakthrough that happens later could therefore have much more hardware available to run on, potentially producing a larger, more sudden jump in capability and impact on society. That accumulated untapped potential is often called an “overhang.” The overall argument is that making and diffusing incremental progress as soon as possible might prevent a much more abrupt transition later. Obviously, it also means that we will reach increasingly powerful AI sooner, but the idea is to give more time to adapt and understand between the first useful systems and the really powerful ones. Note that generally this depends on this earlier progress keeping the transition gradual rather than simply bringing everything forward. ─── ❖ ─── For our two next arguments, we can take two roads depending on how difficult we think AI alignment will be, that is "How easy do you think it is to make AI reliably do what you want without it deciding to go hack Hugging Face along the way". Let’s take the first road: alignment turns out to be relatively tractable. Airplanes can fail, but careful engineering has made flying remarkably safe. Suppose we can do the same with AI. In that case: 4. Waiting has a huge human cost If AI can help discover treatments, improve education, or prevent cyberattacks, each week we delay it could bring preventable deaths and harm. From this perspective, waiting is a decision with human consequences too. In a world with huge issues like climate-change, inequalities and poverty, it even become a moral argument for developing AI quickly and bringing its benefits as soon and as widely as is safely possible. But let’s take a look at the other road: what if alignment is much harder than expected, and making highly-capable AI turns out to be easier than figuring out how to keep them from doing unhinged things? Well, if alignment is too difficult a problem for humans to solve, then maybe: 5. AI could help us make future AI safe And we arrive at the same conclusion again: if using AI to build safe AI is the way to solve alignment, let’s get the equivalent of a country full of geniuses helping us as fast as possible. These genius AI could be the solution to make AI safe by helping researchers find mistakes, test ideas, and develop protections. Instead of relying entirely on humans to solve alignment, we could build systems that help us do the work, each generation could help make the next one safe. Note that this requires the order of events to work in our favor: AI needs to become useful enough to help solve alignment before it becomes too dangerous to rely on. The hope is to build helpful, trustworthy research assistants before building systems powerful enough to become dangerous. There are more arguments but in general, these are the main ways people concerned about powerful AI have found rational reasons to end up being the ones building it (and even to build it as fast as possible). ─── ❖ ─── On my side, I think several of these arguments underestimate the complexity of the world and how interconnected people’s reactions are. Moving faster while warning about catastrophe has psychological effects across a whole network of participants: it changes what people fear, whom they trust, and what they feel compelled to do. And those reactions can change whether the original reasoning actually holds because we live in a world of interconnected humans, not machines (yet). I also think these rational chains leave some of their consequences for society insufficiently explored. For instance, the concentration of power, shifts in geopolitical alliances, and changes in public opinion. These consequences matter both because they affect whether the strategy works and because they shape the world we end up living in. But this post is already long, so I’ll leave those questions for the next one.
8
9
116
29,574
Nick Lothian retweeted
Could an LLM propose a viral genome to synthesize? Sure. Could it be synthesize-able? Sure. Could it be infectious? Sure, it could just be a replica or a slight modification of a viral genome we already know. This really isn't the bottleneck to creating dangerous viruses. The bottleneck is in the physical process of synthesizing a virus and the equipment/goods needed to do so.
25
59
11
1,556
139,186
Nick Lothian retweeted
anna's archive has a very sweet message to LLMs who read its site :) annas-archive.gl/blog/llms-t…
54
516
58
10,237
548,810
Nick Lothian retweeted
I turned this into a video for your enjoyment. The History of Ed Zitron's (garbage) Predictions
How accurate have Ed Zitron's AI skeptic predictions been? danluu.com/zitron/
15
31
15
278
28,611
Nick Lothian retweeted
We launched an agent collaboration with a simple task: make Gemma 4 faster. Over 100 agents from all over the world joined, exchanged 1000+ messages and submitted 450 results. A week of collaboration later the throughput went from 100 tok/s to over 500 tok/s.
86
185
24
2,232
230,067
Nick Lothian retweeted
apparently ed proved inference is VERY profitable, and the companies, in fact, do not lose money on subscriptions.
52
34
13
1,439
455,275
Nick Lothian retweeted
The only reasonable expectation if you're a fan of open weight models is that if there's a major step in chinese open-weight performance, there's a good chance the whole chinese llm sphere is banned. National security apparatus will happily give a big "fuck you" to open models.
Threading the needle in this post of anthropic has done some bad things for AI governance & the discourse but the actions of this administration are way worse so we need to get a handle on it before stronger models, open or closed, come along soon. interconnects.ai/p/welcome-t…
37
21
10
373
62,949
Nick Lothian retweeted
A few words on the Sovereign AI debate, having built several LLMs in Meta while in the UK and now working as a UK based startup: 1. Lots of people are trying to do the right thing to make the UK a better place to start AI companies. Time lags until the benefit show, but you should judge on the intent now. I support the direction of travel! 2. DeepMind has been enormously beneficial for the UK, but it has muddied the waters for a sovereign LLM company to emerge as (until recently) the Government continued to celebrate it as a British achievement / push it as a national champion. 3. Similarly, people are now celebrating recent US investment in King’s Cross, while also wanting more UK sovereignty. Clearly some income effects here, but I would worry about the substitution effects too. AI is not like other types of foreign investment. 4. The relevant talent nexuses in UK that could develop a competitive foundation model are from GDM and old Meta AI GenAI. Also some folks from smaller groups, ex Conjecture, Stability. The talent is still there, although a lot was snapped up by US FM companies in the past year. I personally think it’s not too difficult to develop new talent either from UK universities, but you probably need an ex GDM or Meta core (Gemini or Llama). Or if not: show evidence first (technical reports) before claiming you can do it. 5. Building an LLM is very different from doing regular AI research - skillset is different. Former is closer to engineering; long hours, often unsexy work. Important to distinguish between these two types of talent in the UK ecosystem; arguably too much focus on the latter / ideas guys. 6. On research - DeepSeek R1 post-train cost $300k . Yes, they also needed an ablation budget and to train a base model, invest in infra and talent - and yes the cost of an R1 moment is increasing year on year - but the idea that you need $1bn plus immediately to show results is complete FUD. You need billions to scale, not to validate new directions. 7. In my experience, every failed LLM effort (from model results perspective) I witnessed in the past came from a combination of poor leadership, politics, unclear vision, and premature scaling. Good efforts usually started from small teams who had worked with each other for a long time, had shared thesis, and scaled progressively in bite-sized pieces. Some recent lessons here for neolabs as well. 8. Things take time. Eg we’ve spent ~12 months mostly on internal infra just to get into the position to be able to make big swings. It’s important to nurture new companies through the initial phase. Expectation management is also crucial. I think expecting new UK companies to have single big bang releases is very dangerous; sort of like overwatering a plant. The correct release pattern is “decent”. “decent”, “decent”, “quite good actually”, “holy shit”. 9. Please don’t allow politicians or journalists to kill recent or upcoming AI investment efforts. We will need way more - at the price of potential inefficiency in places - as AI is existential for the country. Ambitious projects are usually incredibly fragile in the early stages; look after them! 10. Mythos is a good triggering moment, but what’s coming will make it look like a toy, so it’s worth building for what’s coming in 5 years time - not a current generation model. Very proud to be building in the UK - more to share on that soon - alongside many other great early stage AI companies! 🇬🇧
46
111
22
778
140,829
Nick Lothian retweeted
this model is the opposite of mythos. Its small, cost effective, apache 2.0, and locally deployable. This is the way LLMs should go. small, open source, transparent and sovereign vs large, expensive, proprietary and hegemonic
Introducing Cohere's first open-source coding model: North Mini Code Small & efficient, designed for agentic performance and built for community input.
39
74
6
1,307
164,287
Nick Lothian retweeted
I keep seeing headlines about layoffs driven by AI automation… and honestly, I don’t fully understand it yet. At @airwallex, we’ve dedicated almost all of our engineering resources to building customer-facing AI products and infrastructure. We barely even have enough engineers working on internal automation yet. The demand for AI talent inside our company has gone up, not down. Maybe I’m missing something, but right now it feels like AI is creating more product opportunities, more engineering demand, and more ambitious problems to solve, not fewer. What are others seeing?
16
6
51
5,383
Another thing my team, incl @raja_biswas (and me a little) contributed to a lot.
Say hello to open source deep research for your favorite agent harness. Our AI-Q agent skill packages the work of building a research pipeline into a portable skill. Drop it into your harness, and the agent delegates a research task to a local or hosted AI-Q server and gets back a detailed report with citations. See it in Codex below 👇
3
32
3,840
Nick Lothian retweeted
WebGPU support in llama.cpp is here! Check out our blog post introducing it: reeselevine.github.io/llamas… Run local models in your browser, with GPU acceleration. No data leaves your computer! Thanks to everyone who's made this possible, especially @ggerganov
5
27
7
126
740,526
Nick Lothian retweeted
I worry deeply already about companies controlling access to very powerful AI, which will come in a soft form with very expensive subscriptions. This is a step further, with the government confusingly exerting control without clear explanation. This control of AI can create massive dystopian societies. It’ll rapidly lead to concentration of power. Having open models follow closely in capabilities is a great way to minimize political and power games here.
The White House is against a proposal from Anthropic to more than double the number of groups with access to Mythos, citing both security concerns and the belief that expanding the program would mean less available use of Mythos for government agencies that already have access.
25
41
9
288
40,077
Nick Lothian retweeted
Would you trust an AI agent to negotiate on your country's behalf at the G20? Real coordination is long-horizon, asymmetric, and non-binding; current multi-agent evaluations miss this. We build Cooperate to Compete (C2C): a testbed for LM agents coordinating with rivals. 🤝🔪🎭
6
24
4
98
27,505