Ivan (@ivanbokii) header
Ivan (@ivanbokii) profile picture

Software Engineer. Interested in system design and hammock-driven development. Edinburgh walker.

Edinburgh, Scotland
Joined December 2008
Photo by Ivan (@ivanbokii)Photo by Ivan (@ivanbokii)Photo by Ivan (@ivanbokii)Photo by Ivan (@ivanbokii)Photo by Ivan (@ivanbokii)Photo by Ivan (@ivanbokii)Photo by Ivan (@ivanbokii)Photo by Ivan (@ivanbokii)Photo by Ivan (@ivanbokii)Photo by Ivan (@ivanbokii)
Ivan retweeted
Simple way to evaluate an AI frontier lab ethics / company culture: attitude towards the user base in the time spans they have the best model.
4
26
3
495
25,000
This is like one of those audio journals you find on a corpse in a sci-fi horror game
At my company we stopped doing code reviews. There's no point now
104
2,768
21
38,403
814,659
rick beato is jow part of the local inference open weights underground. this timeline is killing me. youtu.be/YTLnnoZPALI?is=SDn-…
16
4
1
56
7,424
Two announcement events recently that I simply don't get. Amp (a coding agent tool) declaring coding agents as dead. Linear (a issue tracking tool) declaring issue tracking as dead.
26
5
12
179
59,995
This is more evidence that current frontier models remain completely reliant on content-level memorization, as opposed to higher-level generalizable knowledge (such as metalearning knowledge, problem-solving strategies...)
🚨 Shocking: Frontier LLMs score 85-95% on standard coding benchmarks. We gave them equivalent problems in languages they couldn't have memorized. They collapsed to 0-11%. Presenting EsoLang-Bench. Accepted to the Logical Reasoning and ICBINB workshops at ICLR 2026 🧵
187
307
43
2,934
313,933
Replying to @AnnaLeptikon
Computers used to be unforgiving. I wonder what will happen if the next generation of computer scientists does not grow up with “syntax error in line 20” but with “you are absolutely right, let me try…”
27
27
2
347
16,072
I give it less than 6 months before Garry stops preaching LOC and starts preaching maintainable code bases. And with that one move he will go from junior engineer to a bit more senior. We watching his Eng journey live 🍿
unavailable
61
85
7
2,921
164,269
I don't have too too much to add on top of this earlier post on V3 and I think it applies to R1 too (which is the more recent, thinking equivalent). I will say that Deep Learning has a legendary ravenous appetite for compute, like no other algorithm that has ever been developed in AI. You may not always be utilizing it fully but I would never bet against compute as the upper bound for achievable intelligence in the long run. Not just for an individual final training run, but also for the entire innovation / experimentation engine that silently underlies all the algorithmic innovations. Data has historically been seen as a separate category from compute, but even data is downstream of compute to a large extent - you can spend compute to create data. Tons of it. You've heard this called synthetic data generation, but less obviously, there is a very deep connection (equivalence even) between "synthetic data generation" and "reinforcement learning". In the trial-and-error learning process in RL, the "trial" is model generating (synthetic) data, which it then learns from based on the "error" (/reward). Conversely, when you generate synthetic data and then rank or filter it in any way, your filter is straight up equivalent to a 0-1 advantage function - congrats you're doing crappy RL. Last thought. Not sure if this is obvious. There are two major types of learning, in both children and in deep learning. There is 1) imitation learning (watch and repeat, i.e. pretraining, supervised finetuning), and 2) trial-and-error learning (reinforcement learning). My favorite simple example is AlphaGo - 1) is learning by imitating expert players, 2) is reinforcement learning to win the game. Almost every single shocking result of deep learning, and the source of all *magic* is always 2. 2 is significantly significantly more powerful. 2 is what surprises you. 2 is when the paddle learns to hit the ball behind the blocks in Breakout. 2 is when AlphaGo beats even Lee Sedol. And 2 is the "aha moment" when the DeepSeek (or o1 etc.) discovers that it works well to re-evaluate your assumptions, backtrack, try something else, etc. It's the solving strategies you see this model use in its chain of thought. It's how it goes back and forth thinking to itself. These thoughts are *emergent* (!!!) and this is actually seriously incredible, impressive and new (as in publicly available and documented etc.). The model could never learn this with 1 (by imitation), because the cognition of the model and the cognition of the human labeler is different. The human would never know to correctly annotate these kinds of solving strategies and what they should even look like. They have to be discovered during reinforcement learning as empirically and statistically useful towards a final outcome. (Last last thought/reference this time for real is that RL is powerful but RLHF is not. RLHF is not RL. I have a separate rant on that in an earlier tweet nitter.cf/karpathy/status/182127…)
DeepSeek (Chinese AI co) making it look easy today with an open weights release of a frontier-grade LLM trained on a joke of a budget (2048 GPUs for 2 months, $6M). For reference, this level of capability is supposed to require clusters of closer to 16K GPUs, the ones being brought up today are more around 100K GPUs. E.g. Llama 3 405B used 30.8M GPU-hours, while DeepSeek-V3 looks to be a stronger model at only 2.8M GPU-hours (~11X less compute). If the model also passes vibe checks (e.g. LLM arena rankings are ongoing, my few quick tests went well so far) it will be a highly impressive display of research and engineering under resource constraints. Does this mean you don't need large GPU clusters for frontier LLMs? No but you have to ensure that you're not wasteful with what you have, and this looks like a nice demonstration that there's still a lot to get through with both data and algorithms. Very nice & detailed tech report too, reading through.
358
2,082
447
14,125
2,442,633
Been diving into some great AI/ML blogs lately: - Blog by @chipro - Blog by @eugeneyan (links below) I'm curious to find new gems. What are you reading these days?
4
3
19
2,162
That’s a true masterclass in russian imperialism, when you seamlessly believe that Ukrainians have to forgive russians in the name of some (imaginary) love, all while struggling to understand why Ukrainians keep fighting so fiercely to protect home and families they actually love
3
9
569
2025 is 0% complete.
343
11,332
939
87,628
4,725,269
The inspiration from @ncooper57 to get tmux to do the heavy lifting to make ShellSage work is genius! Thanks to this, ShellSage is <100 lines of code (plus comments and prompts). But the result is amazing. Try it out and tell us how you go. Details here: answer.ai/posts/2024-12-05-i…
5
7
2
81
8,804
If you haven’t tried Atuin yet, you should! Has become a daily-use tool for me in the last couple of years.
This quoted post is unavailable.
1
6
1,507
★ Dynamicland's new website documents ten years of progress toward a humane dynamic medium. dynamicland.org
23
197
72
821
190,333
Founder Mode is now streaming on @netflix
100
251
54
3,115
525,265
Big news: I'm helping with @martinkl with a second edition of Designing Data-Intensive Applications! An early release of the first 3 chapters is now available (O'Reilly Learning subscribers only at this point) and we're hoping to finish it next year. learning.oreilly.com/library…
10
34
5
191
8,219
I want @cursor_ai but in @pycharm . I keep switching between the two!
1
1
5
544
Very clear thinking from @destraynor here. Nice presentation too. Moody future without sensationalism. Obviously smart marketing too. Intercom is re-rooting itself in very fertile soil, in the perfect weather.
Welcome to Off Script! In episode 1, @destraynor shares some poignant and emotive ideas to help us understand the great importance of this time in tech, some stories about our internal reaction to the dawn of modern AI, and some frameworks for how you can respond too. Enjoy.
6
9
156
37,694
I’m not worried, I’ll just invite Devin to all my meetings so it can’t get anything done either
22
133
15
1,694
123,749