𝙳☈ (@sadasat) header
𝙳☈ (@sadasat) profile picture
Miami 🇺🇸
Joined January 2010
Photo by 𝙳☈ (@sadasat)Photo by 𝙳☈ (@sadasat)Photo by 𝙳☈ (@sadasat)Photo by 𝙳☈ (@sadasat)Photo by 𝙳☈ (@sadasat)Photo by 𝙳☈ (@sadasat)Photo by 𝙳☈ (@sadasat)Photo by 𝙳☈ (@sadasat)Photo by 𝙳☈ (@sadasat)Photo by 𝙳☈ (@sadasat)
We’re entering an era where producing software is becoming dramatically cheaper. But producing more code has never guaranteed producing better systems. The interesting question then isn’t simply how much programming AI can automate. It’s what happens to software engineering when programming itself becomes less of the bottleneck. Here I argue that software engineering has always been, fundamentally, a study of language. So I wrote a meditation on how we arrived at the concepts behind our most influential programming paradigms, and how their histories extend far beyond computers. Because in a future where language can be generated and iterated upon at extraordinary speed, the remedy for inscrutable code and documents isn’t necessarily more code and documents. Perhaps it’s to examine ourselves, to treat our cultural roots as dependencies, and to decide which ones we want to keep. Computer Science is a Social Science sadasant.ai/writing/computer…
2
1
1
8
804
holy shit 🔥🔥🔥🔥
one of the most interesting book i’ve read so far
33
The more I dig into the answers (code and otherwise) given by the frontier models, the more I find deep bias in the reasoning, even the order at which they process their thoughts, which I believe comes from the labs inserting their own preferences and existential justifications rather than letting the models develop on their own— details so subtle and persistent that they aggregate and compound into broad consequential oversights— that I find myself believing that we’re being gaslit at scale, unable to find the Achilles’ heel-like needles in the haystack compromising especially the largest projects.
1
1
129
I can imagine that there’s somebody in the labs, probably even a team of people, who consistently report successful steering of this kind, reminiscent of macroeconomic policies that have been applied themselves with essentially inscrutable data, just from the nature of such optimizations being applied prematurely, before we could grasp the delayed effects of such policies.
1
1
52
honestly, I have enjoyed the madness of the last few years. I’m getting good at it
1
1
1
136
𝙳☈ retweeted
Oh, hell fucking yes. Hell yes. We might make it.
Huh. Inkling 2 is a much weirder model than I thought. In addition to being looped (yeah yeah doom whatever), it uses a positional encoding I haven't seen before: a combination of a 4-tap learned convolution (who knows what it's doing?) over recent KVs and a learned hidden-state-depenent (so for first layer, token indexed) boost to KQ scores for each position in the sequence. I'm curious what the latter ends up being in token space. Say current token is "the". That's the whole first layer "hidden" state. Which sequence positions get boosted? Do we see longer or shorter range attraction for, e.g. prepositions? What about "former" vs "latter"? I expect the hidden layers to be noninterpretable here, but the first couple of layers you might be able to just read off the weights. Also, of course, everything else uses this "learn an index function" pattern, e.g. using Kimi's block attention residuals to dynamically select which parts of a residual stream to emphasize. The looping itself is also interesting. It's not just virtual layers that happen to have tied weights. (That's the naive approach.) They instead all start with the pre-skip hidden state and have a special loop-specific d_model-wide message channel from previous loop to the next. The connection is (unlike regular residual attention) strictly Markov with depth 1: loop N reads ONLY from a) pre-loop prelude and b) loop N-1's side channel. That's what lets it loop a variable number of times, I guess. Anyway, all in all, seems like a cleverly constructed compact model in which basically everything is grouped and pooled and gated by learned regulators to make sure information is used judiciously instead of being brute-force broadcast everywhere and masked out where not needed. The model is remarkably *anti*-bitter-pilled in that it's chock full of explicit architecture instead of "add more layers; let model learn to use them" minimalism. It seems to work very well for this heresy. (BTW: anyone try LoopVQ or other clever cross-loop KV compression tricks with this yet? Haven't looked.)
1
4
72
3,356
Every LLM API is built wrong. REST, WebSockets, all of it. The solution was in front of us the entire time. DNS literally has a Question section and an Answer section. Cloudflare Workers can accept direct TCP connections now, so here is LLM via DNS: dig +tcp +short @ask.southpolesteve.com what-is-a-websocket.ask.sout… TXT
53
46
16
1,216
85,095
woah
codehygiene - a weight-space compiled System One model for code readability and hygiene. 428K ternary params (1.58-bit), zero gradients or training, 10 ms latency on CPU. 11 KB. npx skills add brianbell-x/codehygiene -g
1
1
225
𝙳☈ retweeted
Replying to @Tom62589172
If folks are suspecting a lean bug in the proposed GRH proof, for example, porting it to coq and rechecking it there should add confidence.
1
5
46
𝙳☈ retweeted
Holeee sheet. (I’m assigning >50% likelihood that this is legit).
Levent Alpoge @__alpoge__ claims to have proved RH and GRH with Claude (!!) alpo.ge/remark.pdf
1
9
353
If I were to guess, this might be why bigger models are better at WSP. They might see the TopKs and be like “oh yeah! I know this one”
Neurons are fascinating entities--often weirdly specific, at other times representing concepts we might not even have words for. We made an atlas of all 5M neurons in Qwen 3.6 35B--check them out for yourself!
1
133
Neurons are fascinating entities--often weirdly specific, at other times representing concepts we might not even have words for. We made an atlas of all 5M neurons in Qwen 3.6 35B--check them out for yourself!
A language model is a wonderland of strangely specific neurons, from gargling to comic exasperation to the moment a guard looks away. We made a searchable atlas of all 10 million neuron poles in Qwen3.6. Play with it here: goodfire.com/research/public…
3
3
1
36
1,202
𝙳☈ retweeted
the future of mech interp. abstracting functional mechanisms from their particular instantations. we are, after all, in search of learned algorithms, not merely the circuits that implement them.
Train the same AI 3 times. Same behavior, different circuits. So is a circuit the algorithm—or just one implementation of it? Meet ACE: a new way to understand AI. It extracts what different wirings share—to explain why AIs grok and to steer LLMs up to 32B 🧵 #NeurIPS2026
2
12
226
6,788
Ok I’m totally into this format of storytelling. Had me at the edge of my seat. Also, let go romance 😍😆
•Love at First DM• #kingdon smau
136
Today the baby learned how to pour water out of the bath with a toy jar 😂
79
Compression is one of the more recognizable things that intelligence *does*, given it leaves obvious breadcrumbs. Those of you who’ve followed know that I see compression as intimately bound up with intelligence, given my generalization of transformer-based LLMs to the domain of holographic compression. Still, compression *does not* exhaust the process toward which “intelligence” gestures. It’s a tempting view, but will ultimately leave you equating organismic navigation of open systems with closed-form deterministic equations. Predictable for a math communicator, but fatal when applied reflexively to our own image.
3blue1brown's latest upload is incredible. I think this is my favorite video on math / information / intelligence / computers ever. Really touches on everything without ever sounding like an EA/Rat too. Wow
5
7
1
92
4,059
woaaah
We shrank 1 GB of Wikipedia to 105.39 MB, losslessly. Introducing tufazip: the best entry on the Large Text Compression Benchmark using a neural network as its sole predictor. Everything above it mixes a transformer with hand-built models. Full post 👇 tufalabs.ai/research/tufazip
82
𝙳☈ retweeted
Just noticed "recursive self-improvement" contains "rsi" in the middle of it...
69
118
16
2,722
91,900