All computers, all the time - find me on Urbit, or pick up a phone. (You got my number, right?)
41.6970, -86.2375
Joined October 2007
- Tweets3K
- Following1.8K
- Followers480
- Likes26.9K
This morning upgraded to Strata 0.1.56 engine and I am pretty sure my decode performance went up - I just saw it hit 110 t/s decode. (I've never seen that before!) 64GB DDR4 and a 5060Ti 16GB, this is q2_0 with 256K context window. He's explaining the Mistral Large 4 paper to me.
@[email protected]🐊🐊🐀 retweeted
We tasked Mistral Large 4 with creating a real-time, 3D tracker of the roughly 19,000 satellites and debris fragments orbiting Earth.
The model built a working app, tested it like a user, made it fast, and passed strict verification.
Please meet Orbital Trace :
@[email protected]🐊🐊🐀 retweeted
Mistral VP of Science @PierreStock says they have efficiency in their DNA and people should give them more money:
"Now, it's 300 people, and [Le Chonk] we managed to train it on only 10 megawatts of compute from scratch. 10 megawatts of compute is roughly 4,000 GB300s, to give you an order of magnitude."
"That's super low. It's a fraction of what every competitor uses, and we trained it on our own compute."
"It's pretty amazing, and we really got efficiency in our DNA. But now, the thing I want to tell everyone is that it means that you should give us more money because the yield is there....Please give us 1 gigawatt."
"We've now spent one year building the reinforcement learning machine in training… [Le Chonk] is the best open-weight model in the world on cyber defense, on coding, and on coding agents."
ML4 is now open-weight! Frontier capabilities on code agents & cyber, way more efficient and compact than its Preview version three weeks ago (!) and cherry on the cake: technical report telling its epic backstory. Check it out: huggingface.co/mistralai/Mis…
*in bed with hyperscalers - This is naive. As I've done more research, I now understand the Engram/PLE is patented technology that you are granted a conditional license to use as long as you look the other way, if Alibaba violates your patents. (I haven't got any patents, but...)
Communism
i think Sam is right. local inference is a terrible allocation of scarce resources. the state should allocate you resources like compute, food, housing, phones, heck even large pools of equipment for other fields, from a central reserve. that way we don't waste resources on inefficiencies for individual whimsy
Already working on my deployment of Le Chonk - we're going for proof of life with pipeline parallelism because I can't displace Nemotron-3-Ultra (folks are in literal love and I'm fearful about a revolt if I have to tell them it's going away)
@[email protected]🐊🐊🐀 retweeted
Simply not true. I recently optimized a mid sized data center. Their performances went up by more than 600% (around 830% on ppts and 480% on throughput) on just few iterations. Data centers at their current stage are wasting the most DRAM in the world…
sam: local inference is a terrible allocation of scarce resources
“a lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we're going to be able to run them on our computers, and so we're not going to need all the stuff in the data centre
but from a global allocation perspective, actually it's quite terrible to have everybody, like, have a bunch of the world's limited supply of DRAM sitting in their own computer and being used five percent of the time
you want maximum utilisation in the cloud if you're trying to deal with this very supply-constrained environment
somehow many people's intuition is the opposite of that”
@[email protected]🐊🐊🐀 retweeted
nitter.cf/duolingo/status/211481… because Merriam Webster says it's a word so hey
This quoted post is unavailable.