🇵🇱 dead meme connoisseur // bring back bringing back // 25 // noscope or no hope
Sagittarius A*
Joined October 2014
- Tweets3.8K
- Following1.6K
- Followers142
- Likes106K
please scale please scale please scale please scale
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
Neverduft retweeted
波兰牛肉.mp4
Dancing Polish Neuro.mp4
Neurosama吸WannaCry吸嗨了（×）
由于过分追求把腿绷直导致重心完全不稳，应该竖下来的）
拍的时候被人当啥子看了（虽然好像本来就是啥子）
北京大学百团大战day2
#neurosama #heartheartart #kigurumi #cosplay #着ぐるみ
2016 era Robocraft my beloved 😭 I've always loved games that let you get really creative in functional ways
Neverduft retweeted
you don’t actually have to be “pro ai” you just have to realize that it’s likely our only chance to avoid an even more certain ruin
