Jim B retweeted
Still feels surreal to have been on stage with one of the greatest CEO's of our time Dr Lisa Su where she explicitly called out GPU MODE and the work we did to enable the world's first $100K competitive kernel competition
I never in a thousand years would have imagined that a humble reading group would morph into a machine that can generate more kernel data than what exists on all of Github combined and that the best members of our community could beat the best baselines by the best human experts
Thank you GPU MODE, my favorite corner of the internet
Who would have thought it???
New modeling study finds that if everyone in the United Kingdom had worn N95 or FFP2 masks from the start of the pandemic, the rate of COVID-19 transmission would have been 9 times lower.
ALT What if everyone had masked up? Analysis of app data has an answer Modelling study based on almost 250,000 positive COVID-19 tests in the United Kingdom shows that universal masking could have cut transmission markedly.
39
Jim B retweeted
babe wake up, new muP paper dropped
arxiv.org/abs/2407.17465
And holy smokes does this look promising!
Jim B retweeted
Scalify
scale propagation for efficient low-precision LLM training
Low-precision formats such as float8 have been introduced in machine learning accelerated hardware to improve computational efficiency for large language models training and inference. Nevertheless, adoption
Jim B retweeted
It’s impossible not to cry every day. This disease has taken everything. For a year and a half I’ve lived in a nightmare and made no progress getting out. Severe #LongCovid is something no one should have to endure.
Jim B retweeted
Exciting news: Graphcore joins @SoftBank_Group to build next generation of AI compute. graphcore.ai/posts/graphcore…
Jim B retweeted
I had so many dreams.
I wanted my own science show on Netflix.
I wanted to start a family with Kyle.
I wanted to scuba in Australia.
Now, I just want to walk again.
I want to have a real conversation.
I want to read a book again.
I want to listen to music and see my family.
Jim B retweeted
Super long 🧵 incoming. I really enjoy looking at the full stack of computing - maximizing intelligence per picojoule.
Been wanting to do a far-too-in-depth overview of the ENTIRE thing - from manufacturing a GPU to training a GPT :D
Here we go!
Jim B retweeted
Hi everyone yes, I left OpenAI yesterday. First of all nothing "happened" and it’s not a result of any particular event, issue or drama (but please keep the conspiracy theories coming as they are highly entertaining :)). Actually, being at OpenAI over the last ~year has been really great - the team is really strong, the people are wonderful, and the roadmap is very exciting, and I think we all have a lot to look forward to. My immediate plan is to work on my personal projects and see what happens. Those of you who’ve followed me for a while may have a sense for what that might look like ;) Cheers
Anyone else experience a trickle new followers with fake looking profiles in the past couple of weeks?
4
51
Jim B retweeted
8-bit floating point is the next tour de force to train and deploy LLMs.
Today we present our new @graphcoreai preprint:
📢 Training and inference of large language models
using 8-bit floating point 📢
arxiv.org/abs/2309.17224
Details on the 🧵[1/7]
Jim B retweeted
We are excited to release FastChat-T5: our compact and commercial-friendly chatbot!
- Fine-tuned from Flan-T5, ready for commercial usage!
- Outperforms Dolly-V2 with 4x fewer parameters.
Link: github.com/lm-sys/FastChat#F…
Jim B retweeted
Our new paper Unit Scaling enables FP16/FP8 training *out-of-the-📦*
No need for loss scaling / complex automatic scaling methods 🙌 Hoping to change the way we think about low-precision training.
Paper: arxiv.org/abs/2303.11257
Notebook: ipu.dev/AzbWWv
🧵[1/n]
Jim B retweeted
I just got off the Zoom with @iScienceLuvr and @johnowhitaker where we completed recording the final @fastdotai part 2 video -- there are 18 of them!
I'm working on creating the website for the new course now and will let you all know as soon as it's ready! :D
Jim B retweeted
Those who partake in "culture wars" believe it's about the future direction of society, perhaps even the survival of their idea of civilization.
The more mundane reality: it's about siphoning away the savings of gullible marks & consolidating the political power of a few conmen.
😍😍 literally my first computer...!
Just as slow as I remembered 😂
I even have to reduce the screen size in doom to get a better frame rate like the old days.
Didn't have real speakers though... PC speaker FTW.
Hidden feature at @huggingface for the nostalgic ones
Run windows 3.1 and play doom in it huggingface.co/spaces/coolsp…
1