Meta RE. Performance Optimization and HPC for AI.
NYC
Joined June 2024
- Tweets10
- Following575
- Followers11
- Likes13
Alexander Erben retweeted
Three years and 100+ projects in, our mission is the same: give everyone clear, trusted insight into where AI is headed. Our new post unpacks the principles behind every research choice—why we take some ideas on and pass on others.
epoch.ai/blog/what-is-epoch
Alexander Erben retweeted
Dozens of teams have asked my advice on running LLMs. How fast is @deepseek_ai V3 with vLLM on 8 GPUs? What's the max throughput of @Alibaba_Qwen 2.5 Coder with SGLang on one H100?
Running & sharing benchmarks ad hoc was too slow
So we built a tiny app, the LLM Engine Advisor
Alexander Erben retweeted
Introducing PCCL, the Prime Collective Communications Library — a low-level communication library built for decentralized training over the public internet, with fault tolerance as a core design principle.
In testing, PCCL achieves up to 45 Gbit/s of bandwidth across datacenters in Europe and 25 GBit/s training intercontinental across North America and Europe.
Alexander Erben retweeted
Update to the GPU Glossary with probably the highest ratio of effort expended to tokens produced: a breakdown of the SASS instructions for a very simple matmul-and-add (mma) using a Tensor Core, courtesy of godbolt.
modal.com/gpu-glossary/devic…
Just attended my first #vLLM NYC meetup after moving to the US - cool to find a community equally interested in performance optimization and making AI accessible.
S/O to @robertnishihara who explained Ray to me like I am not supposed to know it already 😅
Alexander Erben retweeted
We’ve added four new benchmarks to the Epoch AI Benchmarking Hub: Aider Polyglot, WeirdML, Balrog, and Factorio Learning Environment!
Before we only featured our own evaluation results, but this new data comes from trusted external leaderboards. And we've got more on the way 🧵
