
Joe Isaacson retweeted
Would you rather have better documentation for PyTorch prototype features but more BC breakages or secret APIs until we have more confidence?
🚀🚀🚀 come work with PyTorch on GPU kernel optimization, distributed training/inference/post-training, compiler design, ABI stability
If you’re excited about optimizing code that runs equally well on a single or thousands of GPUs and if you have the ability to submit a single substantial PR to a major OSS library, we want you on the PyTorch team - especially if you’re early in your career.
2
673
Excited to share the GPU MODE kernel authoring competition! We’re stoked to see the creative solutions people come up with to push bounds of performance. And, all for a good cause — towards a public dataset of performant kernels 😁
Great talk on the importance of RLEF and CoT for coding generation performance
Just gave a talk on "Grounding LLMs in Code Execution" at the NeurIPS Hacker-Cup AI Competition, here are the slides docs.google.com/presentation…
1
1
434
David Harmeyer on Facebook’s history of Hacker Cup. How to add N flights to N airports to minimize travel distance.
Stoked for the Hacker AI Cup Workshop to start at @NeurIPSConf ! Speakers @weiwei_msr @syhw @neal_wu @simran_s_arora @MishaLaskin @thomascapellajp @marksaroufim ! Come join, room 210
Congrats on the launch of 🌽 KernelBench! We're excited to partner with 🍿 Popcorn to lower the barrier to GPU programming!
Replying to @simonguozirui
We are not the only ones thinking about this direction. The vision was nicely outlined by @karpathy in his talk (see picture below, credit to karpathy and @accel), and we found many others also eager to attempt this.
Project 🍿: gpu-mode.github.io/popcorn/ in the GPU Mode server is a community effort towards this goal of LLMs generating performant GPU kernels. We hope KernelBench could be a valuable contribution for the community, to help understand current capabilities and serve as a playground to evaluate various techniques.
If you are interested, we would love to hear your thoughts and welcome you to join the mission!
Link to the relevant part of Karpathy’s talk: youtu.be/aR6CzM0x-g0?feature…
Joe Isaacson retweeted
We ran a lot of evals recently using Mistral Large 2 for the hacker cup challenges, Weave’s native integration with the Mistral SDk was super useful to understand token usage and latency, as well as measure accuracy
🚀 Just launched: a guide to observability with @MistralAI!
🔍 Why observability? For visibility, production requirements, reproducibility, & continuous improvement in LLM systems.
😎Monitor input prompts, models, & outputs at the individual level. Track workflow patterns, tokens, latency, anomalies, errors, & quality at the application level.
🫶Seamless integrations with @langchain LangSmith, @langfuse, @ArizePhoenix, @wandb, @promptlayer, @AgentOpsAI
docs.mistral.ai/guides/obser…
Joe Isaacson retweeted
PyTorch has historically been super slow for a lot of "small" RL workloads, but the reason is kinda dumb. RL often uses tiny neural networks, and end up ridiculously CPU overhead bound.
Luckily, with cudagraphs and torch.compile, we often see >5x speedups!
I remember the first time @kevin_zakka showed me a profile of a RL setting he was looking at, I was aghast at how low the GPU utilization was :P
But cudagraphs is ridiculously great at improving your CPU utilization :) Go check out LeanRL to see how it's done.
Today we're opensourcing a LeanRL, a simple RL library that provides recipes for fast RL training using torch.compile and cudagraphs.
Using these, we get >6x speed-ups compared to the original CleanRL implementations.
github.com/pytorch-labs/lean…
A thread ⬇️
A huge thanks to @caseyaylward and the @Accel team for running the highest quality sys ml hackathon I’ve ever see . It was great to meet so many GPU MODE friends IRL, and hack on torchao. And Congrats to the winners for fast flexible attention!
Replying to @caseyaylward
14/ We had so much fun bringing this special community together IRL for an amazing day of technical talks/hacking. Big thanks to everyone who turned out and our planning committee (here’s an “after” photo taken at midnight). Full videos of the talks to follow soon #cudamodeirl.
5
144
Joe Isaacson retweeted
From the beginning, @neurosp1ke and I wanted a community that was cozy, inclusive and fun where you could make friends and learn how to make your code faster
Anyone who shares our ideals is welcome. We’re announcing we’re changing the name of our community to “GPU MODE”.
Excited for our release of LeanRL, showcasing the simplicity and performance of torch compile across a breath of RL Models. Big shout out to @vwxyzjn for the inspiration and support
Today we're opensourcing a LeanRL, a simple RL library that provides recipes for fast RL training using torch.compile and cudagraphs.
Using these, we get >6x speed-ups compared to the original CleanRL implementations.
github.com/pytorch-labs/lean…
A thread ⬇️
Joe Isaacson retweeted
Registration is open for the #NeurIPS2024 HackerCup, the world's first AI programming competition by Microsoft Research, Meta, and PyTorch. Test your AI on the programming problems that challenge the best human minds and see how your AI ranks. Learn more. msft.it/6013lm3f3
Joe Isaacson retweeted
We recently enabled quantization-aware training in torchtune using torchao. QAT helps recover any quality degradation resulting from quantizing the model - especially useful when you’re resource constrained at inference time.
👇🏽awesome work by Andrew Or!
pytorch.org/blog/quantizatio…
I'm beyond stoked to be hosting the first IRL hackathon for the CUDA MODE community with @Accel @nvidia and @PyTorch ! Get ready to hack across a range of AI performance -- inference optimization, custom kernels, low bit dtypes, sparsification, ...
This September, we’re hosting a CUDA MODE hackathon alongside our friends at @nvidia and @PyTorch. Join us in San Francisco to spend the day hacking on new projects and hear from some of the most well-respected ML leaders at @Meta/@PyTorch, @EurekaLabsAI, @vllm_project, @togethercompute, @nvidia, and more. ⬇️
Check out the first AI track of the Meta Hacker Cup ! 5 rounds, some of the hardest problems in code generation, same problems released to human and AI competitors this year. Large Discord group with competitors, just getting started
codeforces.com/blog/entry/13…






















