Abhinav Bhatele (@bhatele) header
Abhinav Bhatele (@bhatele) profile picture

Prof @UMDCS and @umiacs, Lead Parallel Software & Systems Group @hpc_group. @IITKanpur & @UofIllinois Alum. Ex-@Livermore_Lab. Views are my own.

Joined June 2009
Photo by Abhinav Bhatele (@bhatele)Photo by Abhinav Bhatele (@bhatele)Photo by Abhinav Bhatele (@bhatele)Photo by Abhinav Bhatele (@bhatele)Photo by Abhinav Bhatele (@bhatele)Photo by Abhinav Bhatele (@bhatele)Photo by Abhinav Bhatele (@bhatele)Photo by Abhinav Bhatele (@bhatele)Photo by Abhinav Bhatele (@bhatele)
Sending sparse data between GPUs? Unfortunately, existing communication libraries don’t exploit sparsity to speed up performance. Come to room W176abc at 2:35 pm to hear Dalton talk about his work on SpCCL.
2
7
105
It is always great to meet with long time collaborators and friends at @Supercomputing. But I’m also excited to meet with students/postdocs who might be interested in joining our group. Come talk to us!
Replying to @hpc_group
We have an excellent line up of tutorials, posters and papers. If you miss a session and want to get in touch with us, you can find our contact info on our website: pssg.cs.umd.edu/team #HPC #AI #HPCUnites #SC26 @umdcs @UMDscience @umiacs
2
80
We have arrived at @Supercomputing! Follow along to see all the activities that PSSG members are involved in.
1
2
7
340
We're hiring! @UMDCS is seeking faculty at all ranks for multiple tenure-track positions across computer science research areas. Learn more and apply: go.umd.edu/OpenPosition-10-2…
1
14
3
43
12,789
We are pleased to welcome two new PhD students to our group this Fall -- Abhinav Dutta from IIT Patna, and Ryan Himes from UPenn.
1
2
6
319
We are fortunate that we were able to convince @YafanHuang to join UMDCS. For the first time ever, PSSG has two faculty members instead of one, and I am looking forward to many years of fruitful collaboration with Yafan!
We are excited to announce that Dr. @YafanHuang, who recently started at @umdcs as an assistant professor is joining PSSG as a Co-Director. Dr. Huang is an expert in data compression & fault tolerance methods. We are starting work on several new research projects this year! #HPC
1
2
14
1,089
🔊 @UofMaryland Computer Science announced 11 faculty promotions, effective July 1, including four professors, five associate professors, one principal lecturer and one senior lecturer. go.umd.edu/FacultyPromotions…
3
1
8
1,856
🧠 From early programming lessons to scaling #AI on supercomputers, Abhinav Bhatele (@bhatele) shares his path and research at @UofMaryland's Department of Computer Science. Read more: go.umd.edu/Bhatele-4-2026
🤖 Made with AI
2
15
1,271
Congratulations to Dr. Joy Kitson for successfully defending her PhD dissertation last Friday (April 10)! That makes her the fifth PhD student to graduate from our group. Dissertation title: "Scaling Agent-based Epidemic Diffusion on HPC Clusters"
1
5
242
Congratulations to Dr. Cankur and Dr. Davis! Best wishes for a bright future!!
On Mar 25, we set a personal record with 2 PhD defenses in a day! Congrats to @onurcankur & @jhdavis_josh for their excellent work: Onur's title: "Longitudinal Data Analytics of HPC Systems & Applications" Josh's title: "LLM-driven Development of Performant & Portable GPU Codes"
10
513
If you are interested in collaborating with us or you are an undergrad planning to apply for a PhD, come and talk to us (don't feel shy!). You'll find me or PSSG students at the events below. #SC25
We have arrived in St. Louis for @Supercomputing 2025. Learn more about our group's research through the various talks, panels and tutorials below. We will also be at the @UofMaryland booth (3123) at various times. #SC25 #UMDCS #HPC #AI #MLSys #AI4Science
1
6
1,451
A large number of PhD students in my group have graduated or will be graduating by Spring, so I am recruiting several PhD students for the next admission cycle (Fall 2026). If you want to work with us, apply by Dec 5 and drop me a short email. Please repost/share widely. #HPC #AI
9
64
4
212
38,373
Fine-tuning massive LLMs requires reliability at scale. The Lightning open source stack gives you just that. Check out Democratizing Al: Open-source Scalable LLM Training on GPU-based Supercomputers. ✅ 405B param MoE language model ✅ Fully finetuned on 32,768 GPUs on a massive 1.4 exaflop/s cluster ✅ All running on LitGPT Led by @siddharth_3773 and @bhatele at @UofMaryland. Read the paper ➡️ go.lightning.ai/44EjLFu Use LitGPT ➡️ github.com/Lightning-AI/litg…
4
9
72
5,066
Congrats @siddharth_3773! Siddharth is going to join NVIDIA after he graduates to continue working on parallel training and inference. Follow him to keep track of what interesting stuff he’ll do next 😊
We are on a roll, second successful dissertation defense in a week (March 28)! Congratulations to @siddharth_3773 on becoming the second PhD graduate from PSSG!! Dissertation title: "Optimizing Communication in Parallel Deep Learning on Exascale-class Machines" #HPC #AI #HPC4AI
1
13
888
Riken, including our center R-CCS, is hiring researchers big time, from PIs to postdocs to interns. We have the best support for international researches; in fact the % of non-JP researchers at R-CCS is approaching 50 % now. Come work with us! riken.jp/en/careers/openings…
15
29
12,624
Incredibly proud moment for me as an advisor. The first cohort of students in @hpc_group started in 2020 and they are now starting to graduate. Also a bittersweet moment as we bid adieu to some senior students this Spring & welcome new ones in the Fall. Congrats @DanielNichols10!
We present our very first newly minted Dr. Daniel Nichols (@DanielNichols10), who successfully defended his dissertation today. Congrats & best wishes for a bright future ahead!! Dissertation Title: "On Learning Behaviors of Parallel Code and Systems Across Modalities" #HPC #AI
2
22
1,223
We have a postdoc opening in our group on the intersection of HPC and AI, specifically on developing and applying Code LLMs to HPC software development. Please help us spread the word: umd.wd1.myworkdayjobs.com/en… Position will remain open until filled. #HPC #AI
2
6
644
Poster session in progress … if you miss us and want to chat, you can find more information on pssg.cs.umd.edu.
2
5
538
Abhinav Bhatele retweeted
Come see our #NeurIPS2024 poster today on" Loki: Low-rank Keys for Efficient Sparse Attention" Paper: arxiv.org/abs/2406.02542 Code: github.com/hpcgroup/loki Loki is a sparse attention method that reduces the computational and memory costs of LLM inference! By exploiting the low-dimensional space of key vectors in self-attention, Loki achieves faster attention computation without compromising model performance.
Can we get away w/ reducing attention keys to a lower-dimensional space to optimize compute during inference? @prajwal1210 & @siddharth_3773 investigated using PCA on key vectors & found that the rank of attention keys is much lower than the full dimensionality. #NeurIPS2024
1
10
43
6,025
Abhinav Bhatele retweeted
Why is addition hard for next token predictors? Come hear about our fix for this at #NeurIPS! We’re presenting Abacus Embeddings 🧮 tomorrow (Friday) from 11-2 in the East Exhibit hall at poster 2907. Drop by to see how we can improve your language model.
Introducing 🧮Abacus Embeddings, a simple tweak to positional embeddings that enables LLMs to do addition, multiplication, sorting, and more. Our Abacus Embeddings trained only on 20-digit addition generalise near perfectly to 100+ digits. 1/n
12
1
24
3,766