
Prof @UMDCS and @umiacs, Lead Parallel Software & Systems Group @hpc_group. @IITKanpur & @UofIllinois Alum. Ex-@Livermore_Lab. Views are my own.
Joined June 2009
- Tweets510
- Following276
- Followers679
- Likes1.7K
Abhinav Bhatele retweeted
Sending sparse data between GPUs? Unfortunately, existing communication libraries don’t exploit sparsity to speed up performance. Come to room W176abc at 2:35 pm to hear Dalton talk about his work on SpCCL.
It is always great to meet with long time collaborators and friends at @Supercomputing. But I’m also excited to meet with students/postdocs who might be interested in joining our group. Come talk to us!
Replying to @hpc_group
We have an excellent line up of tutorials, posters and papers. If you miss a session and want to get in touch with us, you can find our contact info on our website: pssg.cs.umd.edu/team
#HPC #AI #HPCUnites #SC26 @umdcs @UMDscience @umiacs
2
80
Abhinav Bhatele retweeted
We have arrived at @Supercomputing! Follow along to see all the activities that PSSG members are involved in.
Abhinav Bhatele retweeted
We're hiring! @UMDCS is seeking faculty at all ranks for multiple tenure-track positions across computer science research areas.
Learn more and apply: go.umd.edu/OpenPosition-10-2…
Abhinav Bhatele retweeted
We are pleased to welcome two new PhD students to our group this Fall -- Abhinav Dutta from IIT Patna, and Ryan Himes from UPenn.
We are fortunate that we were able to convince @YafanHuang to join UMDCS. For the first time ever, PSSG has two faculty members instead of one, and I am looking forward to many years of fruitful collaboration with Yafan!
We are excited to announce that Dr. @YafanHuang, who recently started at @umdcs as an assistant professor is joining PSSG as a Co-Director. Dr. Huang is an expert in data compression & fault tolerance methods. We are starting work on several new research projects this year! #HPC
Abhinav Bhatele retweeted
🔊 @UofMaryland Computer Science announced 11 faculty promotions, effective July 1, including four professors, five associate professors, one principal lecturer and one senior lecturer.
go.umd.edu/FacultyPromotions…
Abhinav Bhatele retweeted
🧠 From early programming lessons to scaling #AI on supercomputers, Abhinav Bhatele (@bhatele) shares his path and research at @UofMaryland's Department of Computer Science.
Read more: go.umd.edu/Bhatele-4-2026
🤖 Made with AI
Abhinav Bhatele retweeted
Congratulations to Dr. Joy Kitson for successfully defending her PhD dissertation last Friday (April 10)! That makes her the fifth PhD student to graduate from our group.
Dissertation title: "Scaling Agent-based Epidemic Diffusion on HPC Clusters"
Congratulations to Dr. Cankur and Dr. Davis! Best wishes for a bright future!!
On Mar 25, we set a personal record with 2 PhD defenses in a day! Congrats to @onurcankur & @jhdavis_josh for their excellent work:
Onur's title: "Longitudinal Data Analytics of HPC Systems & Applications"
Josh's title: "LLM-driven Development of Performant & Portable GPU Codes"
10
513
If you are interested in collaborating with us or you are an undergrad planning to apply for a PhD, come and talk to us (don't feel shy!). You'll find me or PSSG students at the events below. #SC25
We have arrived in St. Louis for @Supercomputing 2025. Learn more about our group's research through the various talks, panels and tutorials below. We will also be at the @UofMaryland booth (3123) at various times. #SC25 #UMDCS #HPC #AI #MLSys #AI4Science
Abhinav Bhatele retweeted
Fine-tuning massive LLMs requires reliability at scale. The Lightning open source stack gives you just that.
Check out Democratizing Al: Open-source Scalable LLM Training on GPU-based Supercomputers.
✅ 405B param MoE language model
✅ Fully finetuned on 32,768 GPUs on a massive 1.4 exaflop/s cluster
✅ All running on LitGPT
Led by @siddharth_3773 and @bhatele at @UofMaryland.
Read the paper ➡️ go.lightning.ai/44EjLFu
Use LitGPT ➡️ github.com/Lightning-AI/litg…
Congrats @siddharth_3773! Siddharth is going to join NVIDIA after he graduates to continue working on parallel training and inference.
Follow him to keep track of what interesting stuff he’ll do next 😊
We are on a roll, second successful dissertation defense in a week (March 28)! Congratulations to @siddharth_3773 on becoming the second PhD graduate from PSSG!!
Dissertation title: "Optimizing Communication in Parallel Deep Learning on Exascale-class Machines"
#HPC #AI #HPC4AI
Abhinav Bhatele retweeted
Riken, including our center R-CCS, is hiring researchers big time, from PIs to postdocs to interns. We have the best support for international researches; in fact the % of non-JP researchers at R-CCS is approaching 50 % now. Come work with us! riken.jp/en/careers/openings…
Incredibly proud moment for me as an advisor. The first cohort of students in @hpc_group started in 2020 and they are now starting to graduate. Also a bittersweet moment as we bid adieu to some senior students this Spring & welcome new ones in the Fall. Congrats @DanielNichols10!
We present our very first newly minted Dr. Daniel Nichols (@DanielNichols10), who successfully defended his dissertation today. Congrats & best wishes for a bright future ahead!!
Dissertation Title: "On Learning Behaviors of Parallel Code and Systems Across Modalities"
#HPC #AI
Abhinav Bhatele retweeted
We have a postdoc opening in our group on the intersection of HPC and AI, specifically on developing and applying Code LLMs to HPC software development. Please help us spread the word:
umd.wd1.myworkdayjobs.com/en…
Position will remain open until filled. #HPC #AI
Abhinav Bhatele retweeted
Poster session in progress … if you miss us and want to chat, you can find more information on pssg.cs.umd.edu.
Abhinav Bhatele retweeted
Come see our #NeurIPS2024 poster today on"
Loki: Low-rank Keys for Efficient Sparse Attention"
Paper: arxiv.org/abs/2406.02542
Code: github.com/hpcgroup/loki
Loki is a sparse attention method that reduces the computational and memory costs of LLM inference! By exploiting the low-dimensional space of key vectors in self-attention, Loki achieves faster attention computation without compromising model performance.
Can we get away w/ reducing attention keys to a lower-dimensional space to optimize compute during inference?
@prajwal1210 & @siddharth_3773 investigated using PCA on key vectors & found that the rank of attention keys is much lower than the full dimensionality. #NeurIPS2024
Abhinav Bhatele retweeted
Why is addition hard for next token predictors? Come hear about our fix for this at #NeurIPS!
We’re presenting Abacus Embeddings 🧮 tomorrow (Friday) from 11-2 in the East Exhibit hall at poster 2907.
Drop by to see how we can improve your language model.




























