Parallel Software and Systems Group (@hpc_group) header
Parallel Software and Systems Group (@hpc_group) profile picture

The Parallel Software and Systems Group is a research group @UofMaryland @umdcs, directed by Prof. @bhatele. Research on HPC, Vis. and ML. RTs ≠ endorsements.

College Park
Joined April 2020
Photo by Parallel Software and Systems Group (@hpc_group)Photo by Parallel Software and Systems Group (@hpc_group)Photo by Parallel Software and Systems Group (@hpc_group)Photo by Parallel Software and Systems Group (@hpc_group)Photo by Parallel Software and Systems Group (@hpc_group)Photo by Parallel Software and Systems Group (@hpc_group)Photo by Parallel Software and Systems Group (@hpc_group)Photo by Parallel Software and Systems Group (@hpc_group)Photo by Parallel Software and Systems Group (@hpc_group)Photo by Parallel Software and Systems Group (@hpc_group)
We have arrived at @Supercomputing! Follow along to see all the activities that PSSG members are involved in.
1
2
7
340
We have an excellent line up of tutorials, posters and papers. If you miss a session and want to get in touch with us, you can find our contact info on our website: pssg.cs.umd.edu/team #HPC #AI #HPCUnites #SC26 @umdcs @UMDscience @umiacs
1
1
4
150
Sending sparse data between GPUs? Unfortunately, existing communication libraries don’t exploit sparsity to speed up performance. Come to room W176abc at 2:35 pm to hear Dalton talk about his work on SpCCL.
2
7
105
We are pleased to welcome two new PhD students to our group this Fall -- Abhinav Dutta from IIT Patna, and Ryan Himes from UPenn.
1
2
6
319
Parallel Software and Systems Group retweeted
We are fortunate that we were able to convince @YafanHuang to join UMDCS. For the first time ever, PSSG has two faculty members instead of one, and I am looking forward to many years of fruitful collaboration with Yafan!
We are excited to announce that Dr. @YafanHuang, who recently started at @umdcs as an assistant professor is joining PSSG as a Co-Director. Dr. Huang is an expert in data compression & fault tolerance methods. We are starting work on several new research projects this year! #HPC
1
2
14
1,089
Parallel Software and Systems Group retweeted
It’s great to join PSSG and UMDCS! Let’s do interesting HPC research! 😊
We are excited to announce that Dr. @YafanHuang, who recently started at @umdcs as an assistant professor is joining PSSG as a Co-Director. Dr. Huang is an expert in data compression & fault tolerance methods. We are starting work on several new research projects this year! #HPC
1
3
145
We are excited to announce that Dr. @YafanHuang, who recently started at @umdcs as an assistant professor is joining PSSG as a Co-Director. Dr. Huang is an expert in data compression & fault tolerance methods. We are starting work on several new research projects this year! #HPC
1
2
7
1,495
Parallel Software and Systems Group retweeted
🔊 @UofMaryland Computer Science announced 11 faculty promotions, effective July 1, including four professors, five associate professors, one principal lecturer and one senior lecturer. go.umd.edu/FacultyPromotions…
3
1
8
1,856
Parallel Software and Systems Group retweeted
🧠 From early programming lessons to scaling #AI on supercomputers, Abhinav Bhatele (@bhatele) shares his path and research at @UofMaryland's Department of Computer Science. Read more: go.umd.edu/Bhatele-4-2026
🤖 Made with AI
2
15
1,271
Congratulations to Dr. Joy Kitson for successfully defending her PhD dissertation last Friday (April 10)! That makes her the fifth PhD student to graduate from our group. Dissertation title: "Scaling Agent-based Epidemic Diffusion on HPC Clusters"
1
5
242
On Mar 25, we set a personal record with 2 PhD defenses in a day! Congrats to @onurcankur & @jhdavis_josh for their excellent work: Onur's title: "Longitudinal Data Analytics of HPC Systems & Applications" Josh's title: "LLM-driven Development of Performant & Portable GPU Codes"
1
1
8
737
We have arrived in St. Louis for @Supercomputing 2025. Learn more about our group's research through the various talks, panels and tutorials below. We will also be at the @UofMaryland booth (3123) at various times. #SC25 #UMDCS #HPC #AI #MLSys #AI4Science
1
4
1
10
1,572
Come work with us! All of our PhD students will be @Supercomputing, don’t feel shy if you want to chat.
A large number of PhD students in my group have graduated or will be graduating by Spring, so I am recruiting several PhD students for the next admission cycle (Fall 2026). If you want to work with us, apply by Dec 5 and drop me a short email. Please repost/share widely. #HPC #AI
3
206
We are on a roll, second successful dissertation defense in a week (March 28)! Congratulations to @siddharth_3773 on becoming the second PhD graduate from PSSG!! Dissertation title: "Optimizing Communication in Parallel Deep Learning on Exascale-class Machines" #HPC #AI #HPC4AI
1
1
13
1,331
We present our very first newly minted Dr. Daniel Nichols (@DanielNichols10), who successfully defended his dissertation today. Congrats & best wishes for a bright future ahead!! Dissertation Title: "On Learning Behaviors of Parallel Code and Systems Across Modalities" #HPC #AI
1
10
1,556
We have a postdoc opening in our group on the intersection of HPC and AI, specifically on developing and applying Code LLMs to HPC software development. Please help us spread the word: umd.wd1.myworkdayjobs.com/en… Position will remain open until filled. #HPC #AI
2
6
644
Can we get away w/ reducing attention keys to a lower-dimensional space to optimize compute during inference? @prajwal1210 & @siddharth_3773 investigated using PCA on key vectors & found that the rank of attention keys is much lower than the full dimensionality. #NeurIPS2024
2
4
14
8,495
We show that our method, Loki, achieves better accuracies compared to other sparse attention methods for short-context and long-context tasks across a range of models. Paper: arxiv.org/abs/2406.02542 Code: github.com/hpcgroup/loki #NeurIPS2024 #HPC #AI
1
1
248
Poster session in progress … if you miss us and want to chat, you can find more information on pssg.cs.umd.edu.
2
5
538
Parallel Software and Systems Group retweeted
Come see our #NeurIPS2024 poster today on" Loki: Low-rank Keys for Efficient Sparse Attention" Paper: arxiv.org/abs/2406.02542 Code: github.com/hpcgroup/loki Loki is a sparse attention method that reduces the computational and memory costs of LLM inference! By exploiting the low-dimensional space of key vectors in self-attention, Loki achieves faster attention computation without compromising model performance.
Can we get away w/ reducing attention keys to a lower-dimensional space to optimize compute during inference? @prajwal1210 & @siddharth_3773 investigated using PCA on key vectors & found that the rank of attention keys is much lower than the full dimensionality. #NeurIPS2024
1
10
43
6,025
Parallel Software and Systems Group retweeted
Want to learn about our findings around the low-rank structure of attention keys, and how to use it to improve sparse attention? Come to our poster at #NeurIPS2024 tomorrow (Friday) at the East Exhibit Hall from 11-2 (Poster #2000)!
Can we get away w/ reducing attention keys to a lower-dimensional space to optimize compute during inference? @prajwal1210 & @siddharth_3773 investigated using PCA on key vectors & found that the rank of attention keys is much lower than the full dimensionality. #NeurIPS2024
1
5
396