
The Parallel Software and Systems Group is a research group @UofMaryland @umdcs, directed by Prof. @bhatele. Research on HPC, Vis. and ML. RTs ≠ endorsements.
College Park
Joined April 2020
- Tweets148
- Following91
- Followers383
- Likes86
We have arrived at @Supercomputing! Follow along to see all the activities that PSSG members are involved in.
We have an excellent line up of tutorials, posters and papers. If you miss a session and want to get in touch with us, you can find our contact info on our website: pssg.cs.umd.edu/team
#HPC #AI #HPCUnites #SC26 @umdcs @UMDscience @umiacs
We are pleased to welcome two new PhD students to our group this Fall -- Abhinav Dutta from IIT Patna, and Ryan Himes from UPenn.
Parallel Software and Systems Group retweeted
We are fortunate that we were able to convince @YafanHuang to join UMDCS. For the first time ever, PSSG has two faculty members instead of one, and I am looking forward to many years of fruitful collaboration with Yafan!
We are excited to announce that Dr. @YafanHuang, who recently started at @umdcs as an assistant professor is joining PSSG as a Co-Director. Dr. Huang is an expert in data compression & fault tolerance methods. We are starting work on several new research projects this year! #HPC
Parallel Software and Systems Group retweeted
It’s great to join PSSG and UMDCS! Let’s do interesting HPC research! 😊
We are excited to announce that Dr. @YafanHuang, who recently started at @umdcs as an assistant professor is joining PSSG as a Co-Director. Dr. Huang is an expert in data compression & fault tolerance methods. We are starting work on several new research projects this year! #HPC
We are excited to announce that Dr. @YafanHuang, who recently started at @umdcs as an assistant professor is joining PSSG as a Co-Director. Dr. Huang is an expert in data compression & fault tolerance methods. We are starting work on several new research projects this year! #HPC
Parallel Software and Systems Group retweeted
🔊 @UofMaryland Computer Science announced 11 faculty promotions, effective July 1, including four professors, five associate professors, one principal lecturer and one senior lecturer.
go.umd.edu/FacultyPromotions…
Parallel Software and Systems Group retweeted
🧠 From early programming lessons to scaling #AI on supercomputers, Abhinav Bhatele (@bhatele) shares his path and research at @UofMaryland's Department of Computer Science.
Read more: go.umd.edu/Bhatele-4-2026
🤖 Made with AI
Congratulations to Dr. Joy Kitson for successfully defending her PhD dissertation last Friday (April 10)! That makes her the fifth PhD student to graduate from our group.
Dissertation title: "Scaling Agent-based Epidemic Diffusion on HPC Clusters"
On Mar 25, we set a personal record with 2 PhD defenses in a day! Congrats to @onurcankur & @jhdavis_josh for their excellent work:
Onur's title: "Longitudinal Data Analytics of HPC Systems & Applications"
Josh's title: "LLM-driven Development of Performant & Portable GPU Codes"
We have arrived in St. Louis for @Supercomputing 2025. Learn more about our group's research through the various talks, panels and tutorials below. We will also be at the @UofMaryland booth (3123) at various times. #SC25 #UMDCS #HPC #AI #MLSys #AI4Science
The Distributed Deep Learning tutorial is happening right now in Room 123. #SC25 #DistributedAI #AxoNN #YALIS
All the tutorial material is on github: github.com/axonn-ai/distrib-…
2
151
Come work with us! All of our PhD students will be @Supercomputing, don’t feel shy if you want to chat.
3
206
We are on a roll, second successful dissertation defense in a week (March 28)! Congratulations to @siddharth_3773 on becoming the second PhD graduate from PSSG!!
Dissertation title: "Optimizing Communication in Parallel Deep Learning on Exascale-class Machines"
#HPC #AI #HPC4AI
We present our very first newly minted Dr. Daniel Nichols (@DanielNichols10), who successfully defended his dissertation today. Congrats & best wishes for a bright future ahead!!
Dissertation Title: "On Learning Behaviors of Parallel Code and Systems Across Modalities"
#HPC #AI
We have a postdoc opening in our group on the intersection of HPC and AI, specifically on developing and applying Code LLMs to HPC software development. Please help us spread the word:
umd.wd1.myworkdayjobs.com/en…
Position will remain open until filled. #HPC #AI
Can we get away w/ reducing attention keys to a lower-dimensional space to optimize compute during inference?
@prajwal1210 & @siddharth_3773 investigated using PCA on key vectors & found that the rank of attention keys is much lower than the full dimensionality. #NeurIPS2024
We show that our method, Loki, achieves better accuracies compared to other sparse attention methods for short-context and long-context tasks across a range of models.
Paper: arxiv.org/abs/2406.02542
Code: github.com/hpcgroup/loki
#NeurIPS2024 #HPC #AI
1
1
248
Poster session in progress … if you miss us and want to chat, you can find more information on pssg.cs.umd.edu.
Parallel Software and Systems Group retweeted
Come see our #NeurIPS2024 poster today on"
Loki: Low-rank Keys for Efficient Sparse Attention"
Paper: arxiv.org/abs/2406.02542
Code: github.com/hpcgroup/loki
Loki is a sparse attention method that reduces the computational and memory costs of LLM inference! By exploiting the low-dimensional space of key vectors in self-attention, Loki achieves faster attention computation without compromising model performance.
Can we get away w/ reducing attention keys to a lower-dimensional space to optimize compute during inference?
@prajwal1210 & @siddharth_3773 investigated using PCA on key vectors & found that the rank of attention keys is much lower than the full dimensionality. #NeurIPS2024
Parallel Software and Systems Group retweeted
Want to learn about our findings around the low-rank structure of attention keys, and how to use it to improve sparse attention?
Come to our poster at #NeurIPS2024 tomorrow (Friday)
at the East Exhibit Hall from 11-2 (Poster #2000)!
Can we get away w/ reducing attention keys to a lower-dimensional space to optimize compute during inference?
@prajwal1210 & @siddharth_3773 investigated using PCA on key vectors & found that the rank of attention keys is much lower than the full dimensionality. #NeurIPS2024


























