Christos Perivolaropoulos (@ccperivol) header
Christos Perivolaropoulos (@ccperivol) profile picture

Computer science understander @GoogleDeepmind

Cambridge
Joined September 2021
Photo by Christos Perivolaropoulos (@ccperivol)
Christos Perivolaropoulos retweeted
🎉Very happy that our paper on the relationship between attention sinks and information flow in LLMs has been accepted to @iclr_conf ! 🇧🇷
🚨 How do attention sinks relate to information flow in LLMs? We show how massive activations create attention sinks and compression valleys, revealing a three-stage theory of information flow in LLMs. 🧵 w/ Enrique* @fedzbar @epomqo @mmbronstein @ylecun @ziv_ravid
5
47
4,237
Christos Perivolaropoulos retweeted
Want to improve GPU compute/comms overlap? We just published a new short tutorial for you! A few small changes to the Pallas:MGPU matmul kernel is all it takes to turn it into an all-gather collective matmul that overlaps NVLINK comms with local compute: docs.jax.dev/en/latest/palla…
8
46
1
305
33,892
Christos Perivolaropoulos retweeted
What happens when a pioneering artist hands creative control to an AI? Announcing "Evolution and Foundation," a new London exhibition where Google's Gemini acts as the artist, directing the evolution of complex 3D art. 🗓️ Oct 17-26, 2025 📍 Oxo Gallery, London (@oxotowerwharf)
2
4
5
1,485
Christos Perivolaropoulos retweeted
Curious how to write SOTA performance Blackwell matmul kernels using MGPU? We just published a short step-by-step tutorial: docs.jax.dev/en/latest/palla… At each step, we show exactly what (small) changes are necessary to refine the kernel and the final kernel is just under 150 lines.
4
66
7
413
55,644
Christos Perivolaropoulos retweeted
AlphaEvolve is here! 🧬 this is one special system (especially when optimising things with jagged edges 😊) -- had a fantastic time using it! congrats @SashaVNovikov @matejbalog @ThuyNganVu and team!! 🚀 you can register your interest in using it through the link below:
Introducing AlphaEvolve: a Gemini-powered coding agent for algorithm discovery. It’s able to: 🔘 Design faster matrix multiplication algorithms 🔘 Find new solutions to open math problems 🔘 Make data centers, chip design and AI training more efficient across @Google. 🧵
1
10
76
6,074
Christos Perivolaropoulos retweeted
Vol:18 No:5 → Dandelion: Smaller Clusters, Bigger Speeds—Distributed Transactions Redefined vldb.org/pvldb/vol18/p1264-k…
4
1
24
1,766
Christos Perivolaropoulos retweeted
Super excited to be heading to Singapore tomorrow to present our work on RoPE with Alex, @ccperivol, Razvan, @PetarV_93. Christos and I will be presenting on Fri 25 Apr 7 p.m. PDT — 9:30 p.m. PDT Hall 3 + Hall 2B #242. Happy to meet and catch up :) DMs are open!
3
18
1
217
26,912
Christos Perivolaropoulos retweeted
I love the game of Boggle. This demo showcases our Gemini 2.0 Pro model’s coding abilities in AI Studio. It is mind boggling to think that it can write the full piece of code, including all the right data structures and search algorithms to find all valid words on a Boggle board from a relatively simple prompt asking it to do so. As a computer scientist, I'm also happy that it got its data structures right on the first trie. Discombobulating!
Replying to @GoogleDeepMind
2.0 Pro Experimental is now available to: 🔘 Developers in Google AI Studio and @GoogleCloud’s #VertexAI 🔘 @GeminiApp Advanced users in the model drop-down on desktop and mobile. goo.gle/40OsRfj
76
73
12
626
122,462
Christos Perivolaropoulos retweeted
Making LLMs run efficiently can feel scary, but scaling isn’t magic, it’s math! We wanted to demystify the “systems view” of LLMs and wrote a little textbook called “How To Scale Your Model” which we’re releasing today. 1/n
25
386
77
1,913
468,677
Kudos to everyone who worked on this. It looks amazing!
Introducing Veo 2, our new, state-of-the-art video model (with better understanding of real-world physics & movement, up to 4K resolution). You can join the waitlist on VideoFX. Our new and improved Imagen 3 model also achieves SOTA results, and is coming today to 100+ countries in ImageFX.
1
259
Christos Perivolaropoulos retweeted
Thank you to the @scifordl Workshop at #NeurIPS2024 for featuring our paper as one of the best paper runner-ups in its 'Debunking challenge'!!! 🚀🧑‍🔬 Also, sincere thanks to @ccperivol and @fedzbar for their tireless work presenting our little softmax study throughout the day! 🙌
4
33
3,153
Christos Perivolaropoulos retweeted
As I've been asked a few times recently: I won't be going to NeurIPS (travelled way too much! 😅) But if you're going, I warmly invite you to stop by @fedzbar's poster this Thursday! Federico worked on this as part of his Student Researcher placement with us @GoogleDeepMind
Heading tomorrow to Vancouver for NeurIPS! Please do reach out if you want to chat about reasoning in Transformers / LLMs :) I'll be presenting our work "Transformers need glasses! 👓" on Thursday at 4:30pm at East Exhibit Hall A-C #1806.
6
49
6,584
Christos Perivolaropoulos retweeted
Heading tomorrow to Vancouver for NeurIPS! Please do reach out if you want to chat about reasoning in Transformers / LLMs :) I'll be presenting our work "Transformers need glasses! 👓" on Thursday at 4:30pm at East Exhibit Hall A-C #1806.
3
24
1
168
21,420
Christos Perivolaropoulos retweeted
Our team is hiring Student Researchers @GoogleDeepMind for '25! 🧑‍🔬 Interested in understanding reasoning capabilities from first principles? 🧑‍🎓 Currently studying for a BS/MS/PhD? 🧑‍💻 Have solid engineering and research skills? 🌟 We want to hear from you! Details in thread.
14
84
2
879
110,191
Christos Perivolaropoulos retweeted
This is the poster I'm most happy about in my career, even though the actual amount of writing effort was minimal 🙃 Coming soon to NeurIPS Workshops near you (two spotlights)! 🔦 I sadly won't be there myself, but @ccperivol & @fedzbar will be happy to tell you all about it 🚀
6
70
1
466
40,702
Serious question to torch users who are to some degree aware of JAX and keras3: why do you prefer torch?
2
3
333
Christos Perivolaropoulos retweeted
Take time to study carefully how data flows in a Transformer, and you'll near-certainly find something useful that others missed 🧑‍🔬 Also, don't hold back on sharing a result just 'cause it feels obvious. 'softmax is not enough' was needlessly delayed for 10 months due to this 😶
There is so much low hanging fruit in thinking carefully about what’s happening in a transformer / LLM. Did it really take two years for someone to point out that RoPE will (obviously?) not decay activation with relative distance for random key value pairs?
3
25
246
21,877
Christos Perivolaropoulos retweeted
CGP has now been accepted to @LogConference 😊🎉 Heartfelt congratulations to @sonjj74 on getting his first lead-author paper out of the door -- you've come such a long way from last year, in spite of having no formal AI training! I am impressed. 🙇‍♂️ Onwards! 🚀
Expander Graph Propagation has levelled up! ⏫ Introducing Cayley Graph Propagation 🕸️ While EGP truncates Cayley graphs (reintroducing bottlenecks!), CGP retains all leftover nodes as virtual nodes, preserving all benefits of expander graphs! 🍾 Great work by @sonjj74 & Maya.
8
1
63
8,561
Christos Perivolaropoulos retweeted
Releasing my detailed commented introduction to LLM sampling colab.research.google.com/dr… We get back to the basics and slowly build up to a reproduction of the adaptive temperature strategy from "Softmax is not enough" (from @PetarV_93 et al.)
8
77
3
515
34,079
Christos Perivolaropoulos retweeted
I love a good leaderboard... or several! ⏫ TGR 🐅 is our graph rewiring method for temporal graphs leveraging expander graph propagation. Turns out, TGR is _real good_ 🔥 setting SOTA on _four_ diverse tasks in the TGB dataset. Read on for more 🧵 @kpetrovvic @shenyangHuang
1
5
1
41
8,588