
Applied math PhD student @Harvard. Neural networks and reasoning.
Joined March 2022
- Tweets26
- Following56
- Followers89
- Likes14
Pinned Tweet
Is your gating mistuned?
In our new work, we highlight the critical role of gating for SSMs. Correct gating boosts areas of traditional weakness: in-context learning, retrieval, and long-context performance.
arxiv.org/abs/2609.16540
[1/6]
Default initialization for gating is not optimal for every setting. Many classic weaknesses in SSMs may be addressed through the correct choice of gating.
For many more details, please see our preprint!
arxiv.org/abs/2609.16540
[5/6]
1
30
Many thanks to my extraordinary mentor at Apple @EranMalach, my exceptionally supportive advisor at Harvard @CPehlevan, and our spectacular collaborators @aryolotfi , Emmanuel Abbe, Kostas Vaggelakos, Vishnu Banna, @EtaiLittwin, and @jmsusskind!
[6/6]
2
63
Just how useful are reasoning traces?
Together with Ege Cakar and @CPehlevan, we're thrilled to announce a new preprint!
arxiv.org/abs/2602.14404
[1/7]
Reasoning models do not always dominate their "non-reasoning" counterparts, and much depends on task topology.
For more frumentaceous findings and a deeper theoretical analysis, please see our preprint:
arxiv.org/abs/2602.14404
[6/7]
1
1
89
Many thanks to my extremely talented mentee Ege Cakar, exceptionally supportive advisor @CPehlevan, members of the Pehlevan Group @Harvard and the @KempnerInst! Special thanks for the generous support from @patrickshafto and the @DARPA AIQ program鉂わ笍
[7/7]
2
121
Replying to @CPehlevan
@CPehlevan and I are thrilled to announce our next paper! We explore equality reasoning in neural networks, and the impact of learning richness.
arxiv.org/abs/2503.09781
馃У[1/5]
Our conclusions also hold for visual reasoning tasks. In this example, the model must distinguish whether the pentominoes are the same shape or different. Richer models generalize faster from fewer training shapes.
馃У[4/5]
1
2
98
Many thanks to @CPehlevan, members of the Pehlevan Group at @Harvard, and the @KempnerInst!
If you're at CCN 2025 (@CogCompNeuro) next week, please swing by my poster on Tuesday and say hello!
馃У[5/5]
1
90
William Tong retweeted
Check out my latest work, studying a fundamental trade-off in machine learning: should you train a single large model, or an ensemble of smaller models? A huge thanks to my advisor @CPehlevan and collaborators @wl_tong and @hamzatchaudhry.
arxiv.org/abs/2412.05418
In this Same-Different task, the model receives two, one-hot encoded input tokens, and must output whether they are the same or different.
With a large enough training vocabulary, MLPs generalize to *unseen* tokens.
馃У[4/n]
1
3
505
Many thanks to @CPehlevan, members of the Pehlevan Group at @Harvard, and the @KempnerInst!
Please feel free to reach out with questions or comments!
馃У[5/n]
2
401
MLPs learn in-context.
For example, MLPs (and the closely related MLP-Mixer) attain competitive loss with Transformers on an in-context classification task, comparing by compute.
馃У[1/n]




