William Tong (@wl_tong) profile picture

Applied math PhD student @Harvard. Neural networks and reasoning.

Joined March 2022
Photo by William Tong (@wl_tong)Photo by William Tong (@wl_tong)Photo by William Tong (@wl_tong)Photo by William Tong (@wl_tong)Photo by William Tong (@wl_tong)Photo by William Tong (@wl_tong)Photo by William Tong (@wl_tong)Photo by William Tong (@wl_tong)Photo by William Tong (@wl_tong)
Is your gating mistuned? In our new work, we highlight the critical role of gating for SSMs. Correct gating boosts areas of traditional weakness: in-context learning, retrieval, and long-context performance. arxiv.org/abs/2609.16540 [1/6]
1
3
1
8
1,563
Many thanks to my extraordinary mentor at Apple @EranMalach, my exceptionally supportive advisor at Harvard @CPehlevan, and our spectacular collaborators @aryolotfi , Emmanuel Abbe, Kostas Vaggelakos, Vishnu Banna, @EtaiLittwin, and @jmsusskind! [6/6]
2
63
Reasoning models do not always dominate their "non-reasoning" counterparts, and much depends on task topology. For more frumentaceous findings and a deeper theoretical analysis, please see our preprint: arxiv.org/abs/2602.14404 [6/7]
1
1
89
Many thanks to my extremely talented mentee Ege Cakar, exceptionally supportive advisor @CPehlevan, members of the Pehlevan Group @Harvard and the @KempnerInst! Special thanks for the generous support from @patrickshafto and the @DARPA AIQ program鉂わ笍 [7/7]
2
121
Our conclusions also hold for visual reasoning tasks. In this example, the model must distinguish whether the pentominoes are the same shape or different. Richer models generalize faster from fewer training shapes. 馃У[4/5]
1
2
98
Many thanks to @CPehlevan, members of the Pehlevan Group at @Harvard, and the @KempnerInst! If you're at CCN 2025 (@CogCompNeuro) next week, please swing by my poster on Tuesday and say hello! 馃У[5/5]
1
90
In this Same-Different task, the model receives two, one-hot encoded input tokens, and must output whether they are the same or different. With a large enough training vocabulary, MLPs generalize to *unseen* tokens. 馃У[4/n]
1
3
505
Many thanks to @CPehlevan, members of the Pehlevan Group at @Harvard, and the @KempnerInst! Please feel free to reach out with questions or comments! 馃У[5/n]
2
401
MLPs learn in-context. For example, MLPs (and the closely related MLP-Mixer) attain competitive loss with Transformers on an in-context classification task, comparing by compute. 馃У[1/n]
4
17
70
11,746
We also find that MLPs excel at relational reasoning. In this example Oddball task, an MLP generalizes substantially better than Transformers to out-of-distribution distances. 馃У[3/n]
1
3
471