New preprint! 馃鉃曗灄 Shared circuits predict whether LLMs generalize across formats in arithmetic reasoning w/ @sanapandey, @pengrui_han, @jacobandreas, and @ev_fedorenko If you can solve 2+5, you can solve "two plus five" without learning math again. LLMs are more brittle (1/5)
Image from Andrea de Varda (@devarda_a)
1
22
2
160
7,489
LLMs solve numeric arithmetic (44 + 22 =) almost perfectly, but do much worse on the same problems in words: 馃嚞馃嚙 forty-four plus twenty-two equals 馃嚜馃嚫 cuarenta y cuatro m谩s veintid贸s es igual a 馃嚠馃嚬 quarantaquattro pi霉 ventidue fa (2/5)
Image from Andrea de Varda (@devarda_a)
1
7
745
Can we capture this variability in generalization from the model's internals? We localize the circuit each model uses for numeric arithmetic, then measure how much the verbal formats engage the same units (see fig in first tweet). (3/5)
1
8
486
Circuit overlap predicts generalization at three levels: which formats are hardest (English > Spanish > Italian), which models generalize best, and which individual items get solved (fig below). If the numeric circuit is not engaged, the answer is almost always wrong. (4/5)
Image from Andrea de Varda (@devarda_a)

Sep 7, 2026 路 11:19 PM UTC

1
6
476
The circuit metric (unsupervised) predicts generalization as well as or better than supervised probes. Behavioral generalization is grounded in circuit generalization: if the circuit does not transfer the ability does not either. Pre-print 馃敆 arxiv.org/abs/2609.04463 (5/5)
Image from Andrea de Varda (@devarda_a)
1
7
412