New preprint! 馃鉃曗灄
Shared circuits predict whether LLMs generalize across formats in arithmetic reasoning
w/ @sanapandey, @pengrui_han, @jacobandreas, and @ev_fedorenko
If you can solve 2+5, you can solve "two plus five" without learning math again. LLMs are more brittle (1/5)
Sep 7, 2026 路 11:19 PM UTC
LLMs solve numeric arithmetic (44 + 22 =) almost perfectly, but do much worse on the same problems in words:
馃嚞馃嚙 forty-four plus twenty-two equals
馃嚜馃嚫 cuarenta y cuatro m谩s veintid贸s es igual a
馃嚠馃嚬 quarantaquattro pi霉 ventidue fa
(2/5)
Can we capture this variability in generalization from the model's internals? We localize the circuit each model uses for numeric arithmetic, then measure how much the verbal formats engage the same units (see fig in first tweet). (3/5)
Circuit overlap predicts generalization at three levels: which formats are hardest (English > Spanish > Italian), which models generalize best, and which individual items get solved (fig below). If the numeric circuit is not engaged, the answer is almost always wrong. (4/5)
The circuit metric (unsupervised) predicts generalization as well as or better than supervised probes.
Behavioral generalization is grounded in circuit generalization: if the circuit does not transfer the ability does not either.
Pre-print 馃敆 arxiv.org/abs/2609.04463 (5/5)
