Sagi

Three papers from our lab at #NeurIPS2026, spanning LLM reasoning faithfulness, belief formation, and interpretability of MoEs: ⭐ Faithfulness Metrics Don't Measure Faithfulness: A Meta-Evaluation with Ground Truth arxiv.org/abs/2605.25052 We introduce a benchmark for CoT faithfulness metrics, and find that existing metrics are either close to random or prohibitively expensive to run @GurYoav @anmarasovic ⭐ Indications of Belief-Guided Agency and Meta-Cognitive Monitoring in Large Language Models arxiv.org/abs/2602.02467 An interdisciplinary collaboration applying the HOT-3 indicator of consciousness to LLMs using interpretability tools, revealing surprising patterns in belief formation and action selection in LLMs @noam_steinmetz @GoldsteinYAriel @Liad_Mudrik ⭐ Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts arxiv.org/abs/2605.12476 We find a geometric coupling between routers and their corresponding experts in MoEs, explaining how routers form assignments that support an effective division of labor @sageaarc @noyahochwald

Here we go! ✈️ 🇺🇸🇰🇷 If you're attending #ACL2026 and/or #ICML2026 check out recent work from our group with collaborators: 📍Jul 5 14:00-15:30: Matan will present his TACL work "Universal Jailbreak Suffixes Are Strong Attention Hijackers" @matanbt @mahmoods01 arxiv.org/abs/2506.12880 📍Jul 5 16:00-17:30: My student Or will show how matrix factorization can reveal compositional neuron groups and hierarchies in MLPs. @OrShafran arxiv.org/abs/2506.10920 📍Jul 6 9:00-10:30: Tomer will talk about "MoNaCo: More Natural and Complex Questions for Reasoning Across Dozens of Documents" @TomerWolfson arxiv.org/abs/2508.11133 📍Jul 7 14:00-15:45: Hadas will present our position: "Interpretability Can Be Actionable" @OrgadHadas arxiv.org/abs/2605.11161 (will also appear at the MI workshop) 📍Jul 8 10:30-12:15: I will be presenting a position work with @_galyo and @ymatias on metacognition as a way to move beyond hallucinations arxiv.org/abs/2605.01428 📍Jul 9 17:00-18:45: Or (after a transpacific flight from ACL) will present MFA: an unsupervised approach to disentangle representations of LLMs at scale using local geometry @OrShafran arxiv.org/abs/2602.02464 (will also appear at the MI workshop)

What's in a neuron? 💫 (an atypically long, almost personal post) Neurons in LMs have always been a fascinating object to study. I've been studying them since 2020, viewing them as key-value memory cells, analyzing what they capture in vocabulary space, and how they compose together to form features. arxiv.org/abs/2012.14913 arxiv.org/abs/2203.14680 arxiv.org/abs/2501.08319 arxiv.org/abs/2506.10920 But many neurons still remain opaque! They do many things. Our recent work led by @AsafAvrahamy tackles this challenge by decomposing neuron weights in vocabulary space. We do this by taking the neuron weight vector and learning different ways to rotate it (just a bit) to reveal monosemantic vocabulary channels that it captures. The nice thing about our method ROTATE is that it's data-free and super efficient, relying only on vocabulary kurtosis as a search signal. I've been thinking about this idea since 2024, proposed it to multiple students, but only Asaf was brave enough to take this ;) Very happy with the final outcome. Check out the paper! 👇 arxiv.org/abs/2604.06005