Arnab Sen Sharma (@arnab_api) profile picture

Ph.D. student @KhouryCollege, working to make LLMs interpretable

Boston, MA
Joined September 2022
Photo by Arnab Sen Sharma (@arnab_api)Photo by Arnab Sen Sharma (@arnab_api)Photo by Arnab Sen Sharma (@arnab_api)Photo by Arnab Sen Sharma (@arnab_api)Photo by Arnab Sen Sharma (@arnab_api)Photo by Arnab Sen Sharma (@arnab_api)Photo by Arnab Sen Sharma (@arnab_api)Photo by Arnab Sen Sharma (@arnab_api)Photo by Arnab Sen Sharma (@arnab_api)Photo by Arnab Sen Sharma (@arnab_api)
Do language models have goals? ๐ŸŽฏ When LMs carry out tasks that unfold over thousands of tokens, is it driven by an ๐—ฒ๐˜…๐—ฝ๐—น๐—ถ๐—ฐ๐—ถ๐˜ ๐—ด๐—ผ๐—ฎ๐—น ๐—ฒ๐—ป๐—ฐ๐—ผ๐—ฑ๐—ถ๐—ป๐—ด in its internals? Can we readout and manipulate this encoding? We investigate these questions in our new paper. ๐Ÿงต
4
16
4
71
2,304
Non-recurrent progression of a multi-step task is also true in VLM setting! Checkout parallel work led by @rohitgandikota nitter.cf/rohitgandikota/status/โ€ฆ
Transformers *plan* before they generate a token ๐Ÿ‘€ We found the plan inside a VLM: where to look, what is done, what comes next. See the plan live: plan.baulab.info/#demo (Bonus: code LLMs and image diffusion transformers do it too) Here is how we investigated it ๐Ÿงต๐Ÿ‘‡
1
5
134
Arnab Sen Sharma retweeted
New paper! One way to find jailbreaks is to train a model via RL to attack a target LLM. We introduce Curriculum of Models: train against a sequence of targets, progressing gradually from easy to hard to jailbreak and ending with the original target model.
Automated jailbreak discovery helps developers find vulnerabilities in their models before deployment. We introduce Curriculum of Models (CoM): a new way to train an โ€œinvestigatorโ€ model to surface jailbreaks even when refusals make the reward signal sparse. ๐Ÿงต
4
11
2
111
4,658
Arnab Sen Sharma retweeted
New COLM paper: Identifying Introspection From the Inside When an LLM tells us about its decisions, does it ๐˜ฌ๐˜ฏ๐˜ฐ๐˜ธ what drives its choicesโ€”or is it guessing? In our setting, we find that faithful models decide and report with the same layers. Unfaithful ones don't. ๐Ÿงต
9
71
9
320
23,192
A builtin sub-circuit (attn heads) in VLMs that encodes image segments to language concepts! And, the model uses these encodings for downstream behavior. Checkout this cool work from @sheridan_feucht et al.
[New preprint] How do images in VLMs align with words? In this work, we found a set of attention heads responsible for OCR. But to our surprise, these heads were actually able to verbalize much more than just text. So, we used them to create a simple logit lens for image tokens!
5
302
Arnab Sen Sharma retweeted
Excited to kickstart this effort with @glnmario and a star-studded advisory team! ๐Ÿš€ Also, we're hiring for 2 MTS and 1 Ops manager! Come build the future of white-box audits with us: parallx.ai/careers
Introducing Parallax ๐Ÿ” We're a new UK/EU-based nonprofit research lab building scalable methods and infrastructure to audit the beliefs, goals, and plans that shape model behaviours in long-horizon agentic evaluations. parallx.ai โ€” Thread ๐Ÿงต 1/
9
17
79
4,861
Arnab Sen Sharma retweeted
๐Ÿง ๐Ÿค– The 2026 New England Mechanistic Interpretability (NEMI) Workshop will be Aug. 14 at Boston University! Help spread the word and join the New England mech interp community! Registration and submission info in thread:๐Ÿ‘‡
2
30
10
122
26,730