on natural language processing / machine learning. assistant prof at @NYUDataScience @NYU_Courant prev @UTCompSci @googleai, @uwcse, @Cornell.
Joined September 2016
- Tweets146
- Following988
- Followers6.5K
- Likes5.3K
Pinned Tweet
My lab will move to @NYUDataScience and @NYU_Courant this Fall! I’m excited to connect with amazing researchers at @CILVRatNYU and larger ML/NLP community in NYC. I will be recruiting students this cycle at NYU. Happy to be back to the city 🗽on the east coast as well.
I had a wonderful four years at UT, and will always be very grateful for the friendship and mentorship at UTCS and the larger UT community.
CDS welcomes Eunsol Choi (@eunsolc) as an Assistant Professor of Computer Science (@NYU_Courant) and Data Science!
Her research focuses on advancing how computers interpret human language in real-world contexts.
nyudatascience.medium.com/me…
Eunsol Choi retweeted
🚨 New Paper! 🚨
Your agent is writing code faster than you can understand it 😬
In our controlled online study, coding agents helped users finish tasks quickly, but reduced how well they understood the code they just “wrote”
We also find three ways to avoid this tradeoff 👇🧵
Eunsol Choi retweeted
Does what an LLM's CoT says actually match what it computes internally?🤔 Often, no. But additional post-training can help reduce this misalignment.
We introduce CoT-Interpretability Alignment (CIA)🕵️: detect the model's internal reasoning strategy with interpretability tools, and check if its CoT tells the same story.
We then use CIA as a reward for post-training, which improves alignment while maintaining or improving task accuracy.
📢 New paper: "Making LLMs Say What They Think: Measuring and Improving CoT-Interpretability Alignment"
🔗 arxiv.org/abs/2609.38972
We study sampling diverse output from a suite of LLMs. One key surprise for me was that it's better to carefully pick a single model to sample many times, rather than naively mixing outputs from multiple models.
Eunsol Choi retweeted
Excited to share MyScholarQA - a personalized deep research tool that learns from your papers and lets you customize reports! 🧑🔬🖌️
Our #ACL2026 paper built and evaluated it, showing simulated users (LLMs) couldn't mimic what real users wanted 🙅
Spicy results + a live demo 👇🧵
Do LLMs suffer from human-like cognitive biases? 🤔 Check out @arhjhaveri's new paper on how models navigate hypothesis spaces. We found that confirmation bias degrades LLM performance, and we explore strategies to mitigate it.
Your AI Agent just formed a hypothesis. 💭 How does it validate it?
Not by trying to prove itself wrong. Rather, it selectively seeks evidence that confirms what it already believes, often ending up with the wrong answer!
Confirmation bias isn’t just human. We measure it in LLMs, and we show how to fix it! 🧵
Eunsol Choi retweeted
A lot of useful training data can't be shared due to privacy. How do we create synthetic training data without data owners ever sharing their content?
🚀 Introducing 𝐃𝐏-𝐑𝐅𝐓: using RL to train LLMs to generate high-fidelity domain data without seeing a single private sample.
Excited to introduce our new work on improving information coverage for search systems (LLMs + retriever).
Key idea: train retrievers to search for answers that are missing from earlier retrieval rounds.
🚨NEW PAPER🚨
How can we comprehensively retrieve all relevant docs for multi-answer QA? Agentic search doesn't help.
Introducing RVR, an iterative framework that conditions on prior docs to maximize answer coverage.
📈10% answer recall gain on QAMPARI
w/@hungting_chen @eunsolc
Eunsol Choi retweeted
COLM is going to San Francisco for 2026!
🗓️Dates: October 6-9, 2026
🏨Venue: Hilton San Francisco Union Square
Website and CFPs for papers and workshops coming up soon!
Reasoning models are more uncertain when generating reasoning traces for easy questions compared to generating traces for medium-difficulty questions.
We use this insight to build a light-weight, adaptive inference strategy for reasoning models. Was fun exploring this with my visiting student Xiang. :)
We looked into user interaction logs, searching for learning signal to improve LLMs. We find user data noisy and nontrivial to leverage, yet we learned a lot about user behaviors! See our #EMNLP25 paper.
👀Have you asked LLM to provide a more detailed answer after inspecting its initial output? Users often provide such implicit feedback during interaction.
✨We study implicit user feedback found in LMSYS and WildChat. #EMNLP2025
Eunsol Choi retweeted
At #ACL2025 this week! Please come check out our poster 11:00a.m. to 12:30p.m. today at Hall 4/5.
Can we generate long text from compressed KV cache? We find existing KV cache compression methods (e.g., SnapKV) degrade rapidly in this setting. We present 𝐑𝐞𝐟𝐫𝐞𝐬𝐡𝐊𝐕, an inference method which ♻️ refreshes the smaller KV cache, which better preserves performance.
ALT Figure 1 from our paper which demonstrates our approach and baselines on the left, and an illustration of a proposed "chain-of-key" task and performances of different methods on the right.
Knowledge propagation in LLM is notoriously challenging. Check out our paper that improves it substantially by training a hypernetwork to target knowledge propagation!
Eunsol Choi retweeted
I will be presenting this work at NAACL 2025! More specifically at 5pm on Thursday (May 1st), at Ruidoso. (Session IIS 1)
Looking forward to catching up with old friends and meeting new people! Let’s chat!
🚨New Paper🚨: We introduce BERDS, a BEnchmark for Retrieval Diversity, for subjective questions.
We collect subjective questions with diverse perspectives and develop evaluation metrics to measure retrieval diversity in an open-world setting.
Work done w/ @eunsolc!
🧵
Eunsol Choi retweeted
Interested in how alignment changes the response distribution defined by LLMs? Come check out my poster at 2 PM at #NAACL2025
nitter.cf/thomlake/status/180601…
Does aligning LLMs make responses less diverse? It’s complicated:
1. Aligned LLMs produce less diverse outputs
2. BUT those outputs are comprehensive, aggregating the useful info from base models
3. ICL can “mimic” fine-tuned models with high fidelity
w/ @eunsolc & @gregd_nlp
Please check out Michael's #ICLR2025 poster on training LLMs to ask clarifying questions. LLMs are eager to answer immediately, even when the input is ambiguous. We simulate future turns and then assign rewards based on it, teaching LLMs to see a value in asking clarifying questions!
Can your LLM ask clarifying questions for ambiguous prompts?
Come see our #ICLR25 poster this afternoon where I’ll chat about why RLHF'd LMs fail to ask clarifying questions to resolve ambiguity, and why they often confidently respond to only one interpretation instead.
More ⬇️
Would LLMs think "Houston, Austin, Dallas" is sampled from "cities in Texas" rather than "cities in the US"? I really enjoyed our work exploring reasoning of LLMs about these suspicious coincidences!
Are LMs sensitive to suspicious coincidences? Our paper finds that, when given access to knowledge of the hypothesis space, LMs can show sensitivity to such coincidences, displaying parallels with human inductive reasoning. w/@kanishkamisra, @kmahowald, @eunsolc
Eunsol Choi retweeted
We are receiving repeating questions about the double submission policy in relation to the abstract deadline. Our FAQ addresses this, and spells out what will be included in any double submission checks we do with other venues. Let us know if there are more Qs
Can we generate speech that aligns with abstract, rich style tags (e.g., confused, authoritative)? Anuj's new work makes a step towards it through careful data augmentation!
Introducing ParaSpeechCaps, our large-scale style captions dataset that enables rich, expressive control for text-to-speech models!
Beyond basic pitch or speed controls, our models can generate speech that sounds "guttural", "scared", "whispered" and more; 59 style tags in total.
ALT Randomly sampled examples from ParaSpeechCaps. Our style prompts cover rich tags describing complex styles like rhythm, clarity, emotion, etc. in contrast to erstwhile basic style prompts that only contain gender, pitch and speed levels. We highlight rich style tags with vibrant colors and basic style tags with a gray color.
Eunsol Choi retweeted
Excited to announce our 2025 keynote speakers: @cosmo_shirley, Nicholas Carlini, @LukeZettlemoyer, and Tom Griffiths!
When using LLM-as-a-judge, practitioners often use greedy decoding to get the most likely judgment. But we found that deriving a score from the judgment distribution (like taking the mean) consistently outperforms greedy decoding. Check out @victorwang37's thorough study!
❌LLM-as-a-judge with greedy decoding
😎Using the distribution of the judge’s labels
