Ramakanth Kavuluru retweeted
Interested in putting interpretability on more rigorous foundations? Check out our #EMNLP2026 tutorial slides!
I learned a ton from my fantastic collaborators @rpatrik96, @_shruti_joshi_, @dhanya_sridhar. Recommended for newcomers and experts alike!
📚 tinyurl.com/causality4mechin…
Ramakanth Kavuluru retweeted
We're releasing one of the largest & most detailed look at how Americans use ChatGPT complementing reports from OpenAI, giving a much more deeper view into ChatGPT use
Explore our interactive tools & analysis: stateoftheprompt.net
Let us know if you want to see any analysis!
STATE OF THE PROMPT: Our Rutgers CSS Lab team analyzed a year of real ChatGPT conversations from 70K+ US panelists. See the new interactive tools, plus research reports on health, politics & more.
stateoftheprompt.net/
With @gvrkiran @shagunjhaver @RutgersCommInfo @RutgersU
Ramakanth Kavuluru retweeted
Can an EHR model trained at one hospital read another hospital's records?
Excited to introduce our new paper, "Discrete Neural Interlingua for Ontology-Agnostic EHR Modeling," accepted at NeurIPS 2026. See you in Sydney 🦘🏖️!
Paper: arxiv.org/abs/2610.27047
Code: github.com/mahmoodlab/MEDIAT…
Model: huggingface.co/MahmoodLab/ME…
Video: See👇
The same clinical concept can be an ICD code, a SNOMED term, a local identifier, or free text, and standard EHR tokenizers treat each one as an unrelated symbol. A model trained on MIMIC-III with a closed vocabulary can use only 13% of eICU events. It isn't only a cross-hospital problem. For example, inside one hospital, "acute kidney injury" can be a billing code, a flowsheet identifier, and free text in a note, and the vocabulary shifts again with every coding transition and EHR migration. The redundancy is there before a model ever leaves the building.
MEDIATOR is a discrete, ontology-agnostic tokenizer. Any clinical concept written as text maps to one of 100,000 shared tokens, so "acute kidney failure" (ICD-9), "AKI" (a local code), and "acute kidney injury" (free text) become the same token. It is trained on cross-ontology synonyms from UMLS and guided by LLM-enriched descriptions, code co-occurrence, and a medical relation graph. It is a drop-in token interface for standard EHR Transformers: no manual code mapping, and new code systems can be tokenized without retraining.
Main results:
• In-domain: +4.0 AUPRC points (8% relative) over SOTA, across 20 MIMIC-III tasks. Merging within-dataset redundancy pays off before you transfer anything.
• Zero-shot transfer: on eICU with no vocabulary alignment, MEDIATOR uses all 3.58 million events and improves AUPRC over MedTok by 8.6 points (29% relative) across 14 tasks, up to +21.2 on subarachnoid hemorrhage. Restricted to the same inputs as the closed-vocabulary baselines, it is still the best tokenizer.
• Transferable attribution: one shared token space means attributions compare directly across hospitals. For cirrhosis mortality, token importance is rank-correlated between MIMIC-III and eICU at Spearman ρ = 0.92.
Tokenizing a dataset's entire vocabulary is a one-time step of about 70 seconds on one GPU. Code and weights are public, and the co-occurrence teacher can be rebuilt from public data with near-identical results.
Congradulations to Caiwei Tian Tong Ding and everyone else.
#NeurIPS2026 #ClinicalInformatics #EHR #MachineLearning
Ramakanth Kavuluru retweeted
My Student Researcher project at @Google is out! 🔥
We introduce On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models, showing that self-generated privileged information can effectively drive LLM self-improvement.
📄 Paper: arxiv.org/abs/2610.32770
Ramakanth Kavuluru retweeted
🐭MICE to Meet You: From Re-ranking Only to End-to-End Retrieval ⚡️
Our #EMNLP2026 paper showed that MICE pushes the Cross-Encoder Pareto frontier.
We take it a step further: turning theoretical pre-computation gains into an operational, end-to-end retrieval architecture 🧵
Ramakanth Kavuluru retweeted
LLMs can solve tasks one at a time, yet fail when the same tasks are combined into one prompt. Why?
In our paper, we find that LLMs operate under an implicit thinking budget, with the reasoning effort applied to each task shrinking according to a power law.
Ramakanth Kavuluru retweeted
If you've noticed a distinct dialect emerging in frontier AI models and struggled to understand what they're talking about, you're not crazy!
We turned to psycholinguistics to investigate what makes the writing style so weird, and why this might be happening
🚨New preprint! 🧵
Ramakanth Kavuluru retweeted
Introducing Carbon-X: The best open DNA model.
We midtrained a 14B model on DNA and text, and it beats every open language model and matches GPT-6.1 Sol at 1,000x lower cost.
Ask it about a sequence, what a mutation does, or to generate new DNA.
huggingface.co/HuggingFaceBi…
Ramakanth Kavuluru retweeted
Today we're releasing Carbon-X, a 14B model that brings DNA and natural language together for the first time 🧬
1. Beats every open language model we tested across DNA benchmarks. Within a point of GPT-6.1 Sol at >1,000× lower inference cost.
2. Mid-trained on 300B tokens of interleaved DNA and text, followed by SFT (without distillation!) and RL.
3. A new hybrid tokenizer encodes DNA using ~3× fewer tokens than BPE.
4. Gives other models a DNA specialist subagent: Kimi K3 gains 11.1 points when it consults Carbon-X.
5. Runs locally on a laptop with llama.cpp. Apache 2.0. Your DNA stays on your hardware.
Model: huggingface.co/HuggingFaceBi…
Blog: huggingface.co/spaces/Huggin…
Introducing Carbon-X: The best open DNA model.
We midtrained a 14B model on DNA and text, and it beats every open language model and matches GPT-6.1 Sol at 1,000x lower cost.
Ask it about a sequence, what a mutation does, or to generate new DNA.
huggingface.co/HuggingFaceBi…
Ramakanth Kavuluru retweeted
💡 Training on auxiliary views of Document X > training on Document X itself, even for factual questions drawn directly from X.
Presenting our work on LLM knowledge acquisition at #EMNLP2026 this weekend!
(1) Under a fixed token budget, conceptual reformulations (textbook explanations, blogs, and Q&A) can teach a document’s facts more effectively than repeating the document, even paraphrased.
We call these auxiliary views: beyond mere paraphrasing, they discuss the knowledge in a completely different context and form e.g. a paper blog!
The surprising part is that auxiliary views rarely mention the facts directly, but our evaluation uses the original document's phrasing near-verbatim.
(2) The benefit emerges and grows with model scale.
(3) stronger generator ≠ better learning.
We synthetically generate auxiliary views for our experiments, but across GPT-5, GPT-OSS, Gemma4, GLM-5 and more, the learner model's knowledge acquisition shows no correlation to generator model size or its own accuracy on the evaluation.
(4) After training on auxiliary views, the model's parameters actually move less but concentrate its changes at the middle and final layers.
The takeaway for synthetic data augmentation: build diversity around the knowledge you want to teach, instead of treating diversity as some board corpus-level heuristic.
Also, a possible implication for LLM self-improvement. Models can learn by just generating and training on their own auxiliary views, without a stronger teacher. This is particularly relevant for deeply specialized, long-tailed knowledge.
📄 Paper: arxiv.org/pdf/2609.04180
Ramakanth Kavuluru retweeted
🚨 New Paper! 🚨
Multiple-choice benchmarks use accuracy (boring), but education has way cooler scoring schemes! 🤩
👎 Lose points if wrong
🔁 Try until you're right
🃏 Wager points
At #EMNLP2026, we show these schemes shift LLM ranks, align with LLM Arena, + unmask LLM quirks
Ramakanth Kavuluru retweeted
Super excited to share SnapJudge -- a free, open detector trained for scientific and math-heavy writing, and can handle multiple modalities, including screenshots of paper text, LaTex, and text extracted from PDFs... 🧵
Ramakanth Kavuluru retweeted
Can research agents get better at science by learning from the research they do?
Introducing DiscoveryRSI 🧬: a framework that turns hypothesis-driven research into a recursive self-improvement loop.
📄 Paper: arxiv.org/abs/2610.23839
💻 Code: github.com/Gen-Verse/Discove…
🔴 Live Arena: foundry-arena.rsi-arena.work…
🔹 What is a valuable hypothesis?
We define it as a joint object H = (claim, investigation plan, decision criteria, alternative explanations). A hypothesis is valuable if resolving it would change a research decision. It is testable if the plan can produce evidence that supports, weakens, or bounds the claim.
🔹 How it works
- Lead, Generation, Evolution, Critique and Progress agents grow hypothesis trees through evolve / repair / crossover / restart
- Every version stays linked to its evidence, data, assessments and contribution records
- Selected plans compile into executable Harbor tasks
🔹 Three self-improvement mechanisms
① Hypothesis credit optimization: blinded TrueSkill assessments assign bounded credit to the agent steps that actually contributed, and DiscoveryRSI-27B is trained only on those steps (Qwen3.8-27B init, 256K context; 2,397 of 15,792 trajectories kept; 2.12% of tokens supervised)
② Harness replay: diagnose recurring failures such as repeated exploration, failed tool use, lost artifacts and premature closure, then refine the workflow
③ Rediscovery: run the accepted model and harness updates on fresh questions
🔹 Results (DFM-Bench, 4 life-science domains)
- Hypothesis quality 4.86 → 5.18 → 5.39 over two rounds
- Falsifiability 4.00 → 6.20, alternative explanations 4.44 → 5.98, data resolvability 3.73 → 4.99
- Evolve/repair share 77.5% → 91.7%, restarts 21.7% → 5.0%: the agent learns to keep developing an idea instead of starting over
- Caveat: importance and novelty dip slightly. The model learns to design better tests, not yet to choose bolder questions.
🔹 Expert review
27 physicians and bioscience researchers scored 292 anonymized proposals:
- 41% of DiscoveryRSI-27B revisions improve quality vs 31% (GPT-5.6 Sol) and 25% (GPT-6 Luna)
- Only 12% make things worse vs 32% and 31%
🔹 Does it anticipate real science?
Against 16,440 publications filtered by each model's cutoff, 6.3% of DiscoveryRSI-27B's hypotheses connect to later published research (16.9% in therapeutic targets). Comparison: DeepSeek 5.4%, GLM 2.8%, GPT 2.5%, Claude 2.3%.
Watch six models do research in real time in DiscoveryRSI Arena, and follow every proposal, critique and compile step 👇
foundry-arena.rsi-arena.work…
Co-led by Jianyuan Zhong, Chenwei Wu & Zhaochen Yu, with wonderful collaborators across PhAI Labs, CUHK, Stanford, MIT, Harvard Medical School, Oxford, Princeton, PKU and more.
Grateful to @james_y_zou @Jiaru_Zou @Charles_Y_Wu @zhenfei_yin_yzf 🙏
Ramakanth Kavuluru retweeted
1/ RL for improving task success has made LLMs remarkably capable. But what happens when the model fails? Turns out, training for graceful failure can be surprisingly tricky!
Very excited to share our new paper: Training Language Models to Fail Gracefully
🧵
Ramakanth Kavuluru retweeted
Researchers tend to have their best, most innovative ideas early in their careers. But for decades, researchers have had to wait longer and longer to test their own ideas.
Thrilled by the progress we’ve made so far on this front, including the huge milestone of setting a new record for grants supporting early-stage investigators.
ALT A graphic highlighting "A Record Year!" with 1648 early-stage investigators receiving NIH support through R01-equivalent extramural research grant awards. The background shows a blurred image of people working in a laboratory, with the U.S. Department of Health and Human Services and NIH logos at the bottom.
Ramakanth Kavuluru retweeted
🧵[1/N] Are large language models more creative than humans at combining ideas? 🤔
Combinatorial creativity involves combining familiar ideas in unfamiliar ways, and is central to scientific discovery and technological innovation.
In fact, breakthrough scientific work is often associated with more distant combinations of methods, techniques, and concepts.
Here, we introduce Kombine, a new benchmark of combinatorial creativity for LLMs.
TLDR:
- LLMs are more combinatorially creative than the average human.
- However, conceptual blending is the hardest task for LLMs, and even frontier models struggle.
- LLMs also tend to homogenize, inventing concepts with the same properties across separate models.
- Moreover, humans and LLMs combine in distinct regions of semantic space.
- As a result, we anticipate greater conceptual diversity can be unlocked through human-AI collaboration. 🗣️
Thread below🧵⬇️
Ramakanth Kavuluru retweeted
WildChat, LMSYS, ShareChat… each human-LLM chat dataset aims to capture "real-world" AI use. But do they paint the same picture?
Not quite. From user messages alone, a classifier can name a chat's source dataset, and user models trained on them inherit these signatures 🧵👇
Ramakanth Kavuluru retweeted
What if,
for any scientific claim,
you can search 100M+ papers to find the strongest evidence for and against it
⚡️in 1 second?
That’s now possible with Evidence API! 🚀
paperclip.gxl.ai/evidence-ap…
For example, does drinking coffee reduce diabetes risk?
Ramakanth Kavuluru retweeted
We’re open-sourcing pplx-embed-v2-late, multi-vector embeddings for text and images, 9B and 0.6B, in one shared embedding space. You can use these to index multimodal data with 9B, and query on device with 0.6B. This also enables you to search over PDF pages with no OCR. And scores 92.4% on MADQA, 64% on BrowseComp+. Weights available on @huggingface now.
We're releasing pplx-embed-v2-late, two late-interaction embedding models that retrieve text, images, and pages with a shared embedding space for cross-model querying.
Both models achieve frontier performance and are publicly available on Hugging Face.
perplexity.ai/hub/blog/multi…
