Tim Tomov (@timtomov) header
Tim Tomov (@timtomov) profile picture

PhD Student in Machine Learning @ DAML (TUM), how to know when models don’t know ? Doing uncertainty quantification stuff

Munich, German
Joined November 2015
Photo by Tim Tomov (@timtomov)Photo by Tim Tomov (@timtomov)Photo by Tim Tomov (@timtomov)Photo by Tim Tomov (@timtomov)Photo by Tim Tomov (@timtomov)Photo by Tim Tomov (@timtomov)Photo by Tim Tomov (@timtomov)Photo by Tim Tomov (@timtomov)Photo by Tim Tomov (@timtomov)Photo by Tim Tomov (@timtomov)
Are LLM outputs just language? We often don’t know what the answer to a question will be - but we do know its structure. Numbers, lists, sets, ... Our framework models outputs in their underlying structure for better answers + uncertainty. w/ @dfuchsgruber @guennemann [1/4]🧵
1
4
2
12
7,497
Tim Tomov retweeted
Discrete flow matching usually changes one component per edit in a fixed-size state. Can we extend it to general actions, like inserting whole molecular fragments? Introducing Trajectory Matching 🧵 New paper with M. Lienen, @ludke_david, @guennemann
3
24
1
101
5,300
Tim Tomov retweeted
Blog Post Track is happening again at ICLR this year 🥳 Looking forward to your submissions! (Please no slop) See: iclr-blogposts.github.io/202… @iclr_conf
2
9
600
Can a question be ambiguous for one LLM, but not another? Our paper received an Outstanding Paper Award at LP4FM @ NeurIPS 2026 🏆 I’ll present it as an oral in Sydney 🇦🇺 ! w/. J. Tang, S. Schmidt @dfuchsgruber @guennemann Why ambiguity should depend on the model [1/4]🧵
3
1
6
292
→ For a particular model, we should quantify model ambiguity rather than world ambiguity. Our goal: estimate the set of correct answers a model knows We formalize clarifications and show a one-to-one correspondence between correct answers and clarification classes [3/4]🧵
1
21
🇰🇷 If you’re at ICML: come check out our Meta FairChem + TUM poster exploring an idea: Speculative decoding, but for simulation trajectories! 🗓️ Tue, Jul 7, 2026 • 2:00 PM – 3:45 PM KST 📍 HALL A #1312 w/ @guennemann, Ray (Meng) Gao, @bwood_m. 🙏 to the entire FAIRChem team!
2
12
1
49
4,130
I’ll be at ICML next week in Seoul 🇰🇷, presenting two recent works: (1)Task-Awareness Improves LLM Generations and Uncertainty (2)Task-Aware Calibration: Provably Optimal Decoding in LLMs @dfuchsgruber @raajeevVerma @guennemann @MunichCenterML What, When, Where 👇🏽
2
2
22
952
(1) 🔗 arxiv.org/abs/2601.21500 Free-form language often has underlying semantic structure. Instead of modeling LLM outputs in text space, we model them in their task structure. This improves both decoding and uncertainty estimation. 📍 Thu Jul 9, 10:00–11:45 AM Hall A #1607
1
2
135
(2) 🔗 arxiv.org/abs/2605.10202 Task-awareness makes calibration possible for free-form generations. Use this to get a provably optimal decoding strategy. 📍 Fri Jul 10 — Spotlight Talk & Poster @ EIML Workshop 📍 Fri Jul 11 — Poster @ Agentic UQ Workshop
83
This is an amazing read. Like AI 2027, it's a great way to spend some time thinking about the future and feels very plausible That said, it's already overtaken by recent events thanks to the USG pulling Mythos and Fable already. This should help Europe wake up faster hopefully
"What will happen to Europe if it keeps ignoring AI?" Three American labs each (!!) operate more AI compute than all of Europe combined. Today we're launching Europe 2031: a story of what might happen if that doesn't change.
7
4
62
6,150
Information Bottleneck Theory is a powerful framework introduced by Naftali Tishby for understanding learning and representation in intelligent systems. The central idea is that a good representation should compress the input while preserving information relevant to the prediction task. Formally, the objective balances two quantities: maximizing information about the target Y while minimizing unnecessary information about the input X. A typical objective is: I(T;Y) − βI(T;X) where T is the learned representation and I(·;·) denotes mutual information. In statistics and machine learning, the information bottleneck provides a principled approach to feature extraction, dimensionality reduction, and prediction. In deep learning, it offers a theoretical perspective on why hidden layers learn compressed representations that retain task-relevant information while discarding noise. It has influenced research on generalization, representation learning, self-supervised learning, and neural network interpretability. In reinforcement learning, information bottleneck methods are used to learn compact state representations, improve exploration, and reduce sample complexity. The broader insight is that intelligence is not merely about storing information—it is about retaining the right information. By balancing compression and prediction, the information bottleneck provides a unifying principle connecting information theory, learning, and decision-making. Image: share.google/icpdTVCg7DegzS3…
3
44
3
266
19,813
1/4 Fixing hallucinations means fixing evaluations, as shown in our new paper rdcu.be/fjJFP building on our earlier @OpenAI blog. Accuracy-based scoring rewards models for making their best guess even when unsure, so hallucinations are like students guessing on tests.
3
9
1
44
6,675
What if we could make decoding provably optimal? Building on our recent work, we introduce Task-Aware Calibration, a framework for obtaining optimal LLM decisions for a given task. The idea: Calibrate first, then act optimally w/ @dfuchsgruber @raajeevVerma @guennemann
1
4
7
678
So we get better decisions, but what actually changes? It can, e.g., eliminate biases. In one LLM-as-a-judge task, the model often shows "correlation bias" (rating helpfulness and correctness identically). Task calibration recovers the true, weaker correlation structure.
1
47
So what? While generally applicable, we think this could be especially useful for LLM-as-a-judge: better decisions ➡️ more reliable evaluations. On a personal note: It was a really fun project, and I learned a lot!
31