Tim Tomov retweeted
Discrete flow matching usually changes one component per edit in a fixed-size state. Can we extend it to general actions, like inserting whole molecular fragments?
Introducing Trajectory Matching 🧵
New paper with M. Lienen, @ludke_david, @guennemann
Tim Tomov retweeted
Blog Post Track is happening again at ICLR this year 🥳
Looking forward to your submissions!
(Please no slop)
See: iclr-blogposts.github.io/202…
@iclr_conf
Tim Tomov retweeted
🇰🇷 If you’re at ICML: come check out our Meta FairChem + TUM poster exploring an idea:
Speculative decoding, but for simulation trajectories!
🗓️ Tue, Jul 7, 2026 • 2:00 PM – 3:45 PM KST
📍 HALL A #1312
w/ @guennemann, Ray (Meng) Gao, @bwood_m. 🙏 to the entire FAIRChem team!
Tim Tomov retweeted
This is an amazing read. Like AI 2027, it's a great way to spend some time thinking about the future and feels very plausible
That said, it's already overtaken by recent events thanks to the USG pulling Mythos and Fable already. This should help Europe wake up faster hopefully
Tim Tomov retweeted
Information Bottleneck Theory is a powerful framework introduced by Naftali Tishby for understanding learning and representation in intelligent systems. The central idea is that a good representation should compress the input while preserving information relevant to the prediction task. Formally, the objective balances two quantities: maximizing information about the target Y while minimizing unnecessary information about the input X.
A typical objective is:
I(T;Y) − βI(T;X)
where T is the learned representation and I(·;·) denotes mutual information.
In statistics and machine learning, the information bottleneck provides a principled approach to feature extraction, dimensionality reduction, and prediction. In deep learning, it offers a theoretical perspective on why hidden layers learn compressed representations that retain task-relevant information while discarding noise. It has influenced research on generalization, representation learning, self-supervised learning, and neural network interpretability.
In reinforcement learning, information bottleneck methods are used to learn compact state representations, improve exploration, and reduce sample complexity. The broader insight is that intelligence is not merely about storing information—it is about retaining the right information. By balancing compression and prediction, the information bottleneck provides a unifying principle connecting information theory, learning, and decision-making.
Image: share.google/icpdTVCg7DegzS3…
Tim Tomov retweeted
1/4 Fixing hallucinations means fixing evaluations, as shown in our new paper rdcu.be/fjJFP building on our earlier @OpenAI blog. Accuracy-based scoring rewards models for making their best guess even when unsure, so hallucinations are like students guessing on tests.
Tim Tomov retweeted
The bitter lesson in 26 words:
Don’t be distracted by human knowledge, as AI has been historically.
Instead focus on methods for creating knowledge that scale with computation, like search and learning.
Tim Tomov retweeted
# A new type of information theory
this paper is not super well-known but has changed my opinion of how deep learning works more than almost anything else
it says that we should measure the amount of information available in some representation based on how *extractable* it is, given finite computation. for example, an encrypted text file has less V-information than the same data in plaintext, because it takes more computation to extract. note the contrast to traditional information theory, which would tell us that the two representations have the same amount of Shannon information
i’ve long wondered why certain types of basic questions didn’t have a proper theoretical answer:
> why does distillation outperform vanilla maximum likelihood training?
> why does lora work better than finetuning?
> why does self-attention work better than almost any other similar operation?
> how much “information” remains in a text embedding?
> how much “information” remains in language model weights?
> should i use fine-tuning or RAG?
the true answers to all these questions depend on some way of measuring and comparing *information content* between different representations. v-information is one step towards doing this
besides computational constraints, model architecture probably affects the “information content” in representations, along with the presence of any pretraining data used, as well as model-level statistics about the optimization in the training process – e.g. the length of time a model was trained for probably changes representations pretty drastically
this is all to say, i think there is some true notion of “information” that none of our current paradigms (Shannon information, V-information, etc.) capture. we encounter this idea every day but we dance around it and describe it in vague terms; we measure it from all sorts of angles but can’t quite characterize it theoretically
when the v-information paper came out I thought there would be a lot of follow up work developing more complex and useful notions of information for deep learning. but it hasn’t
yet I still think at the heart of these questions of what-information-lies-in-representations there’s something to be found that’s profound, elegant, and potentially extremely useful. I don’t know what it is and i’m probably not the person who will figure it out. but I really hope someone does. :)
Tim Tomov retweeted
One theorem every ML engineer should know:
The Johnson–Lindenstrauss Lemma.
It states that high-dimensional data can be projected into a much lower-dimensional space while approximately preserving pairwise distances.
Why it matters:
• Explains why random projections work
• Enables scalable learning in high dimensions
• Used in embeddings, compressed learning, and ANN search
• Helps fight the curse of dimensionality
The surprising part:
You can reduce dimensions dramatically without destroying the geometry of the data.
That’s why many ML systems can operate efficiently even with massive feature spaces.
Modern representation learning is deeply connected to this idea:
Good embeddings preserve structure while compressing information.
In ML, compression is often not loss of intelligence —
it’s removal of redundancy.
Tim Tomov retweeted
New preprint:
"Transferable SCF-Acceleration through Solver-Aligned Initialization Learning (SAIL)"
* Standard ML initial guesses target the converged density.
* We target the solver trajectory instead.
Tim Tomov retweeted
Very happy to present our work on a principled robustness evaluation of graph transformers at #ICLR2026! You can talk to us at tomorrow's poster session from 10:30 am to 1 pm in Pavilion 3 - #816. Joint work with Philipp Foth, @geisler_si, @SchwinnLeo, and @guennemann.
Tim Tomov retweeted
A reminder that you cannot learn from data without making assumptions. You should never apologize for making assumptions. Embrace them, be transparent about what they are, and enabled by those assumptions, update based on new information.
Tim Tomov retweeted
I want to take this post to thank the amazing people of X for reviewing and reading our paper: arxiv.org/pdf/2602.18292v1 - where we claim that decoding is an optimisation layer on top of LLMs.
Thank you for letting us know which references we missed. We are adding those and will update the arXiv soon. So far, the list is:
@amritsinghbedi3 Controlled decoding from language models arxiv.org/pdf/2310.17022
@amritsinghbedi3 Transfer Q⋆: Principled Decoding for LM Alignment arxiv.org/pdf/2405.20495
@elon_lit Scaled-Dot-Product Attention as
One-Sided Entropic Optimal Transport arxiv.org/abs/2508.08369
@timtomov Task-Awareness Improves LLM Generations and Uncertainty arxiv.org/pdf/2601.21500
Let me know if I missed any!
Please keep them coming! Would be happy to cite the missing refs!
Also make sure to read those papers above, I think they are super awesome :D
#AI #MachineLearning
Tim Tomov retweeted
Interesting research from Anthropic:
When you have increasingly large models and increasingly complex tasks it's more likely that the models will give you different answers if you run the same query multiple times. On easy tasks, larger models actually become more coherent.
Think of a "cone" of possible trajectories and the branching factor gets bigger with more possibilities (due the larger models "knowing more options to explore" and more complex problems having more "possible aspects"). The amount of time reasoning (trajectory length) then makes it multiplicatively more incoherent at the end state. Having a large model with an easy task means the correct answer is definitely "in there" and it's less likely to become distracted.
They are arguing this is relevant for AI safety because some might have assumed that larger models would have convergent "instrumental goals" and would give a consistently wrong rather than randomly wrong answer.
Apparently the "the hot mess theory of intelligence" (Sohl-Dickstein, 2023) argues that "as entities become more intelligent, their behaviour tends to become more incoherent, and less well described through a single goal."
New Anthropic Fellows research: How does misalignment scale with model intelligence and task complexity?
When advanced AI fails, will it do so by pursuing the wrong goals? Or will it fail unpredictably and incoherently—like a "hot mess?"
Read more: alignment.anthropic.com/2026…
Tim Tomov retweeted
Wondering how to attend an ML conference the right way?
ahead of NeurIPS 2025 (30k attendees!) here are ten pro tips:
1. Your main goals:
(i) meet people
(ii) regain excitement about work
(iii) learn things
– in that order.
2. Make a list of papers you like and seek them out at poster sessions. Try to talk to the authors– you can learn much more from them than from a PDF.
3. Pick one workshop and one tutorial that sounds most interesting. Skip the rest.
4. Cold email people you want to meet but haven't. Check Twitter and the accepted papers list. PhD students are especially responsive.
5. Practice a concise pitch of unpublished research you're working on for "what are you interested in rn?". Focus on big unanswered questions and exciting new directions, *not* papers.
6. Skip the orals. Posters are a higher-bandwidth, more engaging, more invigorating. Orals are a good time to go for a walk or talk in the hallway.
7. for the love of god, do NOT work on other research in your hotel room. Save mental bandwidth for the conference. (This may seem obvious; you'd be surprised.)
8. Talk to people outside your area. There are many smart people working on niches <10 people understand. Learn about one or two that won't help your own work.
9. Attend one social each night. Don't overthink it or get caught up in status games. They're all fun.
10. Take breaks. You can't go to everything, and conferences consume more energy than a normal workweek.
hope this helps, and sad i'm not attending neurips, have fun :)
Tim Tomov retweeted
Excited to share that my first PhD paper, which introduces a new diffusion model for relational databases, has been accepted to #NeurIPS2025! We will be presenting it this week in San Diego. ☀️🌴
Joint work with @ludke_david, @SchwinnLeo, and @guennemann.
🧵 1/
Tim Tomov retweeted
Heading to San Diego for #NeurIPS2025! 🌴☀️
I’ll be presenting 3 recent papers covering generative models for hierarchies, spatiotemporal tissue dynamics, and long-range graph learning.
If you're around, drop by and say hi! 👋
Here’s the schedule 🧵👇
Tim Tomov retweeted
No Free Lunch Theorem: en.wikipedia.org/wiki/No_fre…
Tim Tomov retweeted
Our paper “Modeling Microenvironment Trajectories on Spatial Transcriptomics with NicheFlow” has been accepted at #NeurIPS2025!
@ale__palmaa @guerrantif @fabian_theis @guennemann
(@TU_Muenchen × @HelmholtzMunich × @MunichCenterML × @MdsiTum )
🧵
Tim Tomov retweeted
Open-source Diffusion LLMs easily break GPT-5!
In “Diffusion LLMs are Natural Adversaries for any LLM,” we show that Inpainting on Diffusion LLMs yields efficient, transferable jailbreaks without any model access.
@TomWollschlager, Paul Ungermann, @guennemann, @leoschwinn
🧵




































