Associate Prof at TU Vienna, Adjunct PI at AITHYRA, Associate Member @UniofOxford | Research on Graphs, Geometry, AI4Science
Vienna
Joined October 2023
- Tweets224
- Following263
- Followers696
- Likes734
İsmail İlkan Ceylan retweeted
Stellar lineup at the AITHYRA Next generation AI for Biomedical Science workshop -- next Monday 2 Nov at the IMBA lecture hall in Vienna Biocenter. No registration is required.
İsmail İlkan Ceylan retweeted
Here's a direct link to OpenAI's PDF claiming a refutation of ETH. In particular, it's claimed that 3SAT on n variables is in about exp(n/(log n)^{1/2+o(1)}) time
github.com/openai/math/blob/…
İsmail İlkan Ceylan retweeted
An internal model at OpenAI has produced a claimed refutation of ETH! Russell Impagliazzo, Mohan Paturi, and I have been asked to help organize an open and independent examination of this claim. 1/4
İsmail İlkan Ceylan retweeted
It was inevitable: Large Flow Language Models.
We scale flow LMs to 8B with blockwise flow training, improving pretrained AR via an MTP effect. We use this for self-speculative decoding that verifies flow drafts autoregressively, which is quite fast at batch size one regime.
Can flow language models scale to billions of parameters and handle real generative tasks?
Introducing 🌊 Lafa (Large flow language model), a flow LM scaled up to 8B that matches AR quality, with 🚀 2.5× AR throughput at 7B.
Co-led with @ArnavChavan6 @UdbhavBamba.
Joint work with @wognsfjq96 @hongseu33 @AdtRaghunathan @jw9730
İsmail İlkan Ceylan retweeted
Avengers assemble: 1 year BakerLab + NVIDIA GenAIR = 🔥 rfd4-proteina.org
Presenting RFD4-Proteina, our latest protein design model, significantly better and faster than everything we compared against, both in silico and in the lab! 1/30
İsmail İlkan Ceylan retweeted
Can flow language models scale to billions of parameters and handle real generative tasks?
Introducing 🌊 Lafa (Large flow language model), a flow LM scaled up to 8B that matches AR quality, with 🚀 2.5× AR throughput at 7B.
Co-led with @ArnavChavan6 @UdbhavBamba.
Joint work with @wognsfjq96 @hongseu33 @AdtRaghunathan @jw9730
İsmail İlkan Ceylan retweeted
Can diffusion models generate high-quality images without CFG?
Yes! We introduce ✨Correction Training for Diffusion Models✨, which significantly improves sample quality without relying on classifier-free guidance.
Project page: sunovivid.github.io/correcti…
🧵 (1/n)
İsmail İlkan Ceylan retweeted
You can match training examples to the closest noise samples in each batch to improve unconditional flow models (minibatch-OT) but in conditional models, this is impractical.
Interesting find: simply ignore conditioning for the matching, and performance with CFG still improves!
İsmail İlkan Ceylan retweeted
Our new preprint LLM-to-Flow Distillation (LFD) is finally live! 🧬
I’m convinced language is the right interface for controlling generative models. And that should apply to molecules just as much as images!
LLMs know a lot about science. DPO gives us a way to transfer that knowledge into scientific generators. With LFD, “design molecules that bind to the D2 dopamine receptor” becomes a goal we can actually train a flow model to pursue.
The LLM builds an adaptive curriculum around what the generator can currently produce. As the model improves, the teaching and guidance from the LLM change too.
The cool part is that we can do this with models we already have, without redesigning their architectures or engineering a task-specific proxy reward.
Really happy to share this work with Riccardo, Federico and the rest of the team!! Full thread below 👇
We’re bringing natural-language control to any scientific generator.
We are very excited to introduce LLM-to-Flow Distillation (LFD), a method to adapt scientific generators to high-level natural-language discovery goals.
Starting from textual goals like “design peptides with promising anticancer activity” or “design small molecules binding to the D2 dopamine receptor”, LFD distills high-level goal-related knowledge from LLMs into scientific generators through iterative DPO on LLM-generated sub-goals.
🔥 LFD does not need task-specific proxy rewards, works with any frontier LLM via API-access, and outperforms state-of-the-art reward-guided methods such as FlowGRPO and DiffusionNFT on held-out computational evaluations. (1/6)
İsmail İlkan Ceylan retweeted
New foundation model for general relational structures and tasks using random walks and sequence models!!
One model, wildly different graphs and tasks, and some very intriguing generalization capabilities.
Excited about the future of this! Fantastic work by @louistichelman with great contributions from the usual suspects @hxyscott and @jw9730!
İsmail İlkan Ceylan retweeted
Frontier graph foundation models for various tasks and graph types!
İsmail İlkan Ceylan retweeted
"LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators"
Scientific flow models can generate valid molecules and peptides, but struggle to follow high-level natural-language goals.
On top of that, current methods also rely on explicit reward functions, which are unreliable in scientific discovery.
LLM-to-Flow Distillation (LFD) instead uses an LLM to create progressively harder sub-goals based on what the generator can currently produce, then uses its preferences to fine-tune the generator through DPO.
On peptide and molecular design tasks, it outperforms reward-guided baselines on held-out computational evaluations, even reaching difficult out-of-distribution goals where those methods fail.
alphaxiv.org/abs/2610.17246
İsmail İlkan Ceylan retweeted
🚀 Science is the next big challenge for agentic AI systems.
While LLMs are becoming incredibly capable, it is still an open question how we can translate their intelligence into scientific discoveries, where closing the loop is costly and reliable reward functions are often unavailable.
We introduce LLM-to-Flow Distillation (LFD): a method that combines the scientific knowledge of frontier LLMs with the structural priors of pretrained generators, allowing us to steer these latter toward natural-language discovery goals.
Check out our work in the thread 👇🏼
We’re bringing natural-language control to any scientific generator.
We are very excited to introduce LLM-to-Flow Distillation (LFD), a method to adapt scientific generators to high-level natural-language discovery goals.
Starting from textual goals like “design peptides with promising anticancer activity” or “design small molecules binding to the D2 dopamine receptor”, LFD distills high-level goal-related knowledge from LLMs into scientific generators through iterative DPO on LLM-generated sub-goals.
🔥 LFD does not need task-specific proxy rewards, works with any frontier LLM via API-access, and outperforms state-of-the-art reward-guided methods such as FlowGRPO and DiffusionNFT on held-out computational evaluations. (1/6)
İsmail İlkan Ceylan retweeted
We’re bringing natural-language control to any scientific generator.
We are very excited to introduce LLM-to-Flow Distillation (LFD), a method to adapt scientific generators to high-level natural-language discovery goals.
Starting from textual goals like “design peptides with promising anticancer activity” or “design small molecules binding to the D2 dopamine receptor”, LFD distills high-level goal-related knowledge from LLMs into scientific generators through iterative DPO on LLM-generated sub-goals.
🔥 LFD does not need task-specific proxy rewards, works with any frontier LLM via API-access, and outperforms state-of-the-art reward-guided methods such as FlowGRPO and DiffusionNFT on held-out computational evaluations. (1/6)
İsmail İlkan Ceylan retweeted
Continuous Diffusion LMs went from niche to contender this year, but the recipe is shaky: why self-conditioning? How to embed tokens? 🤨
Our new work takes your one-hot flow from 4% to 50% on TinyGSM without self-conditioning, and explores the design space of CDLMs. 🔥🚀 (1/6)
