
Research @AnthropicAI | Opinions my own
London, England
Joined June 2021
- Tweets63
- Following1.6K
- Followers241
- Likes1K
Pinned Tweet
We present a way to extract reward functions from Diffusion Models, and show it generalizes to Stable Diffusion!
Our algorithm compares an expert diffusion model to a base model, and extracts the reward function for the expert’s task.
🧵Check it out at tinyurl.com/rw-diff!
Felipe Nuti retweeted
Is Muon as good as they say? We looked beyond training speed and found a hidden cost: Muon loses the simplicity bias of older optimizers like gradient descent — and this matters for generalization.
Many works on LLM jailbreaking have hypothesized that certain jailbreaks get around safety guardrails introduced during fine-tuning by sending prompts out of the distribution of the fine-tuning data, leading the LLM to behave more like its unaligned pre-trained checkpoint.
But how can we directly measure—for an individual prompt—when fine-tuning is having less of an effect on the model’s final output?
At ICML 2025 on Thu. 11AM (#E-3005), we show how to do this for any fine-tuned model, assuming access to the original pre-trained model 🧵🪡
Joint work with @frtimlive and João Henriques at @Oxford_VGG .
However, if TuCo is to be a good measurement of the effect of fine-tuning on model outputs, we would expect that it should be lower when attacks succeed at circumventing safety guardrails and eliciting a harmful response.
That begs the question: is this actually the case?
We find empirical evidence that it is: TuCo separates successful attacks from benign prompts and unsuccessful attacks with an AUC > 0.8 on 4/6 models we considered (including Llama 2 13B).
Although it’s not designed for this purpose, this indicates TuCo can show some out-of-the-box performance in jailbreak detection, before the model outputs even a single token.
1
1
94
In summary, by decomposing the model’s final hidden state into its pre‑training and fine‑tuning components, TuCo quantitatively measures fine‑tuning’s per‑prompt effect, and our results show that jailbreak prompts concretely diminish this effect on the model’s responses.
Come check out our poster on Thursday at 11 AM at the East Exhibition Hall A-B #E-3005!
Arxiv: arxiv.org/abs/2506.23423
Code: github.com/FelipeNuti/tuning…
1
77
Felipe Nuti retweeted
In addition to scaling data, we found that fine-tuning SVD on its original architecture is sub-optimal. If your fine-tuned model generates videos with cluttered background/appearance, check it out!
Puppet-Master
Scaling Interactive Video Generation as a Motion Prior for Part-Level Dynamics
discuss: huggingface.co/papers/2408.0…
We present Puppet-Master, an interactive video generative model that can serve as a motion prior for part-level dynamics. At test time, given a single image and a sparse set of motion trajectories (i.e., drags), Puppet-Master can synthesize a video depicting realistic part-level motion faithful to the given drag interactions. This is achieved by fine-tuning a large-scale pre-trained video diffusion model, for which we propose a new conditioning architecture to inject the dragging control effectively. More importantly, we introduce the all-to-first attention mechanism, a drop-in replacement for the widely adopted spatial attention modules, which significantly improves generation quality by addressing the appearance and background issues in existing models. Unlike other motion-conditioned video generators that are trained on in-the-wild videos and mostly move an entire object, Puppet-Master is learned from Objaverse-Animation-HQ, a new dataset of curated part-level motion clips. We propose a strategy to automatically filter out sub-optimal animations and augment the synthetic renderings with meaningful motion trajectories. Puppet-Master generalizes well to real images across various categories and outperforms existing methods in a zero-shot manner on a real-world benchmark.
Felipe Nuti retweeted
📢 Introducing Illusory Attacks: Information-theoretically undetectable adversarial attacks on RL agents and humans!
Spotlight (top 5%) @ ICLR’24 – see you in Vienna!
🔎Can you spot which observation is under attack?🔎
🧵
Felipe Nuti retweeted
♨️ Announcing: GPU-EVM by @gatling_x ♨️
The most performant parallel-EVM in the world by 100x.
Massive unlock: foundational infra for training AI/RL models that interact with the EVM, accelerated L2s, MEV, backtesting, and more
🧵+ waitlist below
Felipe Nuti retweeted
Announcing DragAPart, a model that empowers image diffusion to generate nuanced part-level dynamics. Demo is on 🤗!
Joint work with my amazing advisors @ChuanxiaZ, @chrirupp, and A. Vedaldi @Oxford_VGG.
📰 Paper: arxiv.org/abs/2403.15382
🌐 Page: dragapart.github.io
🧵👇
Felipe Nuti retweeted
A misconception, also sometimes polled by @elonmusk (for humor, I believe), is a false dichotomy between Transformers and Diffusion. Transformer is a neural network architecture. Diffusion is a particular way of modeling the data distribution. These are two separate things. You can train Diffusion models by using Transformers (Transformer U Nets). The actual dichotomy is between Autoregressive (AR) models vs Diffusion models. And the first is great for more abstract tokens: language, VQ or discrete quantized tokens at a patch/chunk level, etc. The latter is great for raw signals and continuous latent space embeddings, especially much higher dimensional. Both are iterative, but in different axes: AR iterates across sequence length by generating token by token, and diffusion tries multiple versions of the output with improvement at every step, but generates the sequence all at once.
Felipe Nuti retweeted
there’s now an international olympiad for AI research (ioai-official.org/) in the spirit of IMO / IOI etc — would be super cool to field a UK high school team!
anyone keen to help make this happen?
Come talk to us our work on extracting reward functions from diffusion models at 10:45, poster #600, at #NeurIPS!
8
264
Felipe Nuti retweeted
Excited to share that I will be at NeurIPS this week, presenting our work "PlanE: Representation Learning over Planar Graphs" with @ZZYSonny, @Ralph_Abb, @ismaililkanc.
Come see us on Thursday at 18:00!
Project page: radoslav11.com/plane/
Popular GNNs are known to be limited in their expressiveness, meaning that they cannot distinguish between certain pairs of graphs. However, that doesn't have to be the case for some special graph classes.
Motivated by the classical literature on graph isomorphism, we propose PlanE, a framework for learning graph invariants over planar graphs. We show that a simple instance from the framework can efficiently learn complete graph invariants!
#NeurIPS #NeurIPS2023













