What if the teacher is wrong?
Standard on-policy distillation learns from it anyway.
RG-OPD doesn't.
We gate distillation using verifier reward + the teacher–student likelihood gap, keeping the same reverse-KL objective.
📈 +2.9 pts vs OPD @ 1K
📈 +8.2 pts vs base @ 8K
🎉 COLM 2026 Workshop on Efficient Reasoning
Paper: openreview.net/pdf?id=kF2lIu…
With @vijaylingam08 @MS_Akhondzadeh @atu_tej @sujaysanghavi @thecekbote
1
6
415
Aleksandar Bojchevski retweeted
Jailbreaking research assumes a fixed model.
Test-time training breaks that assumption: the attacker gets a lever on the weights, not just the prompt.
We show this opens a new attack surface that undermines LLM safety guardrails.
🧵 (1/6)
Aleksandar Bojchevski retweeted
1/5 Ever wondered how to apply conformal prediction when there's epistemic uncertainty? Our new paper addresses this question!CP can benefit from models like Bayesian, evidential, and credal predictors to have better prediction sets,for instance, in terms of conditional coverage.
Optimal Conformal Prediction under Epistemic Uncertainty. arxiv.org/abs/2505.19033
Aleksandar Bojchevski retweeted
🚨 Robust conformal prediction is expensive as we need around 10000 forward passes per input. Or Is it?
Checkout our ICLR2025 paper: openreview.net/forum?id=ltrx…
We extend conformal sets to worst case noise under any smoothing, with much less samples.
Joint work with @abojchevski
Aleksandar Bojchevski retweeted
Want to learn continuous & discrete Flow Matching? We've just released:
📙 A guide covering Flow Matching basics & advanced methods arxiv.org/abs/2412.06264.
💻 An open source codebase with image & text examples github.com/facebookresearch/….
🗣️ A Flow Matching tutorial #NeurIPS2024.
Aleksandar Bojchevski retweeted
Missed out on #Swift tickets? No worries—swing by our #SVFT poster at #NeurIPS2024 and catch *real* headliners! 🎤💃🕺
📌Where: East Exhibit Hall A-C #2207, Poster Session 4 East
⏲️When: Thu 12 Dec, 4:30 PM - 7:30 PM PST
#AI #MachineLearning #PEFT #NeurIPS24
🚀 Exciting new paper alert! Achieve up to 96% full performance with just 0.006-0.25% of trainable parameters!✨ How? It’s all in the singular vectors! Introducing 🎯SVFT: Singular Vectors guided Fine-Tuning for PEFT. Here’s a quick breakdown!🧵 #AI #MachineLearning #NLP #CV
Aleksandar Bojchevski retweeted
A common question nowadays: Which is better, diffusion or flow matching? 🤔
Our answer: They’re two sides of the same coin. We wrote a blog post to show how diffusion models and Gaussian flow matching are equivalent. That’s great: It means you can use them interchangeably.
Aleksandar Bojchevski retweeted
Replying to @guennemann
@guennemann kicking of the LOG meetup in Aachen.
Aleksandar Bojchevski retweeted
1/ Can Large Language Models (LLMs) truly reason? Or are they just sophisticated pattern matchers? In our latest preprint, we explore this key question through a large-scale study of both open-source like Llama, Phi, Gemma, and Mistral and leading closed models, including the recent OpenAI GPT-4o and o1-series.
arxiv.org/pdf/2410.05229
Work done with @i_mirzadeh, @KeivanAlizadeh2, Hooman Shahrokhi, Samy Bengio, @OncelTuzel.
#LLM #Reasoning #Mathematics #AGI #Research #Apple
Move over Taylor there's another SVFT in town.
🎉 Thrilled to share that SVFT is officially accepted at #NeurIPS24! 🙌
See you all in Vancouver!
w/ incredible co-authors @vijaylingam08 @VavreAditya @aneeshk1412 @gauthamkrishna_ Joydeep Ghosh @AlexGDimakis @eunsolc @abojchevski @sujaysanghavi
Aleksandar Bojchevski retweeted
I am happy to announce that @abojchevski, Michael Schaub, Martin Grohe, and I are organizing a @LogConference local meet-up in Aachen (27.11.-28.11.), including keynotes from @guennemann and @BurkholzRebekka. See log-rwth.github.io/log-meetu… for more information.
Aleksandar Bojchevski retweeted
✨🎨🏰Super excited to share our new paper Ensemble everything everywhere: Multi-scale aggregation for adversarial robustness
Inspired by biology we 1) get adversarial robustness + interpretability for free, 2) turn classifiers into generators & 3) design attacks on vLLMs 1/12
Aleksandar Bojchevski retweeted
#ICML2024 With conformal prediction (CP) we return sets that include the true class with guaranteed high probability,
With robust CP we maintain this probability even for worst case (adversarial) noisy input. With CAS (our method) we have robust CP with efficient (small) sets.
Aleksandar Bojchevski retweeted
🎉Happy to announce that our paper – SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors has been accepted for an oral presentation at WANT @icmlconf!✨#ICML2024 #MachineLearning
Paper: openreview.net/forum?id=DOUs…
🚀 Exciting new paper alert! Achieve up to 96% full performance with just 0.006-0.25% of trainable parameters!✨ How? It’s all in the singular vectors! Introducing 🎯SVFT: Singular Vectors guided Fine-Tuning for PEFT. Here’s a quick breakdown!🧵 #AI #MachineLearning #NLP #CV
Aleksandar Bojchevski retweeted
🤔 Ever wondered how vulnerable Graph Neural Networks (GNNs) really are?
🚀 Check out our slightly late, but crucial, #ICLR2024 paper: "Rethinking Label Poisoning for GNNs: Pitfalls and Attacks"! 🔥
Read all about our highly parameter-efficient fine tuning approach that Pareto-dominates LoRA and other baselines.
🚀 Exciting new paper alert! Achieve up to 96% full performance with just 0.006-0.25% of trainable parameters!✨ How? It’s all in the singular vectors! Introducing 🎯SVFT: Singular Vectors guided Fine-Tuning for PEFT. Here’s a quick breakdown!🧵 #AI #MachineLearning #NLP #CV
Aleksandar Bojchevski retweeted
This paper made me smile a lot while working on it, so I want to share a bit about it arxiv.org/abs/2405.09676.
We draw a parallel story to the Eckart-Young Theorem (from numerical analysis) in stochastic optimization/learning problems.
(with Josh Cutler and Dima Drusvyatskiy)























