Marjan Ghazvininejad retweeted
Labs disagree on vision encoders for MLLMs! ⚔️
Gemma 4 12B scrapped the encoder.
Kimi K3 dropped encoder pretraining.
Inkling 2 Small revived the encoder Inkling killed.
We trained 200+ models in a controlled study to settle it:
✅ Keep vision parameters, skip the pretraining. And they don't have to be an encoder! 🤔
Paper: arxiv.org/abs/2611.17334 🧵
Marjan Ghazvininejad retweeted
how should multimodal models encode visual input?
→ pretrained encoder?
→ encoder-free?
→ before the LLM?
published models bundle these choices together. we trained 200+ models from scratch to see which of these choices matters
only one of them does
🧵 [1/n]
Marjan Ghazvininejad retweeted
Human annotations are getting more important as models scale, but they're expensive and still lack fine-grained signals important for learning‼️
We introduce ⭐️IsoVRBench⭐️, video pairs generated by targeted interventions without human labels, for reward model eval and training
Post-training shouldn't cost you diversity. With Gacha Decoding 🎲, it doesn't.
The idea: treat diversity as an instruction-following task and give the model an RNG tool. Unlike prior methods, it gets better as models get stronger.
Reroll your LM 🎲 arxiv.org/abs/2610.01382
Introducing Gacha Decoding 🎲: we show how to diversify LM responses, beating prior work by >11x in sample efficiency.
Reroll your LM for varied ideas, data, RL envs, {your use here}!
We get diversity from instruction following + external RNG, not token entropy. Here's how 🧵
1
9
677
1/ It's been so much fun building Muse Image/Video from scratch with the team. The story of Muse Image's agentic behavior did not begin as a designed feature. It began as an surprise to us, from the self-refinement behavior to the test-time scaling!
ai.meta.com/blog/introducing…
🚀 Hiring: Research Scientists at FAIR, Meta 🚀
We’re looking for strong candidates with expertise in post-training, reinforcement learning, or self-improvement methods for image and video generation.
If you’re passionate about advancing generative AI, please apply:
metacareers.com/profile/job_…
Join us to push the boundaries of generative AI.
Unfortunately, I won’t be at #CVPR this year, but if you’re working in this space, apply and attend Meta Mixer event at CVPR.
🚀 Meta Mixer at CVPR 2026!
Heading to #CVPR2026?
Meta is hosting a special networking mixer for researchers and engineers!
Join us to engage with our team, discuss ongoing projects, and connect with peers in the field.
📍 Denver, CO
📅 June 5, 2026
🕡 6:30–9:30 PM MDT
🔗 cvprnetworkingevent2026.spla…
If you’ll be at CVPR, we’d love to hear from you! Looking forward to engaging conversations and new connections.
#CVPR #CVPR2026 #Networking
Marjan Ghazvininejad retweeted
Train Beyond Language. We bet on the visual world as the critical next step alongside and beyond language modeling. So, we studied building foundation models from scratch with vision.
We share our exploration: visual representations, data, world modeling, architecture, and scaling behavior! [1/9]
Marjan Ghazvininejad retweeted
[1/9] What happens when you treat vision as a first-class citizen during multimodal pretraining? To find out, we studied the design space of training Transfusion-style models that input and output all modalities, from scratch. Here is what we learned about visual representations, data, world modeling, architecture, and scaling behavior!
Paper: arxiv.org/abs/2603.03276
Website: beyond-llms.github.io/
@TongPetersb, @DavidJFan, @__JohnNguyen__, @ellisbrown, @GaoyueZhou, @JasonQSY, @boyangzheng, @webalorn, @han_junlin, @rob_fergus, @NailaMurray, @gh_marjan, @ml_perception, Nicolas Ballas, @_amirbar, Michael Rabbat, Jakob Verbeek, @LukeZettlemoyer, @koustuvsinha, @ylecun, @sainingxie
Marjan Ghazvininejad retweeted
Humans communicate through language and interact with the world through vision, yet most multimodal models are language-first. What happens when we go beyond language? 🤔
Beyond Language Modeling: a deep dive into the design space of truly native multimodal models
Paper: arxiv.org/abs/2603.03276
Project: beyond-llms.github.io/
Marjan Ghazvininejad retweeted
Personalization assumes you need history with a user. What if you don't?
Cold-start is hard: each task&user has many preference dimensions, but each user only cares about a few.
A few strategic questions is all you need, if u know how preferences correlate across population👉🏻🧵
Marjan Ghazvininejad retweeted
‼️Multimodal T2I models generate "reasoning" for image generation—but this often doesn't help produce better images! Can we fix this?
Our reward-weighted post-training enables fully automatic joint text-image generation, boosting prompt alignment and knowledge-based generation.
Marjan Ghazvininejad retweeted
I've spent the past week anxiously following the news in Iran. Protests in the last few days have escalated and taken a dark turn. Thousands have been killed or arrested, and the government has cut off phone and internet access, leaving tens of millions of people unable to communicate.
My heart goes out to the innocent people who are affected by all this, including Iranians who live in the US and elsewhere. ❤️ Many are attempting to continue their studies or work and get through their daily lives under the immense pressure of wondering what's happening in Iran.
Some who are financially supported by their families in Iran are in need now. Some may have lost friends or family members. Many can’t even contact their family members to know if they’re safe. There are others who might have traveled home to Iran for Xmas and are now stuck, unable to get a flight back.
Even those who aren't directly impacted are silently carrying an emotional burden. I've spent much of the past week feeling helpless and longing for some way to help or be involved. I miss the city of my birth and feel the suffering of the people there.
If you're reading this and you work with anybody from Iran, be mindful of what they might be going through. Consider giving them space and understanding, extending deadlines, offering tuition relief.... or just a hug. Much love and hope. ❤️
The death toll from mass protests in Iran has risen to 538, according to data compiled by the U.S.-based Human Rights Activists News Agency on Sunday. abcnews.link/BEhhx1s
Marjan Ghazvininejad retweeted
🚨 Benchmark drift is real!
We show that the widely-used GenEval is now off by up to 17.7% vs human judgment when evaluating modern T2I models.
Introducing GenEval 2 and Soft-TIFA: harder and more compositional T2I prompts, and stronger and more robust human alignment. (1/N)
Thanks @_akhaliq for sharing our work!
Marjan Ghazvininejad retweeted
check out our work on building a unified model for generating interleaved text and video sequences...
we show that interleavingly 𝘁𝗵𝗶𝗻𝗸𝗶𝗻𝗴 𝗶𝗻 𝘄𝗼𝗿𝗱𝘀 before 𝗮𝗰𝘁𝗶𝗻𝗴 𝗶𝗻 𝗽𝗶𝘅𝗲𝗹𝘀 can help improve video generation!! 👇
Can we simplify video generation by decomposing it into interleaved text-video co-generation?
Would explicit, repeated thinking in language improve generation in pixels?
We introduce TV2TV: a unified model that jointly learns
- language modeling (next-token prediction)
- video flow matching (next-frame prediction)
At inference, TV2TV dynamically alternates between textual thinking and video generation.
Model generations below: interleaved text plans and video slices (~1–2s) are co-generated over time, conditioned on a single frame per sport.
📖 arxiv.org/abs/2512.05103

























