AI research & product @meta tbd, @instagram | before: @GoogleDeepMind @YouTube @Google news/discover, search ranking, ads | CS & Econ @yale
san francisco
Joined October 2017
- Tweets95
- Following2.5K
- Followers690
- Likes969
Devansh retweeted
$META Devansh Tandon reveals the tokens-in, engagement-out flywheel behind Meta's recommendation system, where better models lift Reels watch time 30% and the monetization pays for the next training run
"These scaling curves aren't just academic research. They're driving real product impact at scale for some of the biggest consumer businesses in the world."
"Here's a couple of examples I have from Meta's recent earnings reports. Instagram Reels had a strong quarter, 30% year on year watch time."
"And the optimizations we made to improve the quality of recommendations included simplifying our ranking architecture to enable efficient model scaling and longer interaction histories to identify a person's interests."
"We doubled the length of user interaction sequences used for training Instagram and increased the richness of each user interaction."
"So these are direct parallels to the power law scaling curves for LLMs."
"And I want to introduce this idea of a flywheel of tokens in engagement out, which is what's powering all of these recs models scaling."
"You train a model, you then run inference on it, which is the tokens in, that recommendation model results in better content recommendations, it drives consumer engagement, daily active users' time spent, it translates to monetization and ads or subscription, which pays for the next model training run."
"And so every step on the scaling curve is one loop around this flywheel, and a lot of consumer apps are spinning this core flywheel at the heart of their business. So we have a long way to scale these recommender systems."
_______
For the full breakdown of the lecture on Meta's ads recommendation system: firesidealpha.substack.com/p…
Mark Zuckerberg says Meta trains discretion into Muse so it can book a restaurant reservation without disclosing your allergy or pregnancy, something off-the-shelf coding agents skip
"So, I mean, you have to train that into the model. I mean, just like any other capability, it needs to be good at knowing, you tell it a lot of things and then you want it to go achieve your goals."
"But let's say you want it to go get you a reservation at a restaurant, and you're trying to find a good restaurant, but maybe you have some allergy. But you may not want to disclose the allergy, or if you're pregnant or something like that."
"But the Muse will know if you tell it. And you're going to want it to be able to do the job but while disclosing as little information as possible."
"So that's a specific skill. And it's something that's kind of basic social skills or common sense that people have."
"But if you're just building a coding agent, most of these companies haven't trained that in. So that's the type of thing that we can do because we're doing the full stack, right?"
"Because we're training the model, we're not just taking someone else's model off the shelf and trying to build a scaffold and an agent around it. We trained the whole model specifically to be good at that."
_______
For the full set of takeaways from Meta's Connect Keynote: firesidealpha.substack.com/p…
Devansh retweeted
criminally underrated talk from @devanshtandon_
Gemini powers YouTube's Large Recommender Model by **tokenizing every video on youtube** (SemanticID) - a vocabulary several OoMs larger than English, CONTINUOUSLY PRETRAINED every day. can reason across titles/descriptions, throws your entire demographic/watch history into an LRM lmao
20m videos are uploaded *per day* and run thru this (offline) system. absolutely bonkers scale LLM deployment
It's an honor to host the RecSys x LLMs track at @aiDotEngineer SF! In this talk, I discuss semantic IDs, LLM-based data augmentation, and foundation rankers.
Slides: eugeneyan.com/speaking/aie-2…
Talk: youtube.com/watch?v=2vlCqD6i…
This is my talk's semantic ID: 2vlCqD6igVA. I wish knew what each token in the hierarchy and codebook meant lol. If you work on semantic IDs at YouTube and can shed some light, please reach out!
Devansh retweeted
I asked how an LLM powered recommendation system would work and got an excellent answer here.
