Aryo Lotfi retweeted
We generated 458,000 RL environments to train an agent on stateful, multi-step on-device tasks. We share in our preprint how we decided along which axes to enlargen our RL env corpus, via data composition scaling laws.
📄 arxiv.org/abs/2610.37648
🧵1/9
Aryo Lotfi retweeted
We let an agent self-improve by exploring and internalizing ”on this sort of task, keep this sort of thing in mind“ bits of self-feedback. It learned to solve tasks where the original policy failed 128 attempts in a row and GRPO flatlined.
Paper: arxiv.org/abs/2609.37633
🧵1/8
Aryo Lotfi retweeted
Is your gating mistuned?
In our new work, we highlight the critical role of gating for SSMs. Correct gating boosts areas of traditional weakness: in-context learning, retrieval, and long-context performance.
arxiv.org/abs/2609.16540
[1/6]
Aryo Lotfi retweeted
Tired of building complex environments to train AI agents? 🛑
Check out our environment-free way to generate high-quality agentic trajectories
#DigitalWorldModeling
arxiv.org/abs/2607.16900
@seanie_12 @schowdhury671 @chaojiang06 @cydhsieh Ting-Yao Hu @alexttoshev @OncelTuzel
Aryo Lotfi retweeted
(1/5) Open Publication and the Prisoners dilemma.
The prisoner’s dilemma game can offer us much insight into the publication strategy of leading industrial AI research labs. Suppose there are two labs A and B. If A and B both publish they are both better off than if they both don’t publish as they can build on the results of each other.
Aryo Lotfi retweeted
If you are excited about Multimodal and Agentic Reasoning with Foundation Models, Apple ML Research has openings for Researchers, Engineers, and Interns in this area. Consider applying through the links below or feel free to send a message for more information.
- Machine Learning Researcher: jobs.apple.com/en-us/details…
- Machine Learning Engineer: jobs.apple.com/en-us/details…
Aryo Lotfi retweeted
Come work with us! The Machine Learning Research (MLR) team at Apple is seeking a passionate AI researcher to work on Efficient ML algorithms:
jobs.apple.com/en-us/details…
Check out our new work, "RL for Reasoning by Adaptively Revealing Rationales", where we show the benefits of doing GRPO with parts of the solution revealed.
The revealed portion is adjusted dynamically for each sample depending on the difficulty of that sample.
Aryo Lotfi retweeted
🧵 1/8 The Illusion of Thinking: Are reasoning models like o1/o3, DeepSeek-R1, and Claude 3.7 Sonnet really "thinking"? 🤔 Or are they just throwing more compute towards pattern matching?
The new Large Reasoning Models (LRMs) show promising gains on math and coding benchmarks, but we found their fundamental limitations are more severe than expected.
In our latest work, we compared each “thinking” LRM with its “non-thinking” LLM twin. Unlike most prior works that only measure the final performance, we analyzed their actual reasoning traces—looking inside their long "thoughts". Our analysis reveals several interesting results ⬇️
📄 machinelearning.apple.com/re…
Work led by @ParshinShojaee and @i_mirzadeh, and with @KeivanAlizadeh2, @mchorton1991, Samy Bengio.
Aryo Lotfi retweeted
Meet the recipients of the 2024 ACM A.M. Turing Award, Andrew G. Barto and Richard S. Sutton! They are recognized for developing the conceptual and algorithmic foundations of reinforcement learning. Please join us in congratulating the two recipients! bit.ly/4hpdsbD














