Aryo Lotfi (@aryolotfi) profile picture

Research scientist at Apple | PhD @ EPFL | ML researcher focusing on reasoning

Lausanne, Switzerland
Joined November 2012
We generated 458,000 RL environments to train an agent on stateful, multi-step on-device tasks. We share in our preprint how we decided along which axes to enlargen our RL env corpus, via data composition scaling laws. 📄 arxiv.org/abs/2610.37648 🧵1/9
21
34
5
323
17,605
We let an agent self-improve by exploring and internalizing ”on this sort of task, keep this sort of thing in mind“ bits of self-feedback. It learned to solve tasks where the original policy failed 128 attempts in a row and GRPO flatlined. Paper: arxiv.org/abs/2609.37633 🧵1/8
21
57
6
597
94,055
Aryo Lotfi retweeted
Is your gating mistuned? In our new work, we highlight the critical role of gating for SSMs. Correct gating boosts areas of traditional weakness: in-context learning, retrieval, and long-context performance. arxiv.org/abs/2609.16540 [1/6]
1
3
1
8
1,563
(1/5) Open Publication and the Prisoners dilemma. The prisoner’s dilemma game can offer us much insight into the publication strategy of leading industrial AI research labs. Suppose there are two labs A and B. If A and B both publish they are both better off than if they both don’t publish as they can build on the results of each other.
13
53
5
362
88,701
If you are excited about Multimodal and Agentic Reasoning with Foundation Models, Apple ML Research has openings for Researchers, Engineers, and Interns in this area. Consider applying through the links below or feel free to send a message for more information. - Machine Learning Researcher: jobs.apple.com/en-us/details… - Machine Learning Engineer: jobs.apple.com/en-us/details…
11
48
450
40,836
Come work with us! The Machine Learning Research (MLR) team at Apple is seeking a passionate AI researcher to work on Efficient ML algorithms: jobs.apple.com/en-us/details…
5
27
1
197
21,896
Check out our new work, "RL for Reasoning by Adaptively Revealing Rationales", where we show the benefits of doing GRPO with parts of the solution revealed. The revealed portion is adjusted dynamically for each sample depending on the difficulty of that sample.
Why does RL struggle with tasks requiring long reasoning chains? Because “bumping into” a correct solution becomes exponentially less likely as the number of reasoning steps grows. We propose an adaptive backtracking algorithm: AdaBack. 1/n
1
13
1,944
🧵 1/8 The Illusion of Thinking: Are reasoning models like o1/o3, DeepSeek-R1, and Claude 3.7 Sonnet really "thinking"? 🤔 Or are they just throwing more compute towards pattern matching? The new Large Reasoning Models (LRMs) show promising gains on math and coding benchmarks, but we found their fundamental limitations are more severe than expected. In our latest work, we compared each “thinking” LRM with its “non-thinking” LLM twin. Unlike most prior works that only measure the final performance, we analyzed their actual reasoning traces—looking inside their long "thoughts". Our analysis reveals several interesting results ⬇️ 📄 machinelearning.apple.com/re… Work led by @ParshinShojaee and @i_mirzadeh, and with @KeivanAlizadeh2, @mchorton1991, Samy Bengio.
109
560
111
3,072
909,048
66
205
29
3,982
248,561
Meet the recipients of the 2024 ACM A.M. Turing Award, Andrew G. Barto and Richard S. Sutton! They are recognized for developing the conceptual and algorithmic foundations of reinforcement learning. Please join us in congratulating the two recipients! bit.ly/4hpdsbD
31
455
199
1,507
458,898