Eran Malach (@EranMalach) header
Eran Malach (@EranMalach) profile picture

Apple

Joined December 2019
Photo by Eran Malach (@EranMalach)Photo by Eran Malach (@EranMalach)Photo by Eran Malach (@EranMalach)Photo by Eran Malach (@EranMalach)Photo by Eran Malach (@EranMalach)Photo by Eran Malach (@EranMalach)Photo by Eran Malach (@EranMalach)Photo by Eran Malach (@EranMalach)Photo by Eran Malach (@EranMalach)Photo by Eran Malach (@EranMalach)
Eran Malach retweeted
We generated 458,000 RL environments to train an agent on stateful, multi-step on-device tasks. We share in our preprint how we decided along which axes to enlargen our RL env corpus, via data composition scaling laws. 📄 arxiv.org/abs/2610.37648 🧵1/9
21
34
5
323
17,605
Eran Malach retweeted
1/9 I spent the summer doing research at Apple. So I worked on how to deploy small models on small devices, but without losing performance. A surprising result: by running a 7B model multiple times, we can match or beat a 32B model—while running faster and using less energy.
6
13
2
110
5,019
Eran Malach retweeted
We let an agent self-improve by exploring and internalizing ”on this sort of task, keep this sort of thing in mind“ bits of self-feedback. It learned to solve tasks where the original policy failed 128 attempts in a row and GRPO flatlined. Paper: arxiv.org/abs/2609.37633 🧵1/8
21
57
6
597
94,055
Eran Malach retweeted
Is your gating mistuned? In our new work, we highlight the critical role of gating for SSMs. Correct gating boosts areas of traditional weakness: in-context learning, retrieval, and long-context performance. arxiv.org/abs/2609.16540 [1/6]
1
3
1
8
1,563
Are EMA teachers commonly used in video SSL (👀V-JEPA family) actually necessary? Hot take: your EMA teacher is a waste of compute Introducing SALT. ICLR 2026 🇧🇷 | Led by @XianhangLi at Apple MLR.🧵
8
9
3
91
13,880
Eran Malach retweeted
New paper 🥳 RL relies a lot on an agent’s capability to explore. Our strategy-guided exploration makes the agent find new solutions more efficiently. It learns faster, and in some environments its Pass@1 surpasses the base model’s Pass@128. 🧵1/6 📄 arxiv.org/abs/2603.02045
3
13
68
4,507
With @ezra_edelman, we’re presenting our NeurIPS 2025 paper “Let Me Think! A Long Chain-of-Thought Can Be Worth Exponentially Many Short Ones” (with Samy Jelassi, @EranMalach, and @eaboix) at 4:30-7:30 PT in Exhibit Hall C,D,E #3514. If you’re at NeurIPS, come by and say hi!
2
1
18
953
Eran Malach retweeted
I really like this research direction! For a long time, I've been talking about the "brain vs. database" analogy of SSMs vs Transformers. An extension of this that I've mentioned offhand a few times is that I think that the tradeoffs change when we start thinking about building multi-component *systems* rather than single models. For example, if one subscribes to the intuition that modern hybrid models are using the SSM as the main "brain-like" processing unit while the attention is primarily for "database-like" caching to aid with precise retrieval, then I hypothesized that perhaps a more optimal system could be a pure SSM language model combined with explicit external knowledge databases and context caches. This is much more analogous to human-like intelligence that is primarily driven by the brain (an SSM) aided by external knowledge stores (books, the internet) and tool use. This paper shows pretty interesting results that SSMs do seem to have very favorable performance compared to Transformers in this regime of agentic models operating with interactive tool use. Glad to see the intuition validated, and I hope more research continues along these lines!
SSMs promised efficient language modeling for long context, but so far seem to underperform compared to Transformers in many settings. Our new work suggests that this is not a problem with SSMs, but with how we are currently using them. Arxiv: arxiv.org/pdf/2510.14826 🧵
10
63
2
533
66,869
RL has led to amazing advances in reasoning domains with LLMs. But why has it been so successful, and why does the length of the response increases during RL? In new work, we introduce a framework to provide conceptual and theoretical answers to these questions.
2
14
1
64
5,289
Eran Malach retweeted
Why is RL after pretraining so effective? RL quickly upsamples the generalizable-but-rare patterns  in the pretrained model.
11
43
3
399
23,718
SSMs promised efficient language modeling for long context, but so far seem to underperform compared to Transformers in many settings. Our new work suggests that this is not a problem with SSMs, but with how we are currently using them. Arxiv: arxiv.org/pdf/2510.14826 🧵
6
82
12
410
116,013
Our findings highlight that while SSMs are weaker when evaluated “standalone”, they have clear benefits when equipped with tool-use. We hope our work encourages the community to start building interactive SSM-based reasoning and coding agents.
1
2
21
1,504
Work with my amazing collaborators at Apple: @Omid_Sar, @sineadwilliamso, @ArwenBradley, @aryolotfi, Emmanuel Abbe, @jmsusskind, @EtaiLittwin
13
1,341
🤯LLMs can solve the sparse parity learning problem with small sample complexity by realizing they need to run Gaussian elimination over F2.
🧵 New paper: LLM-ERM: Sample-Efficient Program Learning via LLM-Guided Search arxiv.org/abs/2510.14331 We use reasoning LLMs to learn tasks like IsPrime from ~200 samples by proposing short programs, making both the learned function *and* the learning process interpretable 🤯
3
18
2,676
🧵 New paper: LLM-ERM: Sample-Efficient Program Learning via LLM-Guided Search arxiv.org/abs/2510.14331 We use reasoning LLMs to learn tasks like IsPrime from ~200 samples by proposing short programs, making both the learned function *and* the learning process interpretable 🤯
2
10
3
36
8,657
Eran Malach retweeted
🤔 Ever thought a small teacher could train a student 6× larger that sets new SOTA in training efficiency and frozen evaluation performance for video representation learning? 🤔 Do we really need complex EMA-based self-distillation to prevent collapse, bringing unstable loss dynamics while offering little insight into representation quality? 🚨 In our new paper, we investigate these questions and propose SALT (Static-teacher Asymmetric Latent Training): a simple, scalable, and compute-efficient alternative for video representation learning. 📄 Rethinking JEPA: Compute-Efficient Video SSL with Frozen Teachers 🔗 arxiv.org/abs/2509.24317
7
69
7
452
39,302