
CS PhD @UTCompSci | SR @GoogleResearch
Joined October 2016
- Tweets449
- Following638
- Followers284
- Likes11.9K
Atula Tejaswi retweeted
GOOGLE KEEPS SHIPPING SOTA MODELS 🥇
Gemini 4 Neon achieves the highest score on FutureSim!
It launched 500+ subagents(!), performed over 16K tool calls and wrote simulations for 50+ questions spaning state elections, sports matches, job displacement, stock market and more!
Truly incredible comeback from Google.
Atula Tejaswi retweeted
Gemini 4 Neon is rolling out now.
Sitting between Carbon and Hydrogen, it brings frontier intelligence and fast task completion for complex software engineering, knowledge work, and science.
Atula Tejaswi retweeted
GEMINI IS BACK 👑
Gemini 4 Carbon has topped the FutureSim leaderboard dominating every other model throughout the simulation!
It is also very efficient, using the least number of tokens, search tool calls and subagents compared to Opus 5.5 and GPT 6.1 Sol.
Atula Tejaswi retweeted
Excited to have led model cascading in Kiro Auto from idea to launch with a fantastic team. More done, fewer credits, quality held high.
Also moonlighted as a front-end designer for the blog. Come for the science, stay for the animations. 👻
Replying to @kirodotdev
@kirodotdev's Auto mode has a new capability: model cascading.
Auto begins every task on a fast, efficient model and quietly checks the work as it goes. If a task gets harder, it hands off to a more capable model that picks up right where it left off. Across the sessions we analyzed, more than 80% of tasks finished on the efficient model alone.
When builders choose one model up front, they can either overpay on easy tasks or come up short on the more involved projects. Model cascading helps builders avoid that trade-off. kiro.dev/blog/model-cascade/
Atula Tejaswi retweeted
Can diffusion models be used for prime factorization? In our new paper with @bodonoghue85, @weballergy, @demishassabis, and others, we explore the limits of this approach.
For small targets, transformer-based discrete diffusion combined with a novel beam search procedure can reliably predict the factor bits of held-out semiprimes. We pushed the ML-based SOTA from 56 to 68 bits, and conducted an empirical scaling study across data, compute, and problem size.
The key bottleneck: problem size. Unlike natural language, which exhibits easily exploitable local substructure, prime factorization lacks the kind of smooth inductive bias neural nets thrive on - and our unfavorable scaling curves suggest a different approach is needed.
We’re excited to share these findings to provide a rigorous baseline and help guide the community on where to look next.
Atula Tejaswi retweeted
>continual learning blog post
>look inside
>it's rl post-training with some change in policy
Atula Tejaswi retweeted
I've got a lot of RL/post training followers who may not know what scalable oversight is, which is what im working on now at resolution. It's frequently discussed in the alignment context but really the core problem is critical for capabilities too, and may be necessary to even reach no-human-in-the-loop RSI.
Put simply: how do we train models supervised by other models, and how to scalably do RL when the AI judges/reward functions are weaker than the policy and so can be reward hacked? In other words, methods to mitigate reward hacking when RLing fuzzy tasks. If this seems interesting to you, reach out. There's some cool work we'll be doing at res.
Atula Tejaswi retweeted
Update: Can AI agents use Pangram to beat Pangram? In small tests, I found simple prompts to the Codex allowed it to beat Pangram. The final output was 100% human generated. Prompts that helped:
- Have the agent develop a style guide based on past writing
- Have it develop an "anti-style guide" based on AI writing tics that people have complained about
- Instruct it to get to a seed sentence that Pangram doesn’t flag (or at least flags as AI-assisted rather than fully AI-generated). Then the agent knows what strategies work and can iterate from there
I tried exactly this back in February using Claude Code. It worked pretty easily, though ~all of the important ideas had to come from me and CC itself couldn't figure them out.*
I tried it again a week ago. Complete failure, despite Claude being much better and not needing as much guidance from me. Pangram seems to have been dramatically hardened in the last few months. The API kept returning scores > 0.99 for AI writing so there wasn't even a gradient for it to figure out what's working and what isn't.
* Things it couldn't figure out on its own a few months ago, even after a bunch of iterations:
1) Remove well-known tells like em-dashes (!)
2) Do controlled experiments instead of stuffing a bunch of ideas at once
3) Start from known-best text and surgically make edits. Much better than trying to rewrite the whole essay (or even paragraphs or sentences) because the that just produces text from the LLM distribution all over again.
In my more recent experiment, 1) and 2) were obvious to Fable but 3) still wasn't. Pretty surprising.
Anyway, hats off to @pangram. I'll probably give it another shot at some point but I'm more-or-less ready to change my mind as an AI-detection skeptic.
This was the LLM summary from my first experiment BTW. Still gives me a chuckle.
Atula Tejaswi retweeted
🧵 [1/14]: How should we loop a transformer when model size 📦, training compute 🧮, and inference efficiency ⚡ all matter?
Looping is inherently simple: take the hidden representations produced by a transformer (or part of it) and feed them back through the same transformer (or part of it), reusing its weights. 🔁
But every extra loop costs compute. Choosing which layers to loop, how many times, and how to execute those loops determines whether the trade-off is worthwhile.
We explore these choices in our new paper:
“How to Loop Your Transformer: Loop Scaling, Specialization, and Parallel Inference”
Here’s what we found. Thread below 🧵👇
📄 Paper link: arxiv.org/abs/2610.40325
Atula Tejaswi retweeted
How to Loop Your Transformer: Loop Scaling, Specialization, and Parallel Inference
New paper from Meta.
"Our experiments span models with up to 1B parameters, trained on up to 155B tokens using up to 1.07 × 10^21 training FLOPs, with up to 32 loops. We find that looping the full transformer generally gives the strongest quality at a fixed parameter budget, whereas looping a middle block of layers gives a better return when training FLOPs are constrained."
"We compare exact weight sharing across 2 loops with small loop-specific low-rank adapters that allow each loop to learn a slightly different transformation. These adapters provide only modest gains when few loops are used, but become consistently more useful as the same weights are reused across more loops"
"Naively applying Parallel Looped Transformers (PLT) in our setting does not preserve the gains from looping: quality can degrade as more loops are added. We find that preserving the current-token representation, separately normalizing it and the recurrent state, and controlling how they are combined substantially improves PLT."
Atula Tejaswi retweeted
Marin's 535b is now >80% through pretraining! Here’s loss vs. our scaling-ladder forecasts. The sharp drop comes with the 4K→16K context switch, so we're kind of cheating the forecast. (This was planned.) We'll re-eval at 4k too. forge.coreweave.com/wandb/ma…

ALT Forecast versus observed loss: Hero’s evaluation loss falls below both scaling-ladder forecasts, reaching 1.528 at step 314,999. A sharp drop accompanies the switch from 4K to 16K context near step 258,000, making subsequent values not directly comparable with the 4K forecasts. The cooldown mixture begins at step 312,192.
Check out our work on enabling Latent Reasoning without modifying the main LLM, led by @Qtay001!
Atula Tejaswi retweeted
We're looking for new colleagues in Tübingen!
🚀 The ELLIS Institute Tübingen is hiring Principal Investigators as Hector Endowed Fellows!
Enjoy globally competitive packages, full academic freedom & possible co-appointments w/ @MPI_IS and the Tübingen AI Center (@uni_tue).
📍 Tübingen, 🇩🇪
📅 Apply by Dec 13
🔗 institute-tue.ellis.eu/jobs/…
#AI #MachineLearning #Hiring
Atula Tejaswi retweeted
Exciting new results in Theoretical Computer Science from our team at @GoogleDeepMind related to discovering the simplest programs that run the longest before halting! Pushing the frontiers of AI driven open-ended discovery in mathematics and computer science.
Over the past year, the Discovery team at @GoogleDeepMind, in a project led by student researcher @dsantosmarco, has made new discoveries in one of the most beautiful open problems at the edge of computation: Finding the 6th Busy Beaver (BB) number.
While most AI math systems solve human-generated problems, like theorems, this work tackles a different aspect of mathematical creativity: Discovering hidden problems.
For years, the BB Community has pursued one of theoretical computer science’s most captivating collaborative quests: finding the simplest programs that run the longest before halting. Because determining whether these programs ever stop is uncomputable, no universal algorithm can solve it, meaning new mathematical techniques need to be found and invented to make progress.
In this way, the BB challenge serves as an open ended domain for mathematical creativity and, in particular, a generator of hard math problems called Cryptids. These are tiny programs whose halting behavior is governed by and equivalent to deep, unsolved math problems in number theory, similar to the Collatz or Goldbach conjectures. Historically, the BB community had recognized ~20 Cryptids in BB(6), and the best problems were selected for the Busy Math Olympiad (BMO).
Building on their work, Marco built a pipeline powered by Gemini 4 Argon that:
1️⃣ Simulates raw Turing machine traces to compress recurring patterns.
2️⃣ Conjectures mathematical transition rules & runs synthetic experiments to test edge cases.
3️⃣ Proves the equivalence formally in Lean 4.
Using this pipeline, Marco and the team discovered 20 new potential Cryptids and verified 33 more machines previously studied in the BB community. All the proofs were open sourced and shared in the community discord channel and we hope that they will help the community to make progress towards resolving BB(6). We also contributed related halting/non-halting proofs directly to the community GitHub repository.
Finding these Cryptids shows hidden links between simple programs and number theory, and helps mathematicians map the limits of computation. I'm also excited about Gemini 4 Argon's ability to help researchers both solve hard problems and accelerate scientific discoveries, of which Busy Beaver is just one example.
Work led by our student researcher @dsantosmarco with paper contributions from @vivek_veeriah, @Dahoas1 , Andrew Bolt, @zuza777 , Satinder Singh, and myself with additional contributions from Hussain Masoom, @maxencefaldor , @MoritzFirsching and Petar Sirkovic.
Read more about it in our paper: arxiv.org/abs/2610.22603







































