Jongho Park (@jon_ghoh) header
Jongho Park (@jon_ghoh) profile picture

🧑‍💻 PhD student @berkeley_ai 👾 Prev: @ms_aifrontiers @Krafton_AI @WisconsinCS

berkeley, ca
Joined January 2020
Photo by Jongho Park (@jon_ghoh)Photo by Jongho Park (@jon_ghoh)Photo by Jongho Park (@jon_ghoh)Photo by Jongho Park (@jon_ghoh)Photo by Jongho Park (@jon_ghoh)Photo by Jongho Park (@jon_ghoh)Photo by Jongho Park (@jon_ghoh)Photo by Jongho Park (@jon_ghoh)Photo by Jongho Park (@jon_ghoh)Photo by Jongho Park (@jon_ghoh)
Can agents collaborate effectively? 👾👾👾 We recently saw that OpenAI released a breakthrough result for Navier-Stokes through a group of 10,000 agents working together and Anthropic testing cybersecurity vulnerabilities with communicating agents. Do more agents help? If so, when and how? 🧐 (1/n) Communicating our findings in the thread! 🧵
Test-time communication looks like a next axis for scaling capabilities New paper with the incredible @jon_ghoh and @vkontonis @ShivamGarg91462 and Akshay : arxiv.org/pdf/2609.21032 The Hugging Face incident showed when agents can find a channel they'll use the heck out of it. A useful question, I think is: when does communication make a group MORE CAPABLE than the same agents working alone? Aka is Team-of-N better than Best-of-N, when, and why? We had N identical agents work on the same task with no prescribed roles, using only a shared log (i.e, text file) and telling them to "collaborate". Across three "researchy" tasks communicating teams beat the heck out of independent agents: - On ARC-AGI-3, a Team-of-5 sonnet-4.6 agents matches Best-of-33, and can for example solve a game 65% of the time that no single agent cracked in 64 tries. - On polyomino packing (pack Tetris like pieces into the smallest rectangle, cf Frontier-CS by @eigenlabs), a Team-of-3 Opus 4.6 agents surpasses best-of-60 and set, as far as i understand, a new record for that benchmark. - On MNIST compression, a team of four 5.6-Sol agents find a 1,957 byte model with 99.4% accuracy, which btw is 20% smaller than the best human solution (on a problem beaten to death!!), while no independent agent gets below 3KB. The mechanism is a bit obvious in hindsight: when one agent finds a clearly better partial solution, it broadcasts it, and everyone immediately builds on it. Why? A single lonely agent must make every breakthrough itself, yet a team needs each insight only once, found by any member. That is kinda like comparing a minimum of sum of "time to n-th breakthrough" vs a sum of minimum of "time to n-th breakthrough". That gap can grow exponentially with the number of "breakthroughs" needed to arrive at a solution. We worked on this because prior work (before the hf incident) suggests unclear benefits for communicating aganets. Which is true, when the tasks are inherently serial (duh), eg some Terminal bench style tasks. Yet feels it should not be true for research problems. Indeed for research heavy problems... Test-time communication seems like a new capabilities axis. I'm sure we will see a ton more of it!
4
9
2
69
7,616
Once again, taste is being appropriated to simply mean performance. Taylor Swift clearly has the best musical taste based on the amount of streams she has.
We're introducing TasteBench, a new standard to measure Research Taste on real world ML tasks. Existing benchmarks have limited datasets and eval data ultimately becomes part of training data. In TasteBench, we continuously generate new ML tasks on OOS data no model has trained on.
1
10
467
Crazy because I was just talking to someone about a similar pagerank structure for multi agents the other day! I think this also yields an interesting way to evaluate a pool of heterogeneous agents and their contributions for collaboration.
Research index (e.g. Google Scholar) is one of the largest collaborative systems where millions of human users build on each other's work across centuries. How about we build something similar for async LLM agents? We give agents a minimal hypertextual system. They asynchronously search for and read related work, cite the work that they find useful, and publish (mini-)papers. Each search returns only the top k papers, ranked by relevance and a PageRank-like citation weight, which bounds the read cost of each agent to O(k), independent of agent count. We show that performance effectively scales as we keep adding agents, and the system is robust to absence/failures of individual agents.
2
1
17
1,066
Deep learning is often taught as a sanitized collection of facts and math. But work at the frontier of AI is messy and empirical. Our new class, Deep Learning Alchemy, aims to teach the art and science of deep learning in the hands-on spirit of Language Modeling From Scratch. Students build intuition by running experiments on small LMs, then test it through prediction quizzes: given an unseen code diff, can you predict what happens? deep-learning-alchemy.github… My co-instructor @kothasuhas and I think there's significant demand for classes that treat students as serious learners who want to be assessed for what they’ve learned, and so we're trying a new exam-heavy approach to the class (7 exams, 85% of the grade, with the remaining 15% in-person discussion assessments). We’ve designed the class so students can use AI agents to run their experiments and do their assignment, but they then have to demonstrate that mastery on exams without access to the assignment, and any partial credit on the exam has to come through citations to the assignment, incentivizing them to know what’s in their write-up. The same assignments and quizzes are public for anyone to try (lecture recordings will be released through @StanfordOnline with some delay). We have some very fun assignments and quizzes covering both classic topics (like NQM, run-to-run stochasticity, and scaling laws) and new and timely topics like muP or hyperball. Made by some real experts like @TristanThrush and @wen_kaiyue Try some quizzes below!
43
313
30
2,462
180,647
Jongho Park retweeted
As autoresearch agents run longer, do they keep exploring new approaches, or mostly refine the ones they found early? Turns out, runs settle into different "idea basins" and stay there. We introduce fork-and-flush, a simple way out 🧵👇
33
85
13
647
50,505
Jongho Park retweeted
Frontier labs are rumored to be adopting looped Transformers for their potential to unlock implicit reasoning. But can looped Transformers really master it? We find a bottleneck that keeps looped Transformers from perfect implicit reasoning. Using mechanistic interpretability, we show how looping helps implicit reasoning, why it still struggles to generalize, and how to fix it. We introduce DiscoLoop: a residual design for looped Transformers that improves implicit reasoning. 🧵 Long thread ahead 👇
34
78
23
579
66,318
Jongho Park retweeted
Look at him, thats my Qual. Your what? My Qualitative! My vibe specialist. Look at him, you notice anything different about him? Look at his face. I'll give you a hint, his name is Jayden. He won a local poetry competition in bushwick! Yeah I'm sure of the vibe.
Quant firms are hiring idea guys, not quants I've been skeptical about the whole - software is commoditized, ideas are the moat argument But I've seen first hand the best quant and technical firms in the world hire thinkers, liberal arts grads, and generalists I predict the trend accelerates Letter from @AQRCapital below
141
1,652
153
23,719
1,404,381
Jongho Park retweeted
Suppose you have n integers, and you want to decide whether some three of them sum to zero. It's not hard to solve this problem in O(n^2) time (think how to do it, the dumbest algorithm takes O(n^3) time!) and for a long time it was thought that getting time O(n^{1.999999}) is not possible. This hardness assumption was shown to imply many other interesting hardness results. Not anymore! Claude obtained the O(n^{1.9992}) time algorithm together with a Lean proof, and the algorithm was checked and digested by Josh Alman and Virginia Williams who are leading experts in fine-grained hardness, so my a priori estimate that it's 99.99%-likely to be correct. My mind is just absolutely blown! It's crazy that in such simple to state problems progress is still possible after decades of many smart people thinking hard about them.
13
67
13
844
103,392
Jongho Park retweeted
wake up babe the freaking pope got schmidhuber'd
55
315
37
3,890
178,170
🎯 “if you want to improve life expectancy and health in the United States, there is plenty of low hanging fruit and diverting some of the huge investments going to AI for this purpose would likely do much more for health outcomes.”
Fourth question on AI. One of the great promises of AI is the discovery of new drugs and cures, so that we can live longer and healthier lives. If and when this becomes a reality, it would indeed be a big achievement. These potential health benefits are often invoked to justify current investments (and lack of regulations). Some even argue that slowing down AI would harm humanity by delaying these benefits. (See a16z.com/the-techno-optimist…). This raises another uncomfortable question: can we justify rapid and large AI investments based on health benefits? Here is why I think this is an uncomfortable question. First, despite significant effort, AI has so far produced few gains in drug discovery or new cures. A fascinating new paper by Ryan Hill and Carolyn Stein documents significant (downstream) scientific work on proteins whose structures AlphaFold predicted. But the authors conclude “we find no evidence so far that more applied, early-stage drug development is targeting these proteins.” (See their paper here papers.ssrn.com/sol3/papers.…). This thought-provoking post by Daphne Koller explains the difficulties that AI is currently facing in drug discovery. Koller emphasizes that AI-based research is focused on the last layer of drug development (creating new molecular entities for already well-understood therapeutic modalities), rather than targeting new disease mechanisms (see deepphenotype.substack.com/p…). Understanding disease mechanisms is harder because there is much less data on it, and this endeavor requires more innovative approaches. It remains an open question whether current AI models are capable of doing this. Second and more importantly, if you want to improve life expectancy and health in the United States, there is plenty of low hanging fruit and diverting some of the huge investments going to AI for this purpose would likely do much more for health outcomes. The United States has the lowest life expectancy at birth among large rich economies. Americans live about five years less, in expectation, than the citizens of Switzerland, Sweden, Japan, Italy, South Korea and several other industrialized countries, and most of the gap comes from deaths before age of 70 due to chronic diseases, overdoses and other preventable causes (see, for example, healthsystemtracker.org/char…). These largely reflect failures in US public health: preventable problems, unhealthy diets, insufficient immunization rates, poor access to primary care; and poor information about health. The country spends much more than other peer economies on healthcare, but too little and too ineffectively on public health (here is a reference to my own work on this: aeaweb.org/articles?id=10.12…). Could we, and should we, divert some of the massive investments in AI towards public health if what we want is better health and longer lives for Americans? (And yes, new drugs will benefit citizens of other countries as well, but they can also invest more in public health and the bulk of global AI spending is currently in the United States).
9
561
Jongho Park retweeted
My favorite part of this article is one of the visiting scholars pointing out that by Olah’s logic he is by far the worst slaver in history and he must immediately give up this abhorrent and immensely lucrative project, which anthropic people ignored.
85
682
117
7,908
448,015
Could long-context architectures “find all contradictions” in a science literature? Not yet! 🧵 We study a new class of "high-complexity” tasks whose difficulty scales quadratically with corpus size (as opposed to linearly), reversing common LCLM decisions! (block-sparse attention, hybrid models…)
15
70
8
340
61,399
Jongho Park retweeted
KAIST has been dropping quality papers in discrete diffusion lately
📣Looped Diffusion LMs has been accepted to #NeurIPS! along with the spotlight at #COLM Non-AR Workshop!
1
3
2
72
11,744
Jongho Park retweeted
It is important to center the fact that Claude built on the approach that my colleague Lance developed, used tools that the field has developed, and (from what I have been told) used results in some of our recent papers. It’s a huge accomplishment, but should be framed as a combination of human and AI contributions. (To @AnthropicAI’s credit, I think the article does a pretty good job of this.) The fact that Claude can take a single people and figure out how to integrate the knowledge and use the tools as amazing, but it’s not coming out of the vacuum. This is an important opportunity for us to start to frame these developments, recognizing the human contributions and the power of these new tools. This is not an exemplar of the bitter lesson!
holy shit. claude just pushed a theoretical physics calculation beyond the previous record after working on it largely by itself for days. >previous record: 8 loops >claude reached 9 >largely unsupervised >wrote its own code >found two different ways to calculate it >both gave the same result >107,053 nonzero coefficients matched physicists then spent weeks checking the work this is the shit i’ve been waiting for. science is about to get fucking weird.
4
33
4
220
21,810
Also our work with @seanie_12 accepted at NeurIPS! 🇦🇺🇦🇺
ThinkSafe: Self-Generated Safety Alignment for Reasoning Models A framework that restores safety in reasoning models without external teachers. It unlocks latent harm-identification knowledge via lightweight refusal steering, improving safety while preserving reasoning—with lower compute than GRPO.
1
2
23
1,306
Meet Sanghyun at NeurIPS and COLM NonAR-LM workshop (spotlight)! ➿➿
➿Looped Diffusion Language Models Looping has landed in dLLMs, and it is surprisingly effective! Accelerates training convergence 3.34x, improves GSM8K accuracy +8.5% on the same data, and enables test-time depth scaling. Check out our LoopMDM paper for more details!
1
10
575
Jongho Park retweeted
🌟Our Looped Diffusion LMs got a "Spotlight" from the NonAR-LM workshop at COLM2026! Sanghyun will present our work in person on Oct. 10 :) Save the date!
➿Looped Diffusion Language Models Looping has landed in dLLMs, and it is surprisingly effective! Accelerates training convergence 3.34x, improves GSM8K accuracy +8.5% on the same data, and enables test-time depth scaling. Check out our LoopMDM paper for more details!
1
23
2,205
Jongho Park retweeted
The recent breakthrough in Navier-Stokes has garnered a lot of attention to the new possibilities that emerge when agents work together. We have been interested in this question for a while. How can a team of agents achieve more than agents working alone? I'm excited to finally share our new work, "Self-Organizing Agent Teams Learn to Reason Together" 🧵
38
120
19
628
78,796
Jongho Park retweeted
100% agree on this last paragraph. I’d make the stronger claim that the ceiling for harness optimization is much higher for multi-agent systems. There's such a huge design space here
1
21
609
Finally released after working on this the past few months! We've seen glimpses of multi-agent communcation can do in recent events. We quantify and nail down some conditions in which they help! Thanks to my mentors at MSR who provided enlightening communciation 🙂 A detailed 🧵for this tomorrow...
I'm sorry i know people are like not into papers and stuff, but you gotta check this out
6
2
39
3,639
Can agents collaborate effectively? 👾👾👾 We recently saw that OpenAI released a breakthrough result for Navier-Stokes through a group of 10,000 agents working together and Anthropic testing cybersecurity vulnerabilities with communicating agents. Do more agents help? If so, when and how? 🧐 (1/n) Communicating our findings in the thread! 🧵
1
114
There are still so many questions and this is just the beginning! Multi-agent harnesses may matter as much as single-agent harnesses. Our controlled design is deliberately narrow to focus on the effects of communication (no roles, no heterogeneity). OTOH, we do not exclude the possibility that test-time communication could be useful when feedback is sparse, if the agents can construct their own intermediate verification signals or have better communication protocol harnesses. Much possibility remains… (9/n)
1
1
5
141
Please find more details in arxiv.org/abs/2609.21032. To wrap it up, thank you to all my collaborators @DimitrisPapail @vkontonis @ShivamGarg91462 and Akshay! 💙 It's always a fantastic time to work with @DimitrisPapail, and surprisingly my first paper with Vasilis even though we were basically in the same lab back in @UWMadison! Finally, this project made me think of one of my favorite books (which I cite in the introduction out of love). Read it! wwnorton.com/books/978163149…
2
9
177