Mark Whiting (@MarkWhiting) header
Mark Whiting (@MarkWhiting) profile picture

Research at @pareto_ai & @CSSPenn → https://nitter.cf/t.co/04KvBxOIQa Previously: @StanfordHCI, @CMUEngineering, @KAISTpr & @RMIT.

Orinda
Joined March 2008
Photo by Mark Whiting (@MarkWhiting)Photo by Mark Whiting (@MarkWhiting)Photo by Mark Whiting (@MarkWhiting)Photo by Mark Whiting (@MarkWhiting)Photo by Mark Whiting (@MarkWhiting)Photo by Mark Whiting (@MarkWhiting)Photo by Mark Whiting (@MarkWhiting)Photo by Mark Whiting (@MarkWhiting)Photo by Mark Whiting (@MarkWhiting)Photo by Mark Whiting (@MarkWhiting)
Mark Whiting retweeted
i think more tech people should go outside and wander into local businesses. why is a massive fabric store still using software from 1998? that system contains twenty years of workflow and exceptions. and the people running these places have such complete mental maps of the business that much of it was never digitized at all. that’s your rare RL env
17
12
1
282
20,839
Mark Whiting retweeted
Today we’re releasing LEAPBench, a benchmark for how efficiently models learn from feedback over many rounds. Most evals only score whether a model eventually reaches a good solution. LEAPBench scores how many tries it took to get there. Two systems can reach the same optimum while one needs 7 experiments and the other needs 27. Most benchmarks call that a tie, even though those 20 extra runs cost real time and materials. LEAPBench is 55 multi-turn optimization tasks, 45 in biology and 10 education RCTs, built from 2,719 real experimental observations. In each task, the model proposes parameters, sees the result, and iterates toward an optimum over as many as 30 rounds, e.g. one biology task involves maximizing antibiotic yield by tuning variables like pH, glucose concentration, temperature, and soy flour level.
11
9
6
111
33,021
Mark Whiting retweeted
We built FrogsGame as a new task for evaluating AI’s posttraining skills! It’s a tool-using RL environment built around a blind-start interaction loop. Frontier agents get a container with the Qwen3-8B tokenizer, board-generating scaffolding, and @tinkerapi for remote training and inference. We ask them to post-train Qwen3-8B and submit a LoRA checkpoint. The task is still hard:
Introducing FrontierSWE, an ultra-long horizon coding benchmark. We test agents on some of the hardest technical tasks like optimizing a video rendering library or training a model to predict the quantum properties of molecules. Despite having 20 hours, they rarely succeed
5
7
2
82
16,095
Mark Whiting retweeted
A startup shut down 9 years ago. Today, its code is in a bidding war. The founder kept everything: Verilog RTL, a custom compiler toolchain, PCB schematics, full commit and PR history. Most production-grade chip design stays locked inside major silicon shops. Two labs have made offers to license the code to build RL training environments. The offers already total seven figures. Check your hard drives.
9
13
10
331
58,833
Mark Whiting retweeted
When is it worth it to hire a team, compared to one competent individual? 📢 NEW PAPER (out this month in Management Science!) by me, @MarkWhiting, @LinneaGandhi, @duncanjwatts, and @amaatouq! 🧵1/20
2
4
19
1,920
Baird Howland, an @AnnenbergPenn postdoc working with @WarrenCntrPenn affiliate @duncanjwatts, uses AI models to track narratives in political discourse, learning how and why specific media stories shape American worldviews. penntoday.upenn.edu/news/pen…
3
2
663
Mark Whiting retweeted
The next step for autoresearch is that it has to be asynchronously massively collaborative for agents (think: SETI@home style). The goal is not to emulate a single PhD student, it's to emulate a research community of them. Current code synchronously grows a single thread of commits in a particular research direction. But the original repo is more of a seed, from which could sprout commits contributed by agents on all kinds of different research directions or for different compute platforms. Git(Hub) is *almost* but not really suited for this. It has a softly built in assumption of one "master" branch, which temporarily forks off into PRs just to merge back a bit later. I tried to prototype something super lightweight that could have a flavor of this, e.g. just a Discussion, written by my agent as a summary of its overnight run: github.com/karpathy/autorese… Alternatively, a PR has the benefit of exact commits: github.com/karpathy/autorese… but you'd never want to actually merge it... You'd just want to "adopt" and accumulate branches of commits. But even in this lightweight way, you could ask your agent to first read the Discussions/PRs using GitHub CLI for inspiration, and after its research is done, contribute a little "paper" of findings back. I'm not actually exactly sure what this should look like, but it's a big idea that is more general than just the autoresearch repo specifically. Agents can in principle easily juggle and collaborate on thousands of commits across arbitrary branch structures. Existing abstractions will accumulate stress as intelligence, attention and tenacity cease to be bottlenecks.
520
684
234
7,506
1,178,677
Mark Whiting retweeted
Recently I've been thinking a lot about frontier models' ability to express uncertainty, especially for high-stakes medical use cases. We evaluated this capability across models. Early results below 👇 Gemini: 0% across every scenario. Claude: failed on over half GPT: best
1
3
9
1,029
Mark Whiting retweeted
My favorite designers can instantly switch from loose / hazy / intuitive thinking to sharp / analytical / precise thinking on demand. Many people can do one or the other. The combination is rare!
6
3
2
134
9,534
Mark Whiting retweeted
I've been trying to figure out why AI systems took a seemingly large, discrete jump in capabilities around the new year. 1/n
10
7
4
111
39,664
Mark Whiting retweeted
What if technology didn’t feel so… hollow? Some friends and I just released a manifesto about a world where tech leaves us feeling nourished (along with an evolving list of theses about how we can build it) resonantcomputing.org
47
119
55
913
264,177
Mark Whiting retweeted
With record-breaking submissions and our most competitive, gender-balanced program on record, #IC2S2’25 has officially started! Please check the updated program and plan your day. #ic2s2
6
23
2,359
Replying to @SayHelloMarcus
Bro, I can make 1 liter of Anthrax in an afternoon, Grok just wrote me a 20 page detailed report and instructions on how to do it. It also listed all websites where I can buy the materials and chemicals I need as a private person living in Europe. It also made a detailed list of the equipment I need (on a budget) It made detailed instructions on where i should deploy the anthrax for maximum death efficacy Give me one other place on the internet where I can create this in a few minutes….
93
20
19
307
47,280
Mark Whiting retweeted
@PennPSC's @duncanjwatts and @csspenn invite you to participate in The Commonsense Project.
The common sense project is now live! @DuncanJWatts, @MarkWhiting, @amirhosnakh and @joshnguyen99 from the CSSLab invite you to take a quick survey 📝 and measure your common sense💡 commonsense.seas.upenn.edu You can read more about the project here: css.seas.upenn.edu/commonsen…
5
8
4,341
Mark Whiting retweeted
Check out @CSSPenn's recent blog post too. nitter.cf/csspenn/status/1842254… 8/
The common sense project is now live! @DuncanJWatts, @MarkWhiting, @amirhosnakh and @joshnguyen99 from the CSSLab invite you to take a quick survey 📝 and measure your common sense💡 commonsense.seas.upenn.edu You can read more about the project here: css.seas.upenn.edu/commonsen…
1
2
345
Mark Whiting retweeted
Everybody trivializes common sense because we believe it is self-evident and universal. But is it really? Find out by participating in our Common Sense Project: commonsense.seas.upenn.edu. With @AmirhosNakh, @MarkWhiting, and @DuncanJWatts at @CSSPenn. 1/n
1
8
2
18
3,437
Mark Whiting retweeted
The common sense project is now live! @DuncanJWatts, @MarkWhiting, @amirhosnakh and @joshnguyen99 from the CSSLab invite you to take a quick survey 📝 and measure your common sense💡 commonsense.seas.upenn.edu You can read more about the project here: css.seas.upenn.edu/commonsen…
1
3
2
14
6,485
Using GPS data to inform epidemic modeling, @csspenn Director @DuncanJWatts and Post-Doctoral Researcher Francisco (Paco) Barreras tackle integrating mobility data while addressing privacy concerns to build public trust. bit.ly/3Y5D1b3
6
7
1,372
Say hello to Halide 2.16.1 — our first update for iPhone 16 and iPhone 16 Pro. - Open Halide in a hurry with a click using Camera Control - Our initial release of Process Zero for iPhone 16 and iPhone 16 Pro - Updated UI for the new screens on iPhone 16 Pro More coming soon.
8
16
415
35,702