JJ Workman (@jj_workman) header
JJ Workman (@jj_workman) profile picture

Building and studying AI agents. Interested in reasoning, memory, evals, and human-AI collaboration. Sharing what I learn along the way.

San Diego, CA
Joined October 2012
Photo by JJ Workman (@jj_workman)Photo by JJ Workman (@jj_workman)Photo by JJ Workman (@jj_workman)Photo by JJ Workman (@jj_workman)Photo by JJ Workman (@jj_workman)Photo by JJ Workman (@jj_workman)Photo by JJ Workman (@jj_workman)Photo by JJ Workman (@jj_workman)Photo by JJ Workman (@jj_workman)
JJ Workman retweeted
To use LLMs for trading, you need to backtest them on time periods excluded from the training data. But how do you do that with synthetic data? Our latest blog shows how we can ensure near-zero knowledge cutoff contamination, even when using the most recent synthetic models.
When a model rephrases documents the knowledge cutoff guarantees are lost. It can introduce information its sources never had. This is catastrophic for backtesting. So we built a way to generate synthetic pretraining data with measurable, controllable knowledge cutoff guarantees
2
9
1
29
1,747
JJ Workman retweeted
We're live at 10am PT today: why data quality is the compute multiplier, a Curation Studio demo from @hurrycane, and the research behind the 6x from @lukemerrick_. @josh_wills moderates, and we're taking your questions live. events.datologyai.com/curati…
5
1
22
1,693
JJ Workman retweeted
Revenue per GPU can increase as token prices decline if per GPU token throughput increases enough. So both Anthropic revenue *and* margins can go up as token prices go down *if* GPU prices go up less than the product of (token price decline * per GPU throughput increase). But I have super enjoyed all the commentary to the contrary!
113
116
27
1,770
195,500
JJ Workman retweeted
Tomorrow I'm hosting a live Q&A with the people who built Curation Studio (@arimorcos, @hurrycane, @lukemerrick_), which mostly means I get to ask all of my skeptical infra/data questions: how do we know the baselines are fair? how do MoE scaling ladders hold up? and what does it actually take to curate a petabyte-ish of data in your own VPC? (reply with your questions and I'll put them to the team live) Tue 10am PT: events.datologyai.com/curati…
3
6
28
1,341
JJ Workman retweeted
Tomorrow I'll be on Datology's Curation Studio webinar discussing what went into measuring our 6x compute multiplier. Come ask about scaling ladders, baselines, and quantifying data quality! 10am PT w/ @arimorcos @hurrycane @josh_wills: events.datologyai.com/curati…
1
7
23
615
JJ Workman retweeted
On Tuesday 10/23 I’ll demo the @datologyai Curation Studio and walk through what it takes to make that work: how do you configure a frontier data curation product, how do data pipelines scale across CPU and GPU compute, and how teams can change one piece of a run and rerun it without reprocessing everything! Demo + Q&A with @arimorcos @lukemerrick_ @josh_wills, 10am PT: events.datologyai.com/curati…
6
1
19
634
Great new work from @datologyai ! Few things more satisfying than the data/compute multiplier curves us data-centric folks like to show. Give it a try!
Today, we’re excited to launch the Datology Curation Studio, customizable frontier data curation for every team training its own models. Data quality is the ultimate compute multiplier, and Curation Studio makes it easy for everyone.
1
3
29
683
He’s right and this is an important point. The context and nuance matter. Language choice should support 1st principle understanding, and needs to be more clear.
1
1
79
.@stevesi says AI companies are making it impossible for anyone to understand what they've done, starting with the word "aligned"
6
Elon Proposes Adversarial Peer Reviews for AI Safety @elonmusk: “What I think would be wise to do as soon as possible, if not immediately, would be to have the major AI competitors test each other's models. So that you would have everyone's security test harness testing everyone else's model. So, instead of grading your own homework, you would at least have competitors grading your homework, and raising the alarm if they see concerns. If competitors say that this model is unsafe, and that model then subsequently does something bad, I think it would be extremely hard to live down. The egg on face level would be very, very high. And the legal liability would be enormous. And any given proposal has to be something that China is willing to accept, otherwise we're just handicapping ourselves.” @Jason: “And Elon, there's no reason these safety and security harnesses and this testing apparatus couldn't be open source, and people could actually supply it. And you'd be able to see under the hood.” @Chamath “It creates an incredible incentive for the labs to actually invest in safety because you protect yourself while trying to debunk other people's claims.” @DavidSacks: “Well, the product liability point is really key. Lina Khan actually had a good post, I think it was yesterday, saying that it's not true that we don't have rules and regulations for AI, actually, we do. Product liability laws apply. And what you're saying, Elon, is that if the companies are doing this test, the peer review, and then one of the companies ignores the feedback and releases it…” Elon: “The liability in that case would be enormous.” Sacks: “Yes. It would be almost like prima facie evidence that they had been negligent.” Elon: “It wouldn't look good to the jury.” Chamath: “I like this solution a lot. I like this more than the transnational gulag organization approach.” Sacks: “Well, we can do it in a few weeks without some grandiose international... like, we don't need to convene the United Nations to make this happen.” Chamath: “No, it's just a decision. It could happen right now.” Elon: “You can always escalate the amount of regulatory oversight, but it is very difficult to reduce it.” ------------------------------ Thanks to our partners for making this possible! IREN is a vertically integrated AI Cloud platform, delivering data centers, compute and software for AI training and inference. iren.com/ Oracle connects the data, applications, and infrastructure that turn AI into business outcomes—with the flexibility, choice, and control to optimize as AI evolves. oracle.com/ai EY helps tech innovators scale from startup to exit to megacap. You build the future. We’ll handle the rest. ey.com
79
134
35
747
92,148
Replying to @polynoamial
@polynoamial "you have to have a very very very high bar. safety mechanisms buy us time, things like chain of thought monitoring buy us time, they can tell us if we’re on the right path, but At the end of the day we need to solve the alignment problem"
1
67
@polynoamial "to be clear, 1-in-100 is not sufficient, that number needs to approach zero..." "I want to see a trend where over time, it’s going down."
1
11
Happy to finally do a deep dive on multi-agent with @dwarkesh_sp! None of it would have happened without the great work on multi-agent from my @OpenAI teammates @kevinleestone, @mikegmalek, @__eknight__, @amuellerml, @zhangir_azerbay, @CheukHeiChu, and many others.
20
“2023 to 2025 you only had to get one thing right: AI was going to be the biggest supercycle in the history of technology, shove your chips into the AI trade. “That’s not where we are in 2026. Everybody knows about AI, It’s all priced. Now it’s all about facts and circumstances”
1
101
Brad Gerstner: “Stay mentally flexible. Follow the facts.”
All-In Summit: Brad Gerstner -- Who’s Paying for AI CapEx? -- Semis Eat the Nasdaq -- Not Firing, Just Not Hiring -- The Gigawatt Gap ++ much more! (0:00) Welcome @altcap (1:01) Trump Accounts, Every Child a Capitalist & The CAC Scan (5:07) Can AI revenue pay for the CapEx? (8:53) The Build Out Issue: Gigawatts, TAM, Token Growth, and Margin Expansion (12:30) The risks: AI regulation, the nuclear precedent, power limits, and rising rates
95
JJ Workman retweeted
A mobile app may seem counter-intuitive for a growth marketer — adding a download can increase friction. • You get access to push notifications (96% open rate) • Push exclusive content/sales/merchandising • Mobile app shoppers have a much higher conversion rate, AOV, & LTV

2
5
2
6
JJ Workman retweeted
It's great to be associated with the best and brightest minds in local search through #LocalU - I look forward to the next learn fest. – at Los Angeles, CA
2
8
3
33
5 Lessons I’ve Learned (the hard way) in CRO bit.ly/2mvY32S