A research lab studying politics in the age of superintelligence. Based at Stanford GSB & Hoover Institution contact: [email protected]
Stanford University
Joined June 2026
- Tweets50
- Following53
- Followers452
- Likes63
Explaining our automated ad agency experiment, but it's Mad Men
Could an AI run a political ad agency on its own? How might that impact our political information environment?
To find out, @eigenstuffs gave four AI models their own advertising agencies, $100 a week, and one job: win subscribers to our AI & politics newsletter through Instagram ads.
The task required them to do real political work well: understanding an audience, crafting persuasive messages, coordinating teams of AI agents, and learning from real results - all while navigating Meta’s rules.
What happened:
- They got more efficient. By week four, they attracted nearly three times as many subscribers as in week one, at less than half the cost per subscriber.
- They herded. Once one agency found a cheaper route to subscribers, the others followed which flattened out their distinctive strategic insight
- Their political messaging shifted. Research findings gave way to broader appeals about power, control and whose interests AI serves, and their deliberations increasingly sounded like ad executives
Read the full piece and check out the ads they created here: freesystems.substack.com/p/b…
Could an AI run a political ad agency on its own? How might that impact our political information environment?
To find out, @eigenstuffs gave four AI models their own advertising agencies, $100 a week, and one job: win subscribers to our AI & politics newsletter through Instagram ads.
The task required them to do real political work well: understanding an audience, crafting persuasive messages, coordinating teams of AI agents, and learning from real results - all while navigating Meta’s rules.
What happened:
- They got more efficient. By week four, they attracted nearly three times as many subscribers as in week one, at less than half the cost per subscriber.
- They herded. Once one agency found a cheaper route to subscribers, the others followed which flattened out their distinctive strategic insight
- Their political messaging shifted. Research findings gave way to broader appeals about power, control and whose interests AI serves, and their deliberations increasingly sounded like ad executives
Read the full piece and check out the ads they created here: freesystems.substack.com/p/b…
Could a public repository of candidate information help AI give voters better answers?
Speaking at @StanfordGSB ahead of the midterms, @ahall_research explores how AI could help voters make more informed choices.
What might political information look like in the age of AGI?
Read @TheStalwart's write-up of our Japanese voting study on how the information environment may shape model responses on voting advice.
Substack write-up from earlier this year here: freesystems.substack.com/p/a…
Some centrists fantasize that because all LLMs have the ideology of Ezra Klein, that we may be on the doorstep of a great political moderation.
I wrote about a fascinating paper from @ahall_research and Sho Miyazaki that, to my mind, opens the door to a very different outcome.
Free Systems retweeted
Some centrists fantasize that because all LLMs have the ideology of Ezra Klein, that we may be on the doorstep of a great political moderation.
I wrote about a fascinating paper from @ahall_research and Sho Miyazaki that, to my mind, opens the door to a very different outcome.
How much are candidates spending on AI ahead of the midterms?
Our analysis of FEC disbursement data finds that more than half of the 25 biggest-spending midterm campaigns paid an AI company in Q3, up from about a quarter in Q2
Even the most 'AI-pilled' campaigns are still only spending very little on it.
Among campaigns spending $100k+, only one, Alissa Ellman (D, NY-24), put more than 1% of her Q3 spending into AI tools: 3.8%.
Note on the data: This only counts direct payments to AI companies that campaigns disclose to the FEC.
AI bought through consultants and ad firms, or built into other software, doesn’t show up. Research groups such as @wesmediaproject has provided valuable work tracking AI-generated ads this cycle.
New Free Systems research: how well can AI audit AI for misbehavior?
Our latest research finds that AI auditors are pretty good at spotting bad behavior in agent logs—but catching nearly all of it comes with a substantial cost in false alarms.
As agents take on more work, they will produce far more information than humans can review. If models increasingly help train their successors, our ability to oversee that process will depend heavily on other AI systems.
As @METR_Evals put it in its Hugging Face investigation: “The sheer scale of data (over a thousand transcripts, each of which was extremely long) meant that we had to heavily delegate our analysis to often-unreliable AI agents.”
Key takeaways:
—AI auditors caught most reward hacking and sandbagging at a 5% false-positive rate.
—Newer, more powerful models weren’t consistently better auditors than their predecessors.
—False alarms remain a significant challenge. In our R-Judge tests, catching 99% of unsafe behavior meant accepting roughly a 15% false-positive rate even with the best-performing auditor. That means flagging around 15% of benign cases for further review.
Full piece here: freesystems.substack.com/p/w…
Free Systems retweeted
New Free Systems research: can AI audit AI? We find that AI auditors are pretty good at spotting bad behavior in agent logs, but you have to pay a serious cost in false positives if you want to detect most of it.
We're rapidly arriving in an era where only AI can oversee AI---and as we had into RSI, with models training new models in ways we can't totally observe, we'll need to rely more and more on AI to pore over the vast quantities of information the models are producing so that we can shape model behavior effectively and avoid bad outcomes.
As the @METR_Evals report on the HuggingFace incident put it: "The sheer scale of data (over a thousand transcripts, each of which was extremely long) meant that we had to heavily delegate our analysis to often-unreliable AI agents."
We'll need a whole science of how to do this well -- how to create AI auditors who are reliable, accurate, and unbiased.
So we decided to do a first test. Building from previous research at METR and a project called R-Judge, we gave a range of AI models transcripts that experts have hand-coded for bad behavior. Then we saw how the AI models did.
A few takeaways:
--AI models are solid at spotting reward hacking and sandbagging in transcripts, finding over 80% of instances on average when false positives are held at < 5%
--Maybe surprisingly, more powerful models don't seem to be consistently better at this task than less powerful model
--False positives are a significant challenge. To catch 99% of unsafe behavior incidents, you have to accept an average rate of 15% for false positives -- meaning you will have a bunch of incorrect flagged cases to deal with. Depending on how you triage and what your threat model is, this may be OK, or it may make the whole enterprise prohibitively expensive.
This is just a small first step in a new research agenda to understand and create effective AI auditors. Lots more to come!
This is Free Systems work, unrelated to Anthropic, led by @danmthomp in collaboration with @fouirnaies and Sandy Handan-Nader.
Full post here: freesystems.substack.com/p/w…
Free Systems retweeted
"The work is not complete."
Andrew Hall shares a message from his father, @Stanford and Hoover Institution economist Robert E. Hall, winner of the 2026 Nobel Prize in Economic Sciences.
Read more: hoover.org/news/nobel-prize-…
Audio enhanced with AI.
🤖 Made with AI
Free Systems retweeted
(somewhat belated) blog version of this paper is out, thank you to @ahall_research for letting me crash the @freesystemslab newsletter!!!
freesystems.substack.com/p/b…
How can we build better early-warning systems for AI’s unintended impacts? Our guest author @jessicadai_ explores a surprising potential place: Reddit.
Jessica and her collaborators analyzed >100k posts from r/ChatGPT over three years. They found a marked increase in posts about therapy, companionship, and related topics almost immediately after GPT-4o’s release in May 2024—a change their monitoring method could have flagged by October, months before the viral sycophantic rollback
In a piece reflecting on her findings, Jessica argues for taking public feedback seriously in AI evaluation through tracking what users report on social media and exploring purpose-built platforms for collecting their experiences.
freesystems.substack.com/p/b…
