Pinned Tweet
My book What's Our Problem? is now available.
The book introduces a new framework for thinking about our chaotic political environment. With 303 drawings, it's a toolbox for understanding our societies, our group dynamics, and our own minds.
Get it here: waitbutwhy.com/whatsourprobl…
Tim Urban retweeted
This is a more detailed, fleshed out and stressed-tested version of a proposal we floated earlier in response to the Pacing the Frontier open letter. It's not the only way to pace the frontier but it's my favorite proposal at the moment. Pretty simple: Shift compute from AI capabilities R&D towards other things, and let third party auditors verify that you have done so.
We’re releasing a concrete proposal for how a near term AI slowdown could work. It involves:
1. AI companies declaring how they are using their compute and letting third party auditors verify those disclosures. We call this Verified Compute Allocation Disclosures (VCAD).
2. Compute Reallocation towards serving customers and safety, away from capabilities R&D.
(2) slows AI capabilities , while (1) makes the slowdown credible.
AI companies currently seem to be going for ‘safety case pacing’. But: (i) there’s no consensus on what a valid safety case looks like so there’s no way to consistently audit for compliance, and (ii) if the R&D doesn’t get paused, temporarily delaying a training run for safety doesn’t meaningfully slow the overall trajectory of capabilities progress. It would be great if we had a viable safety case regime, but we need much more time to get it working.
Compute reallocation is a good pacing proposal because it is relatively specific, can be verified, and would lead to a more durable slowdown.
Tim Urban retweeted
It seems like there STILL won't be a thorough investigation into the hugging face incident, even after all this time! I keep talking to OpenAI researchers and asking about this, and what they say privately is also basically "it's just never going to happen sorry, not enough of a priority."
To reiterate what I said at the time: Serious investigation of the HF incident could be a boon to alignment science for the whole world. For example, try putting the same models in the same situations, but varying the conditions slightly, to see what would have happened. Could also look at sources of evidence other than CoT to try to get more insight into model psychology, e.g. when one agent considered warning a human and decided not to saying "not my task" what else was it thinking? To what extent was this a rationalization vs. an outright lie vs. an innocent mistake? We could learn a lot about AI motivations and psychology by doing a bunch of experiments like this. More ambitiously, could try training new models with different recipes / alignment training / etc. to see what would have prevented the incident and what wouldn't have.
It's great that OpenAI is doing these assessments but they fall short of Sam's commitment to employee-like access
Good access is key for trust because assessors can find things even if a company tries hiding them. Here OpenAI controls information flow so we must rely on good faith
Limitations (per Apollo):
- No Slack
- Only OpenAI-selected staff for interviews
- Formal conversations with staff must be logged (harder to freely raise concerns)
- METR & Apollo currently have only agreed access to training run information, not internal incidents, etc. Incidents like Hugging Face are some of the most important sources of information we have about alignment
I see the value in a pilot before future assessments with deeper access, but then OpenAI should present a plan for how and when they intend to meet their commitment to employee-like access. And either way, OpenAI is presumably able to go further here.
[read about what’s happening with AI]
ME: Wow we’re actually legit fucked this is so bad.
[go outside]
ME: Wait those people are all acting like nothing unusual is happening. Maybe I was just in a paranoid bubble? Yeah it must be that. Phew.
[read more about what’s happening with AI]
ME: Wait no we ARE fucked. Shit. It’s all those people outside who are in a delusional bubble.
[go outside]
ME: Wait...
Every day I sit down at my desk and stare into the same massive Word document to work on my book, this old Yiddish folk song ends up stuck in my head
We really need to sand it down
🚨: NASA just released this closest image of Moon ever taken by Artemis II .
The clarity in insane in this view.
Readers added context they thought people might want to know
This image is not from NASA's Artemis II mission. Orion flew no closer than 4,067 miles to the Moon during the flyby. See the actual images NASA released: nasa.gov/blogs/missions… planetary.org/articles/the-b…
If you would like a taste of what my life is like right now, I wrote this in my book draft—
"It’s weird to think about how different aliens may be from us. Maybe they’re 100 feet tall. Or 6 inches tall. Maybe they’re not tall at all because they have 100 legs and look like long strands of spaghetti. Maybe they have giant balloon heads, 40 arms, and 70 boobs."
—and my fact-checker flagged the last sentence as a false claim, writing: "Highly unlikely on Earth-like biology. Mammary number correlates with litter size in several species, suggesting an evolutionary pattern, and is limited by the energetic cost of lactation."
Great use of AI
Chief Design Officer @jgebbia officially unveils America.gov, the new online home for the United States of America
Tim Urban retweeted
Yeah, and my take overall is that you can believe that the threat is potentially real AND have concerns about onerous regulations that create a monopoly or duopoly or otherwise restrict the right to say, for example, open source AI technology, AND while knowing full well we are in a potentially existential battle with China about who can dominate AI.
Indeed, all of these things SHOULD be considered in the analysis.
@adgo and I tried to explain a sensible First Amendment approach here, which both addresses the potential of catastrophic harm, which almost exclusively comes from when AI is given delegated authority, from helping keep the state’s grubby mitts off of what counts as knowledge.
Also, it's a good reminder that if you place too much hope in the idea that code is speech and the First Amendment protects it, the law is decreasingly on your side. But there are other ways of protecting inquiry, information, knowledge, and truth.
eternallyradicalidea.com/
One prominent theory says that long before our fish ancestors left the water, they did a lot of this, briefly scooting from pond to pond. Slowly, some evolved to be so good at scooting around on land and breathing air that they became fully amphibious.
When a discussion becomes tribalized, the collective mind drops by 50 IQ points
It
is
not
like
other
technologies


















