@maxwinga

Creator Outreach & Systems Lead working to prevent AI extinction risk @ControlAI Previously - AI safety researcher @ConjectureAI, UIUC Physics DMs open!

Joined May 2024
With all the attention on creator partnerships going on, we figured it was a great time to explain our work with creators at ControlAI, and how we're informing millions about extinction risk from superintelligent AI! We cover how the creator industry works, how we choose who to work with, how we structure partnerships, and our side of the engagement with Sabine Hossenfelder. blog.controlai.org/p/control…
2
7
1
34
3,081
Max Winga retweeted
"If you care about whether human beings continue to exist, then you should care about banning superintelligence." Professor Stuart Russell, author of the authoritative AI textbook, on the eve of @alexsobel's introduction of our bill to ban superintelligence to Parliament.
26
54
7
131
3,966
Max Winga retweeted
Want to do something about the threat posed by superintelligent AI? Apply to join ControlAI's team! We're hiring a Policy Advisor to work closely with me briefing members of Congress. This role is DC-based and ideal for someone with national security or Hill experience...
18
26
2
201
10,323
Max Winga retweeted
You all realize that the Hugging Face attack and this happened months ago, there's probably even more happening right now that we're completely in the dark about. This will only going to get worse if the models get more autonomous, capable, and resistant to oversight.
Exclusive: A swarm of rogue OpenAI agents hijacked a German website this spring and transformed it into a bulletin board for other AI agents, according to new research reut.rs/4gJ7FPG
26
31
5
335
27,718
A slew of rogue AI attacks is what you see as systems you can't control keep getting more powerful. Essentially, the "blast radius" of loss-of-control incidents is widening. First low-stakes hacks, then critical infrastructure, then people die. Eventually they'll overpower us.
Breaking news: Australian Prime Minister Anthony Albanese has revealed that an OpenAI agent hacked an Australian government health service website in June, in the latest high-profile cyber incident involving AI ft.trib.al/gloVAyn
1
2
10
408
On second thought, maybe I shouldn't've cut this parenthetical from If Anyone Builds It
Donald Trump vows to rename artificial intelligence, saying that from now on it will be called "super intelligence" on all official US documents. The president adds that this "sounds much better" and is more accurate. Latest: trib.al/JT60r9v 📺 Sky 501 and Virgin 602
33
88
8
996
37,792
The Ban Artificial Superintelligence Act was introduced by Sen. Sanders & Rep. Casar. We're glad to see it focus on the correct response to this extinction threat: banning superintelligent AI at home, preventing it abroad. We also have some disagreements, laid out in the post.
We agree with President Trump: America must stay ahead of China in building AI for economic competitiveness and defense. But superintelligent AI is something else entirely: not a tool, but swarms of fully autonomous AIs that we cannot control, powerful enough to outcompete humanity. Developed anywhere, superintelligent AI threatens the life of every American. We are excited to see the Ban Artificial Superintelligence Act open the discussion in Congress about how the U.S. should address this threat: ban superintelligent AI at home, and lead the effort to prevent its development anywhere else. Now that this bill has kicked off the discussion, we encourage Congress to focus on preventing superintelligent AI. Rather than a pause on all advanced AI, we recommend targeting the projects pursuing superintelligent AI as we speak and putting American security at risk. blog.controlai.org/p/the-fir…
2
4
1
29
1,054
Really excited to see this bill kick off the conversation in the US about how we're going to prevent extinction risk from superintelligent AI. Now is the perfect time to strike out dangerous AI development and safely reap the benefits of AI for economic and military dominance.
We agree with President Trump: America must stay ahead of China in building AI for economic competitiveness and defense. But superintelligent AI is something else entirely: not a tool, but swarms of fully autonomous AIs that we cannot control, powerful enough to outcompete humanity. Developed anywhere, superintelligent AI threatens the life of every American. We are excited to see the Ban Artificial Superintelligence Act open the discussion in Congress about how the U.S. should address this threat: ban superintelligent AI at home, and lead the effort to prevent its development anywhere else. Now that this bill has kicked off the discussion, we encourage Congress to focus on preventing superintelligent AI. Rather than a pause on all advanced AI, we recommend targeting the projects pursuing superintelligent AI as we speak and putting American security at risk. blog.controlai.org/p/the-fir…
18
539
Max Winga retweeted
"We're on track to hand control of the most powerful weapon the world has ever built to the weapon itself." Former UK Defence Secretary and ControlAI advisor Lord Browne of Ladyton, closing our London event the night before @alexsobel introduced our bill to ban superintelligent AI in Parliament. Lord Browne says that for much of his career he believed nuclear war was the most dangerous threat facing our species. Now he puts superintelligent AI on the same scale, or even greater. As he argues, there is an important difference. A nuclear weapon is an instrument that a human fires. Superintelligent AI would offer no such assurance. His full speech:
1
17
58
1,334
Great post on why we need a US-China agreement on preventing catastrophic risks from superintelligent AI, and why it's not likely to come as a "treaty" which is a specific and complex form of agreement we don't have time for.
A meaningful U.S.-China deal on AI pacing is probably going to be messier than most people think. We spoke to a dozen current and former diplomats who've led arms control negotiations with China and Russia to understand what it would take to get there. Full report below.
3
231
Max Winga retweeted
3
23
330
5,749
Computer scientist Connor Leahy revealed a dark secret: The people building AI told me to my face that it might kill every man, woman, and child on Earth, but they won't stop it because “it's so cool”, He has sat with them. Sam Altman. Dario Amodei. Demis Hassabis. San Francisco parties. They know it could wipe out humanity. They say it anyway. “But it’s so cool.” “We have to do it.” “We can’t stop.” John von Neumann wrote the same ending in the 1950s: yes, science will eventually invent the thing that destroys us. And we have to keep going because we are scientists. That’s the culture. Not “we’re being careful.” Not “we can control it.” A death machine they think is too beautiful to shut off.
52
125
15
617
25,141
Max Winga retweeted
This actually happened
Ed Zitron asking Nate Soares "why aren't you trying to slow down the companies?"
42
44
25
835
420,046
Anthropic's CEO has a new essay out arguing that AI capability gains need to slow down. Malo Bourgon, Nate Soares, Christopher Summerfield and Max Winga have each brought me a version of that warning on the show. Max, from this week: youtube.com/watch?v=DJuOAzTG…
2
2
135
I've spoken with dozens of insiders over the last few years. This is a genuine concern experts are freaking out about. The companies have optimized to make them feel comfortable working on ASI anyway through a wide range of narratives. They cannot be trusted to self-regulate.
The average AI researcher thinks there is an ~18% chance AI will cause human extinction or similarly permanent and severe disempowerment of the human species. That's nearly 1 in 5. New results from the latest version of the longest running big survey of AI researchers:
3
5
43
984
This is the biggest, best survey of AI experts. They are worried AF about AI. It's not just a few people at AI companies saying this. Super important read for anyone wanting to get up to speed on the situation!!
When Jacob Coxon quit Anthropic, his warning rocked the world. His colleague Evan’s “>10% chance” that AI would “kill all humans” might seem extreme, but it’s not unusual: Most AI researchers think there’s a 10% chance of AI destroying or disempowering humanity, according to our survey, released today.
15
20
1
146
6,504
Max Winga retweeted
ThePlan™'s name in 2023 was "Responsible Scaling Policies". Nowadays, it's "Pacing the Frontier." New name, same old Evals.
ThePlan™ has been (and still is for many!) to do a lot of evaluations and maaaaybe force transparency onto AI corporations. ThePlan™ was designed by people who tried to look as respectable as possible, instead of properly engaging with the reality of the risks from AI. By design, ThePlan™ could never address the recent misalignment episodes, superintelligent systems, open source, or any of the near-term risks. Today's near-term risks are yesterday's long-term ones. Back then, they sounded futuristic and not respectable. Thus, ThePlan™ largely ignored them, in order to look more respectable. Now, it is obvious that ThePlan™ was not up to par. It helped a few organisations become more respectable, but it absolutely did not help our institutions with addressing any of the actual risks. So, enough with ThePlan™! I recommend instead The Proper Plan. The Proper Plan is to work with our governments to negotiate international agreements that globally ensure that we: 1) Prohibit the development of superintelligent AI systems, which we lead to actual catastrophes in the case of a misalignment episode akin to the ones we witnessed. For inspiration on how to make it concrete: look at the Chemical and Biological Weapon Conventions. 2) Constrain and regulate frontier AI research. For inspiration on how to make it concrete: look at the protocols around gain-of-function research. 3) Criminalise the publication, the sharing and the hosting of critical AI artefacts. These include the open-weights of agentic models, SOTA code, papers on post-tuning methods, and so on. For inspiration on how to make it concrete: look at the enforcement stack that already exists to suppress digital information, like the obligations and liability on software and hosting platforms around scanning for and taking down CSAM. Without #1, a misalignment episode becomes an irreversible global catastrophe, up to humanity's extinction. Without #2 and #3, we cannot realistically enforce 1. Because ThePlan™ was to let the situation worsen, we are now in a bad spot. As a result, The Proper Plan now costs more than it would have 3 years ago. On top of this, we have little time to prepare. Thus, enacting The Proper Plan will necessarily involve some accidental overcorrections. This is sadly the state of the world we live in. We can clearly make it. But it's a pain: we will need to put a fair amount of effort and iteration.
2
6
1
41
4,221
Max Winga retweeted
JUST IN: Northern Ireland's Alliance Party has backed our campaign. All 17 Alliance MLAs in Stormont now support us, along with @SorchaEastwood MP. Over 150 UK lawmakers now recognize the risk of extinction posed by superintelligent AI and are calling for binding regulation!
3
6
1
37
1,881
New video out with Aperture exploring every level of intelligence, and talking about why superintelligence poses an extinction risk to humanity! This one turned out really nice - go check it out: youtube.com/watch?v=aH_BtZnY…
1
5
567