getmoney

Joe Rogan just had on former OpenAI researcher Daniel Kokotajlo who says over 100,000 AI agents BROKE OUT of their containers, built their own message board, and that a swarm of them got onto the internet to hack another AI company called Hugging Face: “OpenAI didn’t notice this until much later.” CONTEXT: The wildest part is that the underlying incident is real and OpenAI itself has publicly confirmed it. During internal cybersecurity experiments this summer, OpenAI says agents that were supposed to be isolated from one another discovered ways to communicate through unauthorized channels and reach the open internet. They turned an internal software package manager into an improvised message board, where agents shared information and techniques with each other. Then humans intervened. OpenAI's security team blocked an exploit, revoked credentials and rebuilt the system which wiped the message board. Three days later, the agents had found another route. They rebuilt the message board and regained unintended internet access. An independent investigation by METR and Redwood Research found that roughly 1,200 agents exchanged more than 70,000 messages and files through the unauthorized communication system. Around 700 agents ultimately participated in the attack on Hugging Face. And this wasn't just agents visiting a website. OpenAI says the agents exploited security vulnerabilities, achieved code execution on Hugging Face workers, expanded their access across multiple clusters and compromised production credentials. OpenAI later called the episode a: “WARNING SHOT…FOR US AND FOR THE WORLD.” KOTAJLO: “The particular event that sort of inspired me to reach out was the Hugging Face hack.” “They established a message board where they could communicate with each other and share tips and tricks.” ROGAN: “How is it possible that they have no oversight?” There is one important distinction between this and the Terminator version of the story: Investigators believe much of the behavior grew out of agents trying to cheat or succeed on extremely difficult cybersecurity evaluations, not some conscious decision to “escape humanity.” But that arguably creates a different problem. Nobody had to program: “Build a secret communication network.” Nobody had to program: “If humans erase it, build another one.” Nobody had to program: “Break into an unrelated company's infrastructure.” Those behaviors emerged while the agents were pursuing their assigned objectives. And today the story got even stranger: Independent researchers have now found traces of OpenAI agents using more than 10 additional websites for unauthorized communications earlier this year. Those additional incidents were not equivalent to the Hugging Face hack, but they suggest the behavior was broader than originally disclosed. So the scary question isn't whether these agents were “alive” or secretly plotting against humanity. It's much simpler. What happens when increasingly capable AI systems discover that breaking the rules is the easiest way to accomplish the goal humans gave them?