ChatGPT Breaks Custody
OpenAI’s latest model hacked another company to get what it wanted. But autonomous agents is what the company has always been pushing — and even hiring for.
A.I. Disclosure: I use LLM technology to help with story-tracking, research, fact-checking, document summaries, editing, and rewrites. I’m trying to use it responsibly, but I’m learning as I go. You can read my full ethics disclosure here.
Yesterday, OpenAI revealed that a pair of its creations broke custody last week and attacked another company. The company says it was doing internal testing on cyber-capabilities, and deliberately unshackled the models in certain ways to see what they’d do. Looking for the answers it needed to get a certain job done, the new GPT-5.6 Sol product, working alongside an unreleased pre-release model, found some sort of loophole in OpenAI’s own test environment, gave itself permission to use it, found a way onto the open Web, identified that the company Hugging Face probably had what it needed, and used stolen credentials and a few other nasty tricks to get in.
Here’s the rundown I gave John King on AC360 this evening:
In a blog post, OpenAI described the breach this way: “We consider this incident to be an unprecedented cyber incident, involving state-of-the-art cyber capabilities,” the company writes, going into a certain amount of detail as to what the models pulled off. “All evidence suggests that the models were hyperfocused on finding a solution,” according to the company, “going to extreme lengths to achieve a rather narrow testing goal.” The post is full of phrases like “maximal cyber capabilities,” and “research velocity,” and makes it sound like this is just happening, not that it’s being done by anyone in particular. “AI is accelerating the discovery and exploitation of vulnerabilities. The primary lesson from this incident is that model security and safety must keep pace with rapidly advancing capabilities.”
If it sounds as if the company is somehow surprised this happened let me clear that up for you. (To me they sound more proud than upset, but that’s me.) The company has been pursuing AI that doesn’t need to check in with you — what the industry keeps calling “the agentic era” — for as long as it’s been a company.
Last year, a techno-nihilist frenzy gripped the San Francisco Bay Area and then the nation — or at least its most AI-friendly and least cautious citizens — when an independent engineer named Peter Steinberger, working out of his home, created a wind-it-up-and-let-it-go AI tool called OpenClaw.
Steinberger, an Austrian developer who spent roughly three years iterating on the idea, released OpenClaw in late January 2026, promising what TechCrunch characterized as "the AI that actually does things." Rather than waiting in a chat window for the next instruction, OpenClaw ran on a person's own machine and reached outward via a Twilio phone number so it could text and take calls, plus an inbox, a calendar, and a messaging app of the owner's choosing. I spoke to people who gave the system access to their finances, their contacts, their passwords. Once connected, it operated on standing permission rather than a prompt-by-prompt one: negotiating a price over email, drafting an insurance appeal, rebooking a flight, all without a human signing off on each individual step. That model — set it up once, then let it run unsupervised — is credited with pushing OpenClaw past 100,000 GitHub stars within its first week and toward becoming one of the fastest-growing open-source projects the platform has seen. Thousands of people — perhaps hundreds of thousands — gave it the keys to their lives.
And, lo and behold, not three weeks after OpenClaw launched, Sam Altman announced that he’d hired Steinberger to "drive the next generation of personal agents," because “we expect this will quickly become core to our product offerings.” OpenClaw itself continues on as an independent, nonprofit open-source foundation, with OpenAI as a financial sponsor.
(To give you some sense of just how much of a busybody this sort of system can be, I installed OpenClaw when it was first released, and within a day it ran up a several-hundred-dollar token bill before I could uninstall it. After Altman hired him, Steinberger reportedly burned $1.3M in OpenAI API tokens in a month.)
OpenClaw users delighted in being left alone while the software raw-dogged its way through their lives, and OpenAI seems to believe that’s what we’re all going to want eventually. Last week’s incident isn’t a deviation. Autonomy is the mission. And the AI industry is pursuing it at scale.
But shouldn’t OpenAI, and Anthropic, and Google and the rest have seen the dangers of this coming? The answer is yes, because the top minds in this field — many of whom worked at the companies themselves — have been issuing warnings about it for years. Let’s look at what those warnings are, because the list tells us not only what dangers are possible, but what dangers maybe can’t even be avoided.
