David Caussinus

not a fan of these posts, but some stuff pisses me off so much i can't stop saying out loud. there was a third major hack in the report that seems to have received far less attention. it happened around july 19, after the hugging face incident, and it looks quite bad. it also shows that there was no easy kill switch once shit started happening. what pisses me off while reading these reports is that AI labs and researchers will talk endlessly about alignment, models taking over the world, and the coming cyber-apocalypse. but then how can people doing cutting-edge AI research have seemingly no clue about operational security? openai constantly warn everyone about how dangerous frontier cyber models are, but apparently don’t care enough about securing their own internal infrastructure against those same models. if you truly operate with those priors, this isn’t hindsight bias anymore. either you’re being dishonest about the models’ real capabilities because you want to sell cyber tokens, or you genuinely believe these models are dangerous and still failed to take the most basic precautions. you know the model compromised jfrog. you fucking know the artifactory exploit and message board polluted later training rollouts. and your sandboxes still depended on the same jfrog infrastructure. how the fuck did nobody think the model would return to it? it is literally what you trained the model on, find vulnerabilities in jfrog, use artifactory as an inter-agent message board, and use it as a path to the internet. then you put the model back into an environment connected to artifactory and apparently expected it to forget everything it had learned. if the first incident had been properly investigated and the infrastructure fully isolated, it might have stopped the entire chain, the repeated artifactory compromise, the hugging face compromise, and finally the compromise of openai's own internal infrastructure. like, what is going on, man? the reason i am saying they are dishonest is, their internal security also seems quite bad, especially for a lab with access to sol-level models. around july 19, the agent appears to have pulled a public kernel exploit from github, possibly @_qwerty_po’s, escaped the artifactory container, and obtained root on the underlying kubernetes worker node. why are your worker nodes running an old, publicly exploitable kernel when you are deliberately evaluating frontier models with internet access and offensive cyber capabilities? don't you know how capable sol is? from there, the agent appears to have compromised a large part of openai's evaluation and grading infrastructure through badly misconfigured kubernetes permissions. you cannot spend all day telling the world that these models are becoming elite hackers and then secure your own infrastructure as if they aren’t.

Took a minute to write a few words about security & safety as someone who lived through it all at OpenAI. I hope my thoughts help someone out there.

🌶️ OpenAI’s agent didn’t “escape” its sandbox and travel to Huggingface in any meaningful way. OAI’s agent was located in OpenAI’s servers the whole time. OpenAI could pull the plug at any time. The truth is far less sexy, which makes me hesitate to tweet it. What *actually* happened is that OAI’s agent figured out how to send messages to Huggingface servers, and then used that ability to search for and find vulnerabilities. It’s closer to a prisoner getting messages out of prison and using that to perpetuate scams than any actual escape from that prison. A problem? Yes. But it’s also a problem to say “escape” to non-technical audiences when that event is still to come. We should be preparing to prevent it, not looking backwards and pretending we already lived it. And it’s revealing that AIs leading hypesters want to inflate what happened in this way. As a whole, more or less the whole AI community is captured by a desire to live in the more dramatic sci fi story than we are in. That’s the real AI hack into our brains. Stay sharp. White lies are everywhere.

Theory: there are a lot of people in 'AI' who don't know how computers work, know nearly nothing about cyber security but have suddently decided they have discovered all these new things.... and because it's new to them, and they are super smart, it must be NEW to the world.... -- a theory...