freg 🍉🇱🇧

I love that people want to assume that I formed my opinions on the viability of current generation open-weight LLMs for enabling low-skilled criminals to find useful zero-day exploits, based on having just never used an LLM at all. The first thing I did when Anthropic added me to their CVP was build an autonomous zero-day exploitation pipeline. I've since backported it to use open-weight models because the token cost was insane, so I have a fairly good feel for both current frontier and open-weight model capabilities. I'm well aware of what LLMs are capable of doing, and that they can, with the right steering or harness find zero-day exploits. Where we drift off course is with the claim that low-skilled criminals can just pick up something like GLM and find viable zero-day exploits. I think that: a) More than 90% of CVEs are functionally useless for threat actors. Threat actors have a very specific operational needs and "random issue in random software" does not fall into their scope. b) Many (quite possibly even the majority of) critical rated CVEs are not exploitable under real-world conditions in a way that is useful to threat actors. There's entire teams in China dedicated to n-day exploitation and watching them will give you a good feel for what percent of critical RCEs are actually useful. c) Threat actor with no exploit development background and minimal technical skills, even with access to a completely unrestricted open-weight LLMs, would struggle or completely fail to find useful zero-day exploits. d) People with very little experience in software exploitation, Large Language Models, and CTI, are watching professional vulnerability researchers working with next-gen frontier models and ending up with a wildly unrealistic beliefs of just how "easy" what they are doing is. e) There is an extremely noticeable lack of both zero-day exploitation and use of novel (non-public) n-day exploits by all threat actors, especially unskilled ones, and proponents of the "anyone can just pick up GLM and find useful zero-day exploits" theory continually refuse to even acknowledge that fact, let alone attempt to explain it. I'm not coming at this from the position of some dude who just logs on to social media to says uninformed shit. I'm coming at this from the perspective of a researcher who has spent thousands of hours since LLMs came out researching their capabilities, and has also spent their entire career tracking threat actors and their capabilities. I have a mountain of real-world data, because the low-skilled AI-powered threat actors have a high tendency to accidentally infect themselves with their own malware. This often results in them exfiltrate their entire hard disk, along with all their AI prompt logs, to a vibe-coded C2 with client-side authentication (the admin credentials are hardcoded into the javascript or something equally stupid, so anyone can log in and download it). My entire career I've been very pro "degrees and certs don't matter, everyone is welcome, and the best way to gain experience is hands-on practical work". But when I have some dude whose only experience is essentially just "downloaded abliterated GLM and scared himself with it", I feel like it's necessary to draw a line somewhere. It's not just the bad opinions, everyone has bad opinions on something, I have plenty. It's that resources are actively being diverted away from addressing real and pressing issues in cybersecurity, as a result of FUD being spread about imaginary GLM-powered zero-day wielding scriptkiddies.