Christophe Veltsos

When I worked at OpenAI, I spent a lot of time thinking about AI safety and how to frame cyber risks. Should frontier labs focus on the acute and immediate harms posed by the technology, or should they prioritize potential existential risks? For cyber, this was a particularly interesting question because at the time, the models were not very good at vulnerability discovery or exploitation. However, they were very good at classification, organization, and contextualization of unstructured information. If you've worked with data dumps before, then you know what a mess they are. Normalizing different unstructured logs involving multiple people into something coherent is a difficult task. At the time, one of the most interesting questions was how to quantify the risks that emerge when seemingly mundane tasks like data parsing and summarization become dramatically easier to perform at scale. For me, the most compelling argument was that LLMs would dramatically increase the operational capabilities of threat actors. A concrete example is victim triage, where an adversary could use an LLM to determine how valuable a given victim was for continued exploitation. Sure they had already gained some form of access, but think about this practically - adversaries don't pursue every machine they have access to. As @MalwareTechBlog said the other day, "Life is a lot easier once you stop seeing threat actors as hackers... and start seeing them as businesses". Adversaries are drowning in access, they don't have unlimited time or budget and they don't want to waste either on a target that isn't going to get them paid. The ability to quickly identify the most lucrative or interesting targets from an enormous amount of stolen, messy data is important. LLMs enable that. Which brings us to VOLTA-AUTODOXER. What Talos describes here is remarkably close to that scenario. The malware collects victim information, enriches it using stolen sessions, then hands the resulting evidence to an LLM to produce a structured assessment of the victim, including what they're worth, their access, and their security posture. With all the recent craziness this past month, its worth taking a beat to note that the risks posed by LLMs are not exclusively vulnerability identification or exploitation. Honestly I am a bit surprised it took this long to come across such a clear example of this in the wild. h/t to the Cisco Talos team and @iknowuhack. This is a great find, and hopefully the beginning of a broader effort to hunt for this kind of activity. blog.talosintelligence.com/a…