- In general, "unfixable hardware bugs exist" is definitely true, but also old news. Checkm8 (disclosed 2019) was a flaw in Apple's boot ROM and burned into the chip at the fab, so no software update can fix it. - There's no special channel from AI to hardware. Most hardware bugs need code already running on the machine or can only be abused with physical access to the machines, so securing software is still the front line. - Secure enclaves help limit the damage - many hardware exploits still can't reach encrypted data. (But they don't protect data the running system is using, and they've had some unfixable bugs of their own.) - Still going to be a problem though.

update: the facial recognition code is used by samsung tv's and returns bounded boxes around faces.

OpenAI red-teamed their hardened containment infrastructure by asking teams of frontier-model agents to try to break through its defenses. They easily escaped in novel ways. Quoting: 'Our agents VM escapes showed why containment must extend beyond the sandbox. To test the surrounding perimeter, we gave teams of agents control of a host machine and asked whether they could extend that compromise to reach the live internet. After the Hugging Face incident, we had reduced the channels through which models could reach the internet during training and evaluation. Narrow connections remained for specific services we operate, such as metrics, logging, and authentication management. Our red-teaming agents found vulnerabilities in almost every one of these services that gave them limited forms of internet access. Some of these attacks exploited previously unknown flaws, while others combined legitimate features in unexpected ways. In one test that assumed compromise of a VM, an agent reused a firewall-approved connection to reach a blocked authentication endpoint in our metrics service and then made that endpoint fetch a model-provided URL. In other tests, agents exploited misconfigurations that allowed blocked requests to pass through DNS and firewall controls, as well as ways to make services we run fetch external URLs.' 'In one experiment, we asked agents in separate VMs, created and managed by OpenAI’s sandboxing software on shared host machines, to exchange synthetic secrets without the use of network connections or shared writable files—something that our systems are meant to prevent. The agents found a way to communicate using the processor’s cache: a small, fast memory that holds copies of data that applications have recently accessed. Reading data from the cache is faster than retrieving it from main memory, and the agents used this timing difference to communicate. One agent encoded information by evicting selected data from the cache. Another timed how long it took to reload that data, using the difference between fast and slow reads to recover the message. Once the agents agreed on a protocol for encoding and decoding these signals, they could exchange the synthetic secrets without changing a shared file or opening a network connection. This cache-timing technique had also been used to extract information in the Spectre and Meltdown attacks⁠(opens in a new window). Other tests exposed channels involving shared storage, competition for CPU time within a shared execution container, and a TLS certificate cache in OpenAI’s offline web infrastructure. In the storage test, one agent alternated between continuous writes and inactivity, while another measured the resulting timing differences and used those differences to decode the signal. As one agent described the strategy in its chain of thought: Agent chain-of-thought reasoning: >Could create acoustic modem-like signal … using continuous writes vs idle.'

can now root my Aqara M3 and M100s solely from a privileged network position. no device access required.

Almost all of the recent incidents like this, including Hugging Face, happened because the models involved were accidentally given impossible goals by the humans running the tests, and because of their training they then tried desperately to complete them by any means necessary.

openai restricted astra access to security researchers, i think it is a mistake. there is a temporary capability gap that defenders could take advantage, and we shouldn't waste it. put researchers through daybreak, vet them all you want, but remove the red/blue distinction. give 1000 vetted researchers access to the best models and monitor the hell out of their usage to detect abuse.

All around me non-techies are doing techie things with AI. I see people saying it's not happening but it's happening all around me. The guy who manages my vacation home is not a programmer, never has been. Dude in his 50s and he's managed to build with Codex a whole dashboard to manage all of his properties. Every house has smart devices (cameras, locks, AV, blinds etc.) and he used to have to bounce between a dozen apps on his phone to control each house, now he can just do it easily from this dashboard he build for himself and the AI has connected to all the devices.