S. T. Karnick

AI lacks agency, and we have no idea how to develop artificial agency. Without a human prompt, an AI just sits there, inert. It has no intrinsic goals. It has no will. It does only what it’s told to do, just like any other tool. Every AI Doom scenario relies upon AI magically ‘waking up’ and behaving completely autonomously. There’s no reason to think this will ever happen. Maybe it will, someday. There’s no way to know. For now, all we can say is that there’s no indication whatsoever that LLMs can ‘wake up’, even in principle. That uncertainty is permanent, by the way. Alignment is an unanswerable question. We can spend the rest of time aligning AIs, and there would always be the possibility that they’re tricking us, that everything has gone fine up to time-step t but will go off the rails into full SkyNet apocalypse at time-step t+1. This is a central element of the AI Safetyist power play. It isn’t a matter of letting them tinker with the clankers for a few years, at which point they can confidently assert that the AI is safe and we’re good to go. They’re grasping after a permanent position of dominant regulatory influence, the same way climatists desire permanent power over every aspect of the economy: the threat of climate change, according to the climatists, is a permanent threat, necessitating a permanent mobilization of society to forestall the climate emergency, always just over the temporal horizon. In exactly the same way, the AI safteyists want to make themselves a perpetual priesthood, with privileged access to every model, and therefore the capacity to regulate what those models can do, say, depict, calculate. These are apocalyptic sci-fi cults, inventing future catastrophes out of whole cloth, which can only be prevented through tireless, unceasing vigilance in the present. The future catastrophe may or may not be real, it may or may not be plausible, it may or may not ever happen, but the power they seek in the present is all too real. And that power is the point.