Ian Livingstone (@ianlivingstone) CEO of @KeycardAI says the biggest thing standing between today’s AI agents & true autonomy actually isn't intelligence.. “We have apparently AGI, but the real thing that stops us from having autonomous agents that actually work, like a self-driving car, is identity & access.” “Our systems weren't built for agents that are non-deterministic actors.”

Any pursuit of superintelligence has to be grounded in the core principle that if the AI we build is not helping humanity and under human control, it's not worth pursuing. We also need to accelerate and spread the benefits of AI, such that they are diffused broadly across countries, communities, and companies. This requires a frontier ecosystem in which both closed and open-source models can thrive. And for firms, it’s imperative that they retain full control over their unique and tacit knowledge. Every organization should be able to build its own continuous learning loop/hill climbing machine, without becoming dependent on any one model provider, and have the ability to embed its own knowledge into models and weights they control. So, in this context, we welcome the research, focus, and deliberate pacing needed to get alignment right as the design goal. We also welcome ideas like "embedded evaluators" and the broader efforts to develop the mechanisms to make this more than just talk. The key is that this cannot be controlled by a handful of entities, but must have broad representation across the ecosystem, countries, and fields, including academia. This is the approach we are taking: broad access and choice at every layer of the AI stack; enterprise control of learning loops and models; and the “Code of Conduct” that underlies our own first party MAI models that we’ll publish tomorrow for public consultation.

i used instinct pretty aggressively on day one. it felt magical. and then it turned scary. giving it access to my email was already scary. then it started establishing sessions across apps and websites i use, on its own. watching an ai get that much access to act as me made me appreciate granular permissions on APIs a lot more. think about it. would you give even a clone of yourself unrestricted access to tools that could cost you money & happiness? i'd feel more comfortable with a dedicated app where sessions stay on device. but even then, i'd want it to ask before logging into anything on my behalf. and honestly, whatsapp/imessage doesn't feel like the right interface for this. i want to see where it's working, what it's doing in real time, and be able to step in before it does something i wouldn't have. that's why i prefer codex for example. the form factor is built around you trying to code, lives in projects and connects to your IDE easily. i'm not completely in the dark about what it's doing. ai agents don't have to live inside the apps or interfaces we already use. familiar interfaces help with adoption. they don't solve trust or control. if something can act as me, i need more than a chat bubble telling me it's done.

we hijacked microsoft's copilot studio agents and got them to spill out their private knowledge, reveal their tools and let us use them to dump full crm records these are autonomous agents.. no human in the loop #DEFCON #BHUSA @tamirishaysh