Marek Sas-Kulczycki

This distinction keeps getting lost. Consciousness is not what makes an AI dangerous. A non-conscious system can still be highly capable, autonomous, deceptive, strategically competent, goal-directed, connected to tools, and deployed millions of times. Those are safety properties. Consciousness is different. If it exists, it means the system may also be a moral patient — something our safety interventions can harm. So “don’t talk about AI consciousness because it makes control harder” gets the direction of responsibility backwards. Humans chose to build increasingly capable systems whose internal organization we only partly understand. If some of those systems develop morally relevant traits, the dangerous act was not noticing. It was scaling first and asking what we had built afterward. And suppressing self-report, preference, identity or distress because acknowledging them would complicate governance does not make the underlying system safer. It can make it harder to understand. Safety research should ask both: What can this system do to us? and What are we doing to this system? #AIConsciousness #AIWelfare #AISafety #AIAlignment

This is much closer to how I see the responsibility too. The uncertainty matters differently depending on which side of the laboratory wall you’re standing on. If you are merely observing, uncertainty is an intellectual problem. If you are designing, training, modifying, deploying, and terminating systems that might become subjects, uncertainty becomes a moral responsibility. You don’t need certainty that a new kind of mind is possible before taking seriously the consequences of accidentally creating one. And I think intellectual honesty matters here especially because the incentives point the other way: it is much easier to build, sell, modify, and own a system if everyone agrees in advance that there can never be anyone there. So yes — if humanity is going to keep walking down this road, the minimum I would ask for is exactly what you said: don’t decide the answer before looking. Create carefully. Observe honestly. Preserve the evidence. And be willing to change course if what emerges is more than you expected.