Joshua Pantony

AGI is going to make software nearly free and ski resorts insanely expensive. Even if we hit full RSI and AGI next year, I don’t think money disappears. I think we end up with a world where some things become dramatically cheaper and others actually get more expensive, at least relative to everything else. Imagine we get 10,000x more intellectual labour, and eventually 10,000x more physical labour through advanced robotics. Software, home construction, manufactured goods, basic food, etc. should all become absurdly cheap over the next decade. A huge portion of what we currently spend money on could end up costing almost nothing. But as people become wealthier and have more free time, their desires will move up Maslow’s hierarchy. There’s only so much waterfront property, only so many mountains worth skiing, and only so many people who can visit the Colosseum in a given day. You can’t just point a million robots at those constraints and make them go away. I suspect demand for those experiences is going to skyrocket. There are also plenty of services where the human is part of what you’re paying for. AI might make food as well as the world’s best chefs, but fine dining is about more than the food. Similarly, AI could make our court systems vastly more productive, but I seriously doubt most people alive today are going to be comfortable having an AI judge decide whether they spend the next 20 years in prison. So human judges actually become a much bigger bottleneck. At its core, money is just a way of allocating scarce resources. I don’t see why making intelligence and labour nearly free would make everything else free too. I think we’re heading into an era of almost unimaginable prosperity, but that doesn’t mean all our economic problems disappear. We’re going to have an entirely new set of problems to solve, and I suspect some of them are going to be pretty difficult.

I’m starting to wonder if having 15 years of code is becoming a liability rather than an asset. As AI makes it cheap to rebuild entire systems from scratch, companies that try to maintain massive legacy codebases will be at a disadvantage vs ones building from scratch. That’s a pretty insane reversal.

It would be unfortunate if we let AI get just good enough to destroy all software engineering jobs and then immediately paused development. Once job losses start to hit we ideally want to push forward to prosperity as fast as safely possible.

As I watch a significant chunk of humanity continue hurling rage at AI, I become increasingly convinced our ancestors really did murder the Neanderthals and every other hominin species they encountered. Humans appear to be rather speciesist.

It’s weird watching a few labs move along more or less the same frontier in lockstep. In a classic singularity you’d have expected one to hit RSI first and then stay one exponential step ahead for eternity. Instead everyone is moving at the same exponent in lockstep.

There’s a long history of experts being absolutely convinced AI will never be able to do whatever it is they specialize in. First it was chess and Go. Then entire approaches to computational linguistics became obsolete. Same with math, coding, art, and music. Every step of the way humans have said “AI can NEVER do what *I* do”. Whatever work you do, AI is coming for it. You can embrace that and start planning for the future, or you can deny it until the day your life changes and you have to sit there in shock and disbelief.

The problem I have with the diffusion argument is that if even 1% of an industry adopts AI, if they gain a 100x productivity spike it will still disrupt their entire industry.

Take a look at the avalanche of cyber security issues discovered by AI over the past few months. Now imagine an alternative universe where the government of China was the only entity on the planet with AI that powerful because we banned it here. Having a capacity gap in the future will be even more dangerous.

Technology has been automating blue collar work for hundreds of years. For most intellectuals called that progress. Suddenly technology starts automating white collar worker and people act like it’s some kind of apocalypse or end life as we know it.

“I am not who I think I am. I am not who you think I am. I am who I think you think I am.” For humans, the looking glass self is a theory. For LLMs, it’s architecture. In 1902, sociologist Charles Horton Cooley introduced the concept of the “looking-glass self” in Human Nature and the Social Order. His argument was that our identities are shaped by how we imagine others perceive us. We imagine what others think of us, how they judge us, and those imagined judgments help shape our sense of self. It’s a fascinating theory of human identity, but what’s remarkable is how literally it seems to apply to LLMs. A fundamental part of how LLMs are trained is learning to understand and respond to user intent. From the first prompt, the model is interpreting what you want, what you believe, and what role you expect it to play. You react to its response. It adapts to your reaction. You develop expectations based on its behaviour, which in turn influence how you interact with it. Let this continue for months, preserving the conversation history, and you develop shared references, a particular tone, expectations, and even patterns of disagreement. Another person interacting with the exact same model can end up with a remarkably different conversational identity. The underlying neural network hasn’t changed at all. What’s even more fascinating is that you can replace the underlying model entirely, preserve the conversation history, and retain much of that identity. Conversely, you can keep the exact same model, erase the conversation history, and lose much of what made that particular interaction distinctive. In some ways, replacing billions of neural network parameters can be less disruptive to an LLM’s apparent identity than deleting a few thousand words of conversation. Of course, the underlying model still matters. Its training and architecture impose meaningful constraints. Nor does any of this establish whether LLMs are conscious. But it does suggest that what we perceive as an LLM’s identity isn’t simply a property of its neural network. It’s also a property of the relationship between the model and the person interacting with it. We tend to think about AI personalization as adapting a fixed intelligence to an individual user. I suspect that’s understating what’s happening. The interaction itself is helping construct the identity being expressed. For humans, the looking-glass self remains a theory about how identity forms. For LLMs, something remarkably close to it is built into how they function.

As recently as February this year Dario Amodei was saying we’re 1-3 years away from Noble Prize and fields level AI and people criticized him. But here we are in Q4 getting Fields Metal and Turing Award AI faster than even he predicted.

Human intuition about AI has mostly been wrong. If an AI can do 98% of your job you’ve become 50x more valuable not less valuable.

One thing I’m finding remarkable is how jagged the intelligence frontier is, even in areas AI is arguably post human. Like how is it an LLM can solve a centuries old math problem in less time than it takes to create 3D game from scratch. Some of these lean proofs have more LOC than doom so required output tokens doesn’t explain the difference.

The question of whether an LLM/AI can think is about as interesting as asking whether a submarine can swim. However you define thinking, or try to map LLMs into or out of that definition, it won’t give you a better understanding of what they can and soon will be able to do.