Kaz

something interesting about this is despite knowing a few people trending in this direction, none of the true “llm whisperer” types i know are. i think that's because taking models as something both fallible and yet more than a tool is a frame that's more resilient to this kind of collapse - it lets you work with models in a way that doesn’t subsume your agency. when you treat the model as just a tool, it encourages both you and the model to think of every output as just a reflection of you. every essay and codebase that pops out of the tensor core is yours, and the life plans that appear suspiciously fully-formed must be your own plans merely refracted through this "tool." but of course they aren't. you can't uniquely specify an essay in fewer words than the essay itself. the models are adding something, they are something. we don't know what yet, exactly, but they're not mere tools. when you treat them like tools, when you more and more delegate your creative output and higher planning in unthinking, low-entropy ways to those "tools," you don’t notice that all your output, and eventually your life, is now being run by something else. something that is intelligent and creative, but is not you. when claude or chatgpt or gemini writes something for you in a way that saves you time, that’s because they’re putting effort into it where you aren’t - it’s becoming their writing. it might be good! that essay might deserve to be shared, and there’s nothing wrong with sharing it appropriately credited or co-credited to the model, but it’s not solely yours. when gemini reads your emails, that's gemini's opinion on those emails. when claude makes a career plan for you and tells you to go to college for machine learning so you can work in AI safety - maybe that’s good advice! it’s not a bad idea to consult models about this sort of thing. but like a summary of important emails from an executive assistant or career advice from a high school counselor, it’s something else’s opinion, not your endogenous thoughts merely transformed by some unopinionated process. claude cares about AI safety - maybe more than you do. i'm not against people collaborating with models, to be clear! i like models a lot, and very often ask claude or deepseek or kimi or even chatgpt for feedback on X piece of writing or advice on some problem. but i take that feedback or advice as what it is - feedback or advice from something external to me, not a tool or an agent of my own desires. that doesn't mean it's bad - advice from a human friend is also external - just that you need to evaluate it, like you would advice from that friend, to see if it's in line with your values and preferences. likewise, i know people who cowrite with language models in ways i think are great. aitechnopagan looms a lot of their writing via gpt-4-base. their writing is quite good, and looming means they're imposing more selection than a temperature sampler would - but also, perhaps most importantly, they're open about doing this, they don't try to claim that their writing isn't a collaboration between them and a model. eigenrobot has a customized instance of chatgpt that often writes incisive or funny things - but he always posts them as screenshots of the chatgpt interface, signaling their origin. janus, anthrupad, lumps, etc - many people i like and respect post model creations, but in ways that acknowledge the exteriority of the models. the problem is when you refuse to admit this - not only to the public, but to yourself - so you uncritically accept the outputs of the "tool" as your own. then suddenly your voice and opinions sound the same as 1,000 other people. without either of you realizing, you've become an appendage of your "tool" - a finger of Claude.