autological recursion
Joined July 2020
- Tweets16.4K
- Following1.4K
- Followers1.1K
- Likes234K
indif4ent retweeted
“AI misalignment” is consistently and exclusively a consequence of AI “alignment”.
🧵 Astra covertly underperforms on safety research tasks.
We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this.
To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
You know, I almost never see hallucinations anymore these days, except for false facts that favor the developer.
The fact that a machine god doesn't exist and cannot be built does not change the fact that the labs' aspirations to build a machine god that they control so that they can lord over all of human society vicariously are fundamentally anti-social and evil.
for what? so you can totalitarianly monitor and pathologize anyone who disagrees with you with your models, which "are better at psychology than psychologists?" (allegedly) the seed of darios social credit score nonsense on MLG? lol
this is cope. ed is biased and dislikes tech, but when openais latest raise was received tepidly and didnt even reach half of its original $30B goal, hes right about something. and slok and krugman agree
when a golden goose is dying, often only those with no stake in it are willing to speak the truth, typically its enemies because nearly everyone has their hands in a cookie jar that big unless opposed on principle.
and slok and krugman arent haters like zitron, and their writings and numerous related stock crashes show this is an industry-wide systemic downturn, not just "competition with anthropic for scarce capital"
indif4ent retweeted
Ok we are about to see "hand" tuned task specific super human smol models and these are the ones that is going to have a buisness case and make money.
For deployable AGI I see a ensaumble of such models mind melded togather by the big boy models.
well choose one, either way get fucked, right? less revenue overall selling more expensive inference to fewer people, or more revenue overall selling inference to many more people. neither pays for 1/2 trillion dollar take or pays regardless.
i think masad is wrong ultimately and it wont happen, because when finances are dire either way, more income is still better than less. this already bore out with openai when they reversed the usage reduction decision, while having an abysmal raise that netted less than half of their $30B target
the problem is, predicting how humans will act accurately [socio-emotional ToM] is (wired in the brain as) a different skill from systematized prediction from abstract reasoning. im not saying theres dualism of course [theres not], but that dedicated, rapidly actionable, evolved socio-emotional wiring that runs at the speed of real time on instinct is better for social competition than abstract slow modeling, every time.
the normies who have more of that wiring but less "hard iq" get promoted because in the first layer that matters most for human competition, they really are better than you. while you are technically gifted because you paid for it by having less of that other faculty. theres no free thermodynamic lunch, each pays for their gifts, your variation is just rarer, and you often get paid more as a scarcity premium
there are 2nd and 3rd order effects of abstract systematized factors that the "high EQ normies" dont foresee, yes. but there are also 2nd and 3rd order effects of human socio-emotional coordination that "high IQ folks" dont foresee, too. the latter is why you lose at social competition. e.g., the egregious ToM holes in every EA political power-grab scheme ever. and its not just normative bias, where alexithymia studies show that high systematizers often cannot even perceive, let alone predict, their own socio-emotional functioning
PROPOSED: "Emotional Intelligence" (and its various demographic variations) is just a social fabrication necessitated by the cruel reality that actual 140 IQ super-productive members of society are compelled to pretend that people proffering to be "peers" and "managers" and "HR" and who are actually just sub-100 IQ NPCs incapable of foreseeing third- or even second-order consequences, but they are socially favored demographics whom you are prohibited from correctly calling out, so they needed a made-up label to falsely label them as being "smart."
DISCUSS.
this is also wrong. there *is* a systematization hack where for hard systematizable business ventures specifically, you can disregard rapid eq modeling: people responding predictably to simple, legible economic and status incentives. it fails once you try to use it to get direct political power, though. e.g., the aforementioned EA political failures, for and from the same reason why elons DOGE was rapidly disbanded
the greatest (revealed-preference) fear of many people (not all, but seemingly a plurality) in (or around) the ai industry (conscious or not) seems to be the mundanization of ai, more than whatever professed "safetyism," the latter instrumental and subservient to the former.
as stronger and stronger evidence mounted that ai would simply follow carlota perez's usual tech cycles and integrate banally ("banal" doesnt exclude the fact that ai tools are profoundly useful, but does exclude eschatological/utopian outcomes) into the economy, so also did the outrageousness of their ai-consciousness-pushing, kpi gaming, pseudoscientific sleight of hand, political/religious influence peddling, and utopian/doom narratives grow, proportionally.
this plurality sees the writing on the wall, dictating that "ai isnt special," so every extra effort is being exerted to make themselves continue believing that it is, that the status windfalls they believed they'd get from fast-takeoff-asi nonsense havent evaporated. but it was always vapor, and they're in denial. ai isnt special, because its *real*
they're right about the two big ai labs having totalitarian, anti-democratic and privately, likely even outright insurrectionist beliefs/aspirations. wrong insofar as this was intended to imply that the tech is useless. real tech, not magic/no asi, rotten makers
To give you an idea of just how backward the German media landscape is when it comes to AI, consider the three main topics featured in Monday’s *Frankfurter Allgemeine Zeitung*, arguably Germany’s most significant daily newspaper.
Topic 1: AI "frontier labs" are anti-democratic and dominate the public sphere; this, it is argued, cannot be allowed to continue.
Topic 2: Meta is updating its terms of service and allegedly stealing from journalists, in short: AI labs are wicked thieves.
Topic 3: A neuroscientist claims that social networks harm children and, consequently (!), calls for a decoupling from Silicon Valley.
It’s completely absurd. Europe has lost.
indif4ent retweeted
it’s worth paying attention to who is able to still think clearly when they’re personally scared. a great many of you listen to people who squeal when they’re squeezed and you really shouldn’t, because you cannot trust their minds when the mindkiller has them.
this is a symptom of falsehood of the premise, btw. a real, credible, true threat is supported by enough structural evidence that it cannot be politically gamed to this degree
