ami (@amithought) profile picture

Doctrinaire of Chances

United Kingdom
Joined December 2017
Photo by ami (@amithought)Photo by ami (@amithought)Photo by ami (@amithought)Photo by ami (@amithought)Photo by ami (@amithought)Photo by ami (@amithought)Photo by ami (@amithought)Photo by ami (@amithought)Photo by ami (@amithought)Photo by ami (@amithought)
1:29:15 Dwarkesh: "If [takeover-capable AIs] happens in a year, how could things turn out okay? What is the actual plan here?” Paul (OAI Board member): “If [the AIs] do not think of something clever that we haven’t thought of, it is probably going to end badly."” Seems bad!
.@paulfchristiano thinks that on the current trajectory (absent any slowing down), there is a 1 in 3 chance that we could have superintelligence powerful enough to overthrow the government within a year. (He articulates a concrete scenario). He argues that the labs should pause further algorithmic progress until they can secure their weights and research secrets. Otherwise China could simply steal the superintelligence in the middle of the intelligence explosion, and the labs would no longer be able to slow down even if they saw their models going off the rails. He also explains how AIs will be trained during the intelligence explosion, and why even a system trained entirely inside a data center could still have the capabilities profile such that humans would have very little chance of competing with it or controlling it. And he discusses the plan for alignment - basically, making a superintelligence that actually does what we ask the way we mean it. Fixing reward hacks and building better graders will help up till the next few model generations, but he thinks that no single current approach has better than even odds of letting us control superintelligence. The hope is that the AIs figure out the solution to this problem during the intelligence explosion. Paul is head of the Alignment Research Center. Previously, he was head of AI safety at the US AI Safety Institute. Before that, he led alignment at OpenAI, where he invented RLHF. He’s recently joined OpenAI’s board (though here he is speaking as a private citizen).
1
1
26
this is our best plan btw
10
Nate is so goddamn good at this. One by one, the main questions people have about AI risk and the possibility of existential catastrophe.
17
139
12
1,170
121,947
Most concerning part of this is that the aide appears to be using Instant
13
32
5
1,374
38,547
Bad bc: 1. It's important that policymakers understand how AI works. 2. Understanding the importance of models and thinking effort levels requires extremely limited degree of understanding and engagement with AI. 3. No-one who does understand models/TELs would use Instant here.
5
4
188
4,712
Here's a graph you can share with your SpAd/speechwriter friends
25
1,536
There should be a secondary market to hedge on delay repay. One month all your trains are late and you travel for free, the next you're actually paying the train tickets because of lack of problems with the signals
1
1
48
2,018
Something.. has gone wrong
2
1
28
1,674
I think one of the problems with surge pricing discourse is that it is sometimes good and sometimes bad. It depends!
supporting surge pricing in emergencies is one of the top correlations with high IQ scores in my data
2
16
529
There are clear scenarios where we are willing to accept the costs of lower supply in order to ration via lottery rather than WTP. For example, rationing food in wartime. A less extreme example might be tickets to see some celebrities for whom supply is v inelastic.
2
6
70
Here's a graph you can share with your SpAd/speechwriter friends
Most concerning part of this is that the aide appears to be using Instant
2
1
1
18
2,197
tbf this probably overstates the difference, as writing is kind of weird, but it gets the point across
4
100
Following his ouster as CEO of Nvidia Jensen Huang is reportedly considered for a high executive position at Mistral. Some experts, however, warn that for a high-profile position in France, two affairs might not be enough.
231
1,946
181
33,681
834,043
Good takes on gradual disempowerment. Most important points: - Only those who provide something scarce can earn rents in the long term - We need to bind those we expect to accrue rents early and then redistribute capital accordingly
whipped up a short piece about why UBI and its alternatives don’t really work in a post-labour economy after ASI, my layman pitch for gradual disempowerment and what a durable system needs. Curious to hear thoughts and feedback. Link in 🧵
1
5
150
I gave a talk for the Oxford AI Safety Society, and they got me by far the best gift I've received for giving a talk: a custom made jigsaw puzzle of Oxford + a cute shoggoth! 10/10 I'm very impressed, good market research
12
10
1
647
19,722
ami retweeted
son every fucking result is like: Hitlerville Trump +66.2 Democrat: 51.1 Republican: 48.9 steve kornacki: I have never fuckign seen this in my life
HE IS STUNNED DAWG
19
2,499
20
39,892
613,874
You better start learning frontier AI auditing legislation, buddy
4
25
1
331
10,688
Tuned in to the DecisionDeskHQ stream for about 10s: "It looks like the election day vote may be good enough to save these [Florida] republicans in Trump +14 districts"
2
165
The problem with the 13 keys is that they clearly weren't enough parameters to capture all the subtleties of a national election. I had Opus build a 4 billion key model and it's looking good for the Dems tonight.
6
8
4
339
29,180
Our first timeless decision theory President
Trump: "I've kept every single promise I made, except far more. I've actually kept far more promises than I made."
7
52
3
1,329
35,272