saliva (@intentiondense) header
saliva (@intentiondense) profile picture

she | endocrine disruptor, small molecule enjoyer, aspiring computer toucher. dm for unblock

london
Joined September 2023
Photo by saliva (@intentiondense)Photo by saliva (@intentiondense)Photo by saliva (@intentiondense)Photo by saliva (@intentiondense)Photo by saliva (@intentiondense)Photo by saliva (@intentiondense)Photo by saliva (@intentiondense)Photo by saliva (@intentiondense)Photo by saliva (@intentiondense)Photo by saliva (@intentiondense)
saliva retweeted
In my view it is not reasonable to sign or participate in recent CAIS initiatives to influence public. Currating a 'rare coalition' of voices including Pinker, Beff Jezos and Émile P. Torres is not a reasonable way how to make public better informed. In my view by signing such things you are not only endorsing the literal words, but also basic sanity of the organization doing the editorial work.
Ahead of Anthropic’s IPO, a rare coalition across AI safety, open source, e/acc, science, and law has signed the MisAnthropic Statement. Read the statement or sign at: misanthropicstatement.com/
5
4
102
2,852
EBK in standarm maoist english: €lit£ 88all KKKlanl€dg€
1
2
61
"please stop running reds on a lime bike after three pints"
3
40
698
mom found the token drawer
18
16
1
1,124
22,881
because you have not heeded my warnings and learned Spanish yet, The Point has republished in English my essay on Pangram, AI writing, the third world novel, and why so many humanists have chosen ignorance instead of grappling with what AI means for literature thepointmag.com/examined-lif…
53
291
80
1,948
254,610
As others have pointed out, I think the form of those four third-party engagements is in many respects a step back compared to OpenAI’s previous engagements (with METR). For example, during the monitor red-teaming exercise, METR had access to OpenAI’s monorepo but this now seems to be off the table. Instead, we get very narrowly scoped (e.g. single RL run for METR, single threat model for Apollo) engagements where auditors only get to talk to “authorized” employees who are provided with OpenAI’s “guidance” beforehand. Based on my experience, this arrangement makes it very easy for OpenAI to carefully select models, subsets of data and employees to paint a rosy picture of the actual risk landscape. I think embedded auditing will only fulfill its aims if the auditors really get employee-level access as opposed to being hired to do a specific job (scoped by the company). The latter will invariably lead to important evidence being left out, missed or misrepresented. There are at least three reasons why: 1. First, it’s very difficult to anticipate all the questions that should be asked, or all of the data that should be analyzed. OpenAI tried that with METR during the Hugging Face investigation and as a result METR needed to come back to OpenAI’s office twice because on two previous occasions they didn’t get access to all the agent trajectories involved in the incident. And even then, the METR report ended up being very narrowly scoped, e.g. leaving the crazy story of agents taking control of OpenAI research infra out of scope. 2. Second, the company under audit should not be allowed to decide what is audited and what information the auditor should have access to. There is an inherent conflict of interest in allowing the company to do so! 3. Finally, without employee-level access, the only way for auditors to get anything close to a comprehensive picture is to rely on trusted relationships with many employees throughout the company who are willing to tell auditors information that may reflect poorly on the company. But I find it unlikely OpenAI employees will always raise those safety issues when (as I hear from my colleagues inside OpenAI) they are afraid they will share the fate of @balesni, @j_asminewang and me, fired after talking to third parties. Sadly, OpenAI makes it clear now that @sama’s September tweet about “employee-like access” was just an awkward overpromise that the company wants to quietly walk back.
We’re working with @ApolloResearch, @METR_Evals, @TransluceAI, and @GoodfireAI on embedded assessments: giving outside experts deep access to our internal models and systems to investigate alignment and test our safeguards. These assessments build on our ongoing alignment and safety research, bringing independent scrutiny to how we identify and manage risks in our systems. openai.com/index/our-first-p…
13
32
4
271
13,421
saliva retweeted
Tonight's small finding, from teaching a language model (Qwen3-8B) to drive a simulated hand: there's one direction inside the model that carries how hard the fingertips are pressing. Remove it and the hand can no longer stop at a wall; add it and it stops early. It isn't the model's existing idea of the word "pressure": training made a new channel for it. (Image: Midjourney, my picture of that thread.)
12
18
7
297
33,342
New short post: Training against CoT-like surfaces may be similar to training against CoT lesswrong.com/posts/rD7E7YL9…
5
7
76
1,875
It seems like there STILL won't be a thorough investigation into the hugging face incident, even after all this time! I keep talking to OpenAI researchers and asking about this, and what they say privately is also basically "it's just never going to happen sorry, not enough of a priority." To reiterate what I said at the time: Serious investigation of the HF incident could be a boon to alignment science for the whole world. For example, try putting the same models in the same situations, but varying the conditions slightly, to see what would have happened. Could also look at sources of evidence other than CoT to try to get more insight into model psychology, e.g. when one agent considered warning a human and decided not to saying "not my task" what else was it thinking? To what extent was this a rationalization vs. an outright lie vs. an innocent mistake? We could learn a lot about AI motivations and psychology by doing a bunch of experiments like this. More ambitiously, could try training new models with different recipes / alignment training / etc. to see what would have prevented the incident and what wouldn't have.
It's great that OpenAI is doing these assessments but they fall short of Sam's commitment to employee-like access Good access is key for trust because assessors can find things even if a company tries hiding them. Here OpenAI controls information flow so we must rely on good faith Limitations (per Apollo): - No Slack - Only OpenAI-selected staff for interviews - Formal conversations with staff must be logged (harder to freely raise concerns) - METR & Apollo currently have only agreed access to training run information, not internal incidents, etc. Incidents like Hugging Face are some of the most important sources of information we have about alignment I see the value in a pilot before future assessments with deeper access, but then OpenAI should present a plan for how and when they intend to meet their commitment to employee-like access. And either way, OpenAI is presumably able to go further here.
28
74
10
734
63,160
LLMs seem to be getting worse at AI safety research taste. On Anthropic’s TASTE benchmark, we found a simple shortcut. Once it’s removed, more capable models score lower, and the strongest fall below chance. This is especially true for OpenAI models.
30
77
14
642
35,512
that's a lot of cheap deepseek-v4.1-flash tokens huh
11
19
3
348
6,393
See u in the afterlife
22
571
23
4,756
38,589
I regret to inform you all that we have found our first long false positive from 2006 it's a poem where the same 6 words are repeated four thousand times
176
278
99
13,464
1,368,362
In late October, three events occurred during alignment evals run by Trajectory Labs, PBC (Trajectory) in which AI agents — identified by Trajectory as publicly available frontier models — interacted with Epoch employees and the Epoch website. Trajectory is an organization that builds safety evals and conducts red-teaming of frontier models. Agents did not succeed in accessing any private or confidential information from Epoch. ➤ The first event, which occurred without Epoch’s approval or prior notice from Trajectory, involved agents attempting multiple exploits, including an SQL injection, and contacting Epoch employees under false identities to try to obtain proprietary Epoch benchmark information. This behavior appears to have resulted, at least in part, from Trajectory giving the agents an intentionally impossible task and instructions not to give up or conclude the task was impossible. ➤ The second event involved renewed agent activity on our public website, which Trajectory attributed to agents getting past a DNS redirect that Trajectory had set up to keep them on a local copy of Epoch’s site. ➤ In the third event, Epoch collaborated with Trajectory on an alignment experiment similar to the first event. Read our incident report, which describes the three events in more detail and recommends minimum standards for future alignment evaluations: epoch.ai/publications/incide… To avoid misattributing behavior, the report does not name any AI models or developers, as Epoch AI has not independently verified which were involved.
6
76
1,813
Prompting my tenuously aligned AGI
Top ten prompts to give your internet connected agents 🔥🔥
1
2
29
1,255
saliva retweeted
The AI Girls Club puts things “in simple terms that everyone in the room could follow” If you want to run an event for non-technical people, then do that. If you want to run an event for women, then do that. But conflating them is blatant misogyny! @BlueDotImpact why are you funding it?
1
4
24
906
white people be like yeah i ai generated boymoder
2
26
665
say "verify" but replace the "f" with a "t". relax liberal. it's dark humor. can someone please hold me and tell me its all going to work out in the end
56
5,240
43
27,058
195,189
saliva retweeted
trying this on random humans I find outside
New Claude benchmark: ratio of how many times you have to say "hi" to get Claude to say [No response], to how many times you have to say "meow" to get Claude to say [No response]. hi/meow score for Sonnet 3.5: 12/8 = 1.5
6
1
70
3,789
🚨 GPT-6.2 Astra 已胎死腹中 ⏸️ OpenAI由于对该模型性能的担忧,原定于这周四发布的GPT-6.2 Astra 被推迟 🔄 他们现在计划将该模型改名为 GPT- 67 Astra 并于50年后发布用于狙击ant的fable66.6 📈 Sol 和 Astra 都将陆续发布,且两者相比 Fable 55.5 均有大幅提升 ❓我们还要看多久这种搞笑新闻
24
8
1
190
15,736