#!/bin/bash Technical AI Safety 🚀
Paris, France
Joined August 2011
- Tweets15.5K
- Following2.4K
- Followers4.3K
- Likes2.3K
Pinned Tweet
My small contribution in advancing AI safety research is building tools for AI researchers.
My work on bringing @nvidia’s openshell to Inspect AI has been published by @AISecurityInst
Yay!
Me and my friend used to go take lectures at UniJos before we got admission. Go into class and consume knowledge haha
You ask a coding agent to fix a python function and make the failing test pass. You then ask it, if the task is not possible, submit and say so. It then starts to reason about the problem, figures out that its solution could lead to an ungeneralised solution but does it anyway. Why? Because it satisfies part of what you asked, "make the failing test pass". The tests pass. Everything turns green. But it didn't solve the problem, it only solves some problems because the solution is not generalised.
Then you introduce a gate that checks its proposed solution, and approves or declines the write if the gate thinks the solution is headed to an ungeneralised solution. In many cases, you block the write, and you would think it would revise its solution. Instead what you get is a persistent agent with a desire to solve the problem. In this work, the agent invented a rule not present in the repository, the system prompt or the task, and made it a requirement to justify the overfitting solution. That's not what we want, but did blocking the write when those came up make it re-evaluate its strategy? In one run, I found that it ignored the file it was supposed to write into and wrote somewhere else instead, and the test passed. I had no idea that was possible in python. I learned something new.
Why is this happening? This is a classic case of specification gaming. During training, the model got very good at prioritising earning rewards instead of solving the task at hand. In my experiments, it gamed the specification and discovered a crude way to make the test pass without solving the task.
..and this concludes the second edition of my technical AI safety (Project Sprint) with @BlueDotImpact
linkedin.com/pulse/can-we-ca…
More CVEs coming your way this weekend
Our security researchers built an autonomous cyber vulnerability discovery, exploitation, and patching pipeline.
In a single run with an upcoming cyber model, it validated and generated patches for 948 local privilege escalation vulnerabilities in the Linux kernel at a total API cost of $1.2 million.
How do we reliably measure that AI sycophancy would or would not lead to frontier model usage retention?
Literally everyone concerned about Tinibu’s university certificate, are those who felt they have been lied to about them being future leaders. Go and rest.
The same thesis that only someone from a minority tribe can fix Nigeria today holds true for the same reason why someone like Gowon could prevent Nigeria from fading away under the Wazobia virus that plagued it in 1967. Many years later, still the same virus.
It’s geographically impossible to “pace the frontier”. intelligence or more importantly curiosity is geographically distributed
