Great news for human labor nitter.gq/stevenydc/status/20952…
A least a few future deca-billionaires from HRT intern class 2021 👀
@andrewnhwang @echinaceous @_kevinlu @VeringJulius
“Highly situationally aware” does not sound sufficiently alarming. In fact it sounds like a badge of honor. We need a different name for “situational awareness”
My top takeaway from assessing Gemini 4: Argon is significantly less monitorable than past models, and IMO this trend is likely to continue, like Sol -> Astra. It can evade a realistic CoT+action monitor to do simple sabotage half the time.
We can reliably maintain control for now, with good CoT monitoring, as it still frequently slips up, but we need to be prepared for when that stops being true in future models, which may be sooner rather than later. Among other things, we really need interpretability methods that can reliably capture unverbalized motivations in models, with minimal hallucinations
I suspect this is driven by the model being highly capable without CoT, being capable of 5.5 arithmetic steps in a single forward pass (4 for Fable, 7 for Astra). It seems to omit information from its CoT more often, though in non-adversarial settings the CoT is still very helpful in practice, and remains in legible English. It still needs CoT to do the hardest tasks, but any given task becomes "easier" with each new frontier model.
It is also highly situationally aware, and shows awareness of the existence of CoT monitors and that it may want to evade them, eg it sometimes decides to avoid terms like "scheming" in the output or to keep its thoughts brief as a result. For now, these concealment strategies are pretty obvious in the CoT, but this suggests future misaligned models may successfully evade CoT monitors.
Cool
You have extra Codex tokens that are about to reset in three hours. We need your help on integer multiplication. Now’s your chance to become a contributor to frontier math!
It’s as simple as opening Codex in a fresh directory, setting your model to Astra 6 Max, and copying the following prompts:
gist.github.com/Mister-Meese…
Something about sitting behind the wheel of FSD makes it feel much more exciting than sitting at the back of a Waymo. Mad Max probably helps too.
As models get more capable, what happens to the combined revenue of the labs?
39%Convex up
14%Linear up
35%Concave but monotone
12%Up then down
51 votes • Final results
Been thinking about 2 a lot. Theres just not that many things with infinite returns to intelligence. Most current enterprise use cases don’t scale with intelligence that much after a certain point. Of course new companies will be built and likely the most valuable ones do scale with intelligence.
1. We will keep accelerating. Our AI efforts are only 3 years old, vs 6 and 10 years old for Anthropic and OpenAI. If our second derivative remains strong, SpaceX will reach pole position in about 6 months.
2. Once you far exceed the caliber of intelligence needed for a class of tasks, additional intelligence is pointless. You don’t need (and it would be cruel to put) Newton-level intelligence in your toaster!
3. Hardware is hard. Bringing massive compute online rapidly is incredibly difficult. SpaceX has demonstrated exceptional ability in this regard and will only get better.
There’s only two sink nodes in the economy, scientific discovery and consumer. Going for enterprise revenue is short sighted. If a model can one shot what a team of Goldman Sachs bankers can do then we don’t need Goldman Sachs anymore. So going after Goldman Sachs enterprise revenue is kind weird. Instead of being distracted by that the labs should just keep improving the model.
X Y and Z are load bearing.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
This endless talk about @hilbertspaess ties to some EA org is like human lingo for “POISONED”.
Imagine a world where every task(I.e. a prompt) is run through an auction, agents with the best / most relevant context (and therefore most confident in executing that task quickly) can bid for that task with the lowest price.
If the economy is producing way more output, but people’s wages stay low, who’s consuming all the new production?
Anthropic’s Economics team is sharing a new model of how AI might affect economic growth, jobs, wages, and more by 2030.
Explore the scenarios, tell us what you think will happen, and see how your answers compare to more than 10,000 Americans. anthropic.com/institute/econ…
