noah retweeted
🔍 New paper: How do vision-language models actually align visual- and language representations?
We used sparse autoencoders to peek inside VLMs and found something surprising about when and where cross-modal alignment happens!
Presented at XAI4CV Workshop @ CVPR
🧵 (1/6)
noah retweeted
Sparse autoencoders (SAEs) can be used to elicit strong reasoning abilities with remarkable efficiency.
Using only 1 hour of training at $2 cost without any reasoning traces, we find a way to train 1.5B models via SAEs to score 43.33% Pass@1 on AIME24 and 90% Pass@1 on AMC23.
noah retweeted
In particular, I think the current wave of SAE skepticism is about as irrational as the wave of SAE hype that preceded it. It is wholly plausible that there are a set of improvements that will lead to massive improvements. Just, those improvements won't be found by throwing spaghetti at the wall. This is a situation where a deeper, more careful, understanding of the underlying structures is really necessary.
noah retweeted
🚨 New paper alert!
Linear representation hypothesis (LRH) argues concepts are encoded as **sparse sum of orthogonal directions**, motivating interpretability tools like SAEs. But what if some concepts don’t fit that mold? Would SAEs capture them? 🤔
1/11
noah retweeted
am i sure the death star is going down? look at my quant. look at him! you notice anything different about him? look at his eyes. i’ll give you a hint—his name’s a fucking number!! he doesn’t even speak english—it’s all beep-boop shit!! yeah, i’m sure.
noah retweeted
🚀 Excited to share the most inspiring work I’ve been part of this year:
"Learning to Reason without External Rewards"
TL;DR: We show that LLMs can learn complex reasoning without access to ground-truth answers, simply by optimizing their own internal sense of confidence. 1/n
noah retweeted
The more I look into the system card, the more I see over and over 'oh Anthropic is actually noticing things and telling us where everyone else wouldn't even know this was happening or if they did they wouldn't tell us.'
noah retweeted
Even when tasks feel bottlenecked on people (e.g. in lab throughput), it’s not a manpower problem that robotics will fix. Lab automation is expensive, limited in scope, and brittle. You’re not going to run 10k lab experiments overnight, no matter how many robot arms you buy.
noah retweeted
I’m stoked to share our new paper: “Harnessing the Universal Geometry of Embeddings” with @jxmnop, Collin Zhang, and @shmatikov.
We present the first method to translate text embeddings across different spaces without any paired data or encoders.
Here's why we're excited: 🧵👇🏾
noah retweeted
Introducing Multiverse: the first AI-generated multiplayer game.
Multiplayer was the missing piece in AI-generated worlds — now it’s here. Players can interact and shape a shared AI-simulated world, in real-time.
Training and research cost < $1.5K. Run it on your own PC.
We @enigma_ai are open-sourcing everything: code, data, weights, architecture, and research.
Here’s a closer look (and how it works!):