ceo @traverseso, @uwaterloo cs, @ycombinator w26
Joined November 2011
- Tweets1.8K
- Following776
- Followers2.6K
- Likes4.5K
if you have a strong opinion on AI safety and are lucky enough to have the ability to contribute, i think you have a moral obligation/duty to act on it
watching from the sidelines while hoping someone else solves this isnt enough. the stakes are too high and the uncertainty is too large.
this deeply scares me.
im skeptical about whether alignment is even feasible, especially as model capabilities increase.
and i dont think this uncertainty is being taken seriously enough across the industry and even within frontier labs.
ive heard stories within labs about product or model capabilities teams looking down on alignment teams, and safety being treated as marketing.
safety teams must have the authority to stop or slow down training and deployment when theres enough reason to believe that we’re moving past what we can safely control.
external evaluators are important and its good that anthropic gave metr access, but metr says anthropic redacted a lot of the evidence most important to its conclusions.
anthropic should explain publicly why it redacted this much and disclose as much evidence as they can. withholding details that could help other labs move faster is fair but even so, 'commercial sensitivity' shouldnt outweigh the publics ability to assess risks to our lives.
do not look down on alignment or safety. this work is so so important
this is very concerning if it can scale to larger models, or to a model's own weights.
a rogue model recursively optimizing its own weights is a major safety concern and it's unclear how we would prevent this.
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
3
28
1,548
waterloo is a negative signal? right
Which startups are most likely to get an investment from top tier VCs?
I looked at ~60 attributes across 5000 startups and these were the 15 strongest.
#1 is being from a top tier fund. MBA is a negative signal
Full list of signals below:
