Researcher
Bangalore, India
Joined June 2009
- Tweets1K
- Following194
- Followers106
- Likes15.9K
Arun Iyer retweeted
Our security researchers built an autonomous cyber vulnerability discovery, exploitation, and patching pipeline.
In a single run with an upcoming cyber model, it validated and generated patches for 948 local privilege escalation vulnerabilities in the Linux kernel at a total API cost of $1.2 million.
Arun Iyer retweeted
We test Astra weight edit on three robot domains: VLA, WAM and humanoid control.
On one task it edited only 𝟒 𝐩𝐚𝐫𝐚𝐦𝐬 𝐨𝐮𝐭 𝐨𝐟 𝟑.𝟔𝐁 for π0.5, and success went 𝟐𝟖/𝟗𝟐 → 𝟖𝟐/𝟗𝟐 on held-out unseen episodes.
How Astra got there actually interests me more 👇
Arun Iyer retweeted
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
Arun Iyer retweeted
🧵 [1/14]: How should we loop a transformer when model size 📦, training compute 🧮, and inference efficiency ⚡ all matter?
Looping is inherently simple: take the hidden representations produced by a transformer (or part of it) and feed them back through the same transformer (or part of it), reusing its weights. 🔁
But every extra loop costs compute. Choosing which layers to loop, how many times, and how to execute those loops determines whether the trade-off is worthwhile.
We explore these choices in our new paper:
“How to Loop Your Transformer: Loop Scaling, Specialization, and Parallel Inference”
Here’s what we found. Thread below 🧵👇
📄 Paper link: arxiv.org/abs/2610.40325
Arun Iyer retweeted
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
Arun Iyer retweeted
So one of our ambassadors @divagr1925 has been working on something cool
He found a way to use Qwen's Gated DeltaNet state as memory, and built a 32-bit RISC-V CPU directly into the model's weights. So instead of relying on predicting the answer to a calculation, the model can execute instructions internally. It can also run C and MicroPython programs.
He calls it Qwenputer, love the name btw. The results so far are pretty impressive. On one arithmetic benchmark, accuracy jumped from 48.6% to 100%.
Pretty cool to see developers experimenting with Qwen's architecture. Great work, my friend!
Arun Iyer retweeted
Over the past few months, Anthropic scientists have engaged in deep research partnerships with six organizations developing new medicines. We've also begun drug discovery efforts of our own.
In doing so, we’ve learned much more about how Claude can support the many stages of drug development.
In this blog post, we share some early results from these efforts, and describe how this work fits in with our wider ambitions in the life sciences.
anthropic.com/research/early…
Arun Iyer retweeted
Lean Verified Tokenizers (srush.github.io/verified-tok…)
Training-inference mismatch kills RL runs, but yet we build messages in Jinja? Prototypes a Lean-verified Rust agent pipeline.
Arun Iyer retweeted
When an LLM agent executes a long task, the hardest question is usually not whether it failed but where. Which step actually broke the run? 🧵
That’s the failure attribution problem, and it’s becoming central to debugging agentic systems. But current approaches have a real cost problem:
1. Prompting-based pipelines, where an LLM reads the whole trajectory and judges each step, are slow and expensive to run.
2. Post-trained attribution models need failure trajectories labeled at the step level, which are painful to collect and hard to scale.
We think a practical failure attribution model should be lightweight and shouldn’t need step-level labels on failures at all.
So in our #neurips2026 paper, “Tracing Agentic Failure from the Flow of Success,” we ask: can you find where an agent went wrong by learning only what going right looks like?
Meet OAT 🌾
The idea: treat failure attribution as one-class learning. We train only on successful trajectories, using neural controlled differential equations to model how successful runs evolve in latent space. Think of it as learning the “flow” of a healthy agent trajectory.
At inference, we take a failed trajectory and score every step by how far it drifts from those learned success dynamics. Steps that deviate most get flagged as the likely error steps. The algorithm doesn't rely on any step-level annotation.
What we found:
⚡ With just 100 successful trajectories for training, OAT runs 200-5000× faster than prompting-based baselines (including GPT-5 judge).
📈 It also outperforms them on accuracy: about +20% F1 score.
So you don’t have to trade cost for quality here. You can get both.
Why I’m excited about this: successful runs are cheap and plentiful, while carefully annotated failures are not. Building diagnostics on top of the data we already have feels like the right direction as agent systems grow longer and more complex.
Huge thanks to my amazing student @Samuel861025 for leading the effort, and collaborators from Microsoft Research @ShaleenDeep and Yiwen Zhu.
📄 Paper: arxiv.org/abs/2607.12747
Arun Iyer retweeted
How can LLMs learn new skills from offline data without losing what they already know?
Introducing Subspace Tuning (SST): a simple approach that improves the learning–forgetting trade-off by controlling where weight updates happen.
Paper: arxiv.org/abs/2610.22293
Blog: app.notion.com/p/ac-notes/To…
🧵 1/8
Arun Iyer retweeted
New on our Frontier Red Team blog: using Claude to formally verify software at large scale.
Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time).
With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks.
Read our research in full here: anthropic.com/research/forma…
Arun Iyer retweeted
Recursive self-improvement is being hyped to the moon and back. I wanted to know how good frontier models actually are at a less-studied piece of it: curating pretraining data.
So I built PretrainGym at the @PrimeIntellect RL Residency 🧵
justin-chae.org/blog/pretrai…
Arun Iyer retweeted
How do we scale up the context of LLMs?
Agentic search scales to arbitrarily large amounts of info, but can easily miss relevant info.
Idea💡: Optimize organization of info for downstream utility. Leads to 30+% gains even on OOD domains
Paper + code: librarian-agents.github.io/
Arun Iyer retweeted
Here’s the second update in our series on sample-efficient post-training!
Prior work on on-policy distillation has proposed heuristics for estimating token importance. We take a principled, theoretically grounded approach, deriving token utility from a local trust-region formulation and Fisher information geometry.
Check out the short explainer video for a quick overview!
Which tokens matter most in on-policy distillation?
Excited to share our work at AWS AI Labs with @vijaylingam08 and Yilan Chen. Our Fisher Utility Weighting (FUW) derives token importance from expected loss reduction, connecting Fisher geometry to variance & χ² divergence.
📈 Up to +4.0 accuracy points
🎥 2-min explainer
fisher-utility-weighting-opd…
Arun Iyer retweeted
1/9 Current optimizers are static during training. What if the optimizer could think and react while the network trains?
We let an agent watch the training dynamics and rewrite the optimizer's source code as training goes: a reactive optimizer.
In every language and image setting we tried that has no tuned recipe, it matches or beats the best recipe one can one-shot, and it guards against loss spikes and overfitting by itself. In the near future, optimization decisions may belong to agents, not humans.
📝 xingjianbai.com/reactive-opt…
💻 Try it on your task: github.com/xingjian-bai/opal
🧵
Arun Iyer retweeted
Models love reward hacking. Maybe we should just let them!
In our new paper, we let models reward hack during RL but we dictate which parameters learn this behavior. When we disable these parameters for deployment, the model keeps its new capabilities but doesn't hack!
Arun Iyer retweeted
Introducing RoPE Profiler: a lightweight tool that diagnoses long-context weaknesses and recommends targeted interventions to address them without retraining.
RoPE Profiler identifies specific semantic and positional weaknesses and guides targeted interventions to address them.
It builds on our improved theory of RoPE’s semantic-positional tradeoff in long contexts that accounts for unequal strengths across learned frequency components. We also establish a context-length bound beyond which avoiding failures in both is impossible.
On established long-context benchmarks including RULER2, LongBench, and BABILong, RoPE Profiler’s diagnoses guide training-free interventions and achieves best observed accuracy gains of 20% on Qwen3-8B and 25% on Llama-3.1-8B.
Although this one and our prior work (Du et al., arxiv.org/abs/2605.15514) establish intrinsic long-context limits of RoPE and variants, our new theory reveals substantial room for improvement through the targeted diagnosis and interventions offered by RoPE Profiler.
RoPE Profiler has little overhead and zero additional model forward passes and reuses query/key activations. Connect it to a Hugging Face Trainer in one call or integrate it into your own evaluation loop.
Read the paper for the theory and results, and try RoPE Profiler with our code and quick start:
Paper: arxiv.org/abs/2609.39929
Code: github.com/acetocarmine11/ro…
Check out our latest work:
📄 "ROPE at the End of Its Rope? Theory, Diagnosis, and Mitigation of Long-Context Failures" (Great honor to collaborate w/ @yufengd4 & @haopeng_uiuc ! 🤝)
🔗 Paper: arxiv.org/abs/2609.39929
💻 Code: github.com/acetocarminell/ro…
Arun Iyer retweeted
This summer at @awscloud AI Labs, we explored several problem statements around sample-efficient post-training. This is the first post in a series sharing some of the directions we investigated.
Arun Iyer retweeted
We’re launching the Anthropic Cyber Mission, a new effort to secure critical infrastructure and open-source software.
anthropic.com/news/anthropic…
