Chanwoo Park (@chanwoopark20) header
Chanwoo Park (@chanwoopark20) profile picture

Multi-agent (gen) AI | @mit EECS Ph.D. Candidate | Amazon AI PhD Fellowship | @speedrun SR003

Boston
Joined November 2021
Photo by Chanwoo Park (@chanwoopark20)Photo by Chanwoo Park (@chanwoopark20)Photo by Chanwoo Park (@chanwoopark20)Photo by Chanwoo Park (@chanwoopark20)Photo by Chanwoo Park (@chanwoopark20)Photo by Chanwoo Park (@chanwoopark20)Photo by Chanwoo Park (@chanwoopark20)Photo by Chanwoo Park (@chanwoopark20)Photo by Chanwoo Park (@chanwoopark20)Photo by Chanwoo Park (@chanwoopark20)
Fun to see the multi-agent "training" paradigm going mainstream. Played a bit with Grok 4.2 — really cool to see this direction commercialized. It runs 4 identical (same base) models that debate (or collaborate) to solve a problem. But the key isn’t just orchestration — it has to be trained that way. Back in 2024 (pre–DeepSeek era), I was already training identical agents with MARL to reason via structured multi-agent interaction. Multi-agent system should be trained simultaneously and properly.
2
6
1
48
8,644
I love seeing this kind of back-and-forth: using social media not just to promote research, but to get useful feedback and make the work stronger. Jaeyeon is a great researcher!
Since releasing Priming Models, we’ve received great questions. Here’s a short follow-up with new experiments and release updates! TL;DR: Diffusion policy setup / (# real samples) vs. Final FID / T2i checkpoints released / new blog post from @moritz_piening!
2
1
12
1,193
Now, lets not do backpropagation, but Astra-propagation
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math. rikiyat.github.io/blog/weight-sp… github.com/RikiyaT/lm_und… huggingface.co/Qwen/Qwen3.5-0… huggingface.co/RikiyaT/Qwen3.…
2
9
816
We see a lot of cool robot demos, but where are the people? Didn't we want robots that live and work with us? The missing data: how people naturally interact. We present Wearable-Free Human Real2Sim2Real, and demonstrate that the interactive abilities transfer well from human, to simulation, to real G1. Reported in CoRL 2026 Late-Breaking Results 🙌 affordance.homes/lbr Meet you in Texas 🤠
7
15
3
69
6,582
Chanwoo Park retweeted
Foundation models are trained in stages: pretraining, midtraining, and post-training. But is this separation fundamental? We introduce unified training: a single algorithm for the full pipeline. This gives ⚙️ Simpler infrastructure 🔬 A framework for better science (1/n)
54
259
51
1,599
200,674
Truely beautiful work.
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data. Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less? Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
1
2
13
1,140
Chanwoo Park retweeted
Tinker’s Dimuon is Muon with delayed diagonal conditioning, Polar Express and a rank-dependent update norm. I recovered its update rule through the API, borrowing @mrtnm’s approach to StableReinforce. Here’s how 🧵
Dimuon is a new optimizer we built for LoRA fine-tuning. In our SFT tests, it often reaches lower loss in fewer steps than AdamW. We're still experimenting with Dimuon; you can Tinker with it today and share your feedback! See our optimizers guide at tinker-docs.thinkingmachines…
14
31
2
355
20,762
Soooo cool
Surprise 1: Anthropic acquired a one-person biotech . . . Surprise 2: that person is me Surprise 3: I’m here to tell you all about it Yes, this is what you think it is; after 6 yrs of increasingly loose anonymity: the Professor Oak reveal post …and yes, it has gotten so boring around here that I’m literally doxxing myself to spice up the timeline ------- Ok, allow me to introduce myself. My name is Cody Tranbarger. I mostly tweet, but occasionally find time to dabble in drug dev & starting/building biotechs. My past lives span the VC, crossover, startup continuum; Prof Oak has been my way of keeping a digital foot in those worlds, despite having my physical feet planted elsewhere. The past 18 mo, ‘elsewhere’ was Mithril Medicine, a co. I founded to develop bsAbs for LAMA2-CMD & other ECM-related genetic diseases. Mithril was born out of hands-on building with @MikeNGladstone, @JasonPaulRhodes, @LifeSciVC, & team @atlasventure. Somehow, I made enough half-decent calls along the way (even a broken clock is right twice a day, eh?) to convince them Mithril/I warranted a small seed. While I can hardly rationalize such an outrageously cavalier bet, I’m grateful for it, & I'd partner with Atlas again in a heartbeat. In typical AV fashion, Mithril’s primordial form was virtual, decentralized, & ultra-lean. This stage tends to be short-lived, but for Mithril, it worked surprisingly well, for surprisingly long, thanks in part to the rapid maturation of the AI tools I used liberally. Eventually, Mithril became a bit of an experiment itself - to find the upper bound of how far it could scale as a single operator + a bunch of Claude (+ a stellar bench of part-time consultants). In parallel, I started spending time with @nc_frey, @an1lam, & @CoefficientBio team; and promptly experienced, for the 1st time, AI at the frontier. Soon thereafter, Mithril stopped looking like an experiment, & started looking like a preview of an entirely new operating model. From that spark came “Pipeline-in-a-Person.” The idea immediately resonated when we shared it here back in April; and it was central to the shared vision that led to Anthropic’s acquisition of Mithril, & my joining the team. Why join Anthropic? Three reasons. (1) For Mithril, the programs continue, with more resources & infrastructure behind them, and as much Claude as I can put to work. (2) For me, the chance to start several more programs, & find out how many one person can actually run in parallel - an almost unprecedented luxury for a curious drug hunter. (3) For our mission, a path to institutionalize & scale this new way of working, then put it in the hands of all of biopharma, all over the world - which, candidly, I find the most energizing. Will AI x Bio cure diseases? Unequivocally yes, imo… but not by the route that gets the most airtime. Rather than timeline acceleration & compression, I see orchestration & parallelization as today's low-hanging fruit. Most drug hunters see a handful of go/no-go calls in a career, & spend entirely too much of the rest trading emails (my inbox can attest). Give them the bandwidth to run several programs at once, and the time to focus on scientific judgment, and I’m willing to bet they’ll make better calls. And at scale, better calls translate to more drugs, higher hit rates, broader patient impact, & likely, a disease or 10 or 100 cured. I couldn't be more excited to build toward that future @AnthropicAI. Finally, a heartfelt thank you to BioTwitter/X. The end of anonymity is bittersweet, but if anyone’s earned a proper introduction, you all have. It’s difficult to overstate how much this app has done for me: a decade of education, plenty of friends, many job opportunities, several mentors, a few investors, even a co-founder (fmr). It was the generosity of this community that inspired me to start posting 6 years ago, after just as many years lurking. I hope I’ve been able to pay it forward, in my own small way. But, to be clear, Prof Oak isn’t going anywhere. Same beat, same themes, and now, a front-row seat to drug development at the frontier. Stay tuned for dispatches from the trenches; and remember... Gotta Cure 'Em All
658
Chanwoo Park retweeted
Surprise 1: Anthropic acquired a one-person biotech . . . Surprise 2: that person is me Surprise 3: I’m here to tell you all about it Yes, this is what you think it is; after 6 yrs of increasingly loose anonymity: the Professor Oak reveal post …and yes, it has gotten so boring around here that I’m literally doxxing myself to spice up the timeline ------- Ok, allow me to introduce myself. My name is Cody Tranbarger. I mostly tweet, but occasionally find time to dabble in drug dev & starting/building biotechs. My past lives span the VC, crossover, startup continuum; Prof Oak has been my way of keeping a digital foot in those worlds, despite having my physical feet planted elsewhere. The past 18 mo, ‘elsewhere’ was Mithril Medicine, a co. I founded to develop bsAbs for LAMA2-CMD & other ECM-related genetic diseases. Mithril was born out of hands-on building with @MikeNGladstone, @JasonPaulRhodes, @LifeSciVC, & team @atlasventure. Somehow, I made enough half-decent calls along the way (even a broken clock is right twice a day, eh?) to convince them Mithril/I warranted a small seed. While I can hardly rationalize such an outrageously cavalier bet, I’m grateful for it, & I'd partner with Atlas again in a heartbeat. In typical AV fashion, Mithril’s primordial form was virtual, decentralized, & ultra-lean. This stage tends to be short-lived, but for Mithril, it worked surprisingly well, for surprisingly long, thanks in part to the rapid maturation of the AI tools I used liberally. Eventually, Mithril became a bit of an experiment itself - to find the upper bound of how far it could scale as a single operator + a bunch of Claude (+ a stellar bench of part-time consultants). In parallel, I started spending time with @nc_frey, @an1lam, & @CoefficientBio team; and promptly experienced, for the 1st time, AI at the frontier. Soon thereafter, Mithril stopped looking like an experiment, & started looking like a preview of an entirely new operating model. From that spark came “Pipeline-in-a-Person.” The idea immediately resonated when we shared it here back in April; and it was central to the shared vision that led to Anthropic’s acquisition of Mithril, & my joining the team. Why join Anthropic? Three reasons. (1) For Mithril, the programs continue, with more resources & infrastructure behind them, and as much Claude as I can put to work. (2) For me, the chance to start several more programs, & find out how many one person can actually run in parallel - an almost unprecedented luxury for a curious drug hunter. (3) For our mission, a path to institutionalize & scale this new way of working, then put it in the hands of all of biopharma, all over the world - which, candidly, I find the most energizing. Will AI x Bio cure diseases? Unequivocally yes, imo… but not by the route that gets the most airtime. Rather than timeline acceleration & compression, I see orchestration & parallelization as today's low-hanging fruit. Most drug hunters see a handful of go/no-go calls in a career, & spend entirely too much of the rest trading emails (my inbox can attest). Give them the bandwidth to run several programs at once, and the time to focus on scientific judgment, and I’m willing to bet they’ll make better calls. And at scale, better calls translate to more drugs, higher hit rates, broader patient impact, & likely, a disease or 10 or 100 cured. I couldn't be more excited to build toward that future @AnthropicAI. Finally, a heartfelt thank you to BioTwitter/X. The end of anonymity is bittersweet, but if anyone’s earned a proper introduction, you all have. It’s difficult to overstate how much this app has done for me: a decade of education, plenty of friends, many job opportunities, several mentors, a few investors, even a co-founder (fmr). It was the generosity of this community that inspired me to start posting 6 years ago, after just as many years lurking. I hope I’ve been able to pay it forward, in my own small way. But, to be clear, Prof Oak isn’t going anywhere. Same beat, same themes, and now, a front-row seat to drug development at the frontier. Stay tuned for dispatches from the trenches; and remember... Gotta Cure 'Em All
533
1,067
388
14,428
1,770,480
I'm posting a series of prediction questions from Stanford CS312 (Deep Learning Alchemy) exams to test deep learning intuition. First up: optimization. If we increase an LM's batch size 8× (same token budget), can rescaling learning rate or weight decay compensate? Try below ↓ This quiz is part of our optimizer hyperparameter assignment and unit designed by @wen_kaiyue Setup: We train an 8-layer, ~35M-parameter Transformer on 614.4M tokens using AdamW. At batch size 64, the best reference run has peak LR 0.003, weight decay (WD) 0.2, and final validation loss 2.9178. Now increase the batch size from 64 to 512, which gives 8× fewer optimizer steps. Everything else stays fixed except the changes below. Which recipe gives the lowest final validation loss? A. Keep LR and WD unchanged B. LR ×√8 C. LR ×8 D. WD ×8 E. LR ×√8 and WD ×√8 What would you predict, and why?
13
82
10
917
86,578
One thing I wasn't sure about before joining Mistral was whether I'd enjoy training models as much as having the freedom to choose my own research problems. And you know what, freedom is overrated, there's so much to figure out, and I'm having a lot more fun right now.
5
2
149
6,886
Chanwoo Park retweeted
Tinker's new StableReinforce objective is Reinforce + Score Centering. Here's how I reverse-engineered it 🧵
Off-policy RL is difficult; runs can suddenly collapse because of async RL staleness or numerical mismatch. StableReinforce is a new method on Tinker that keeps your RL runs stable, even when learning drifts from sampling. tinker-docs.thinkingmachines…
23
104
20
1,118
150,668
Chanwoo Park retweeted
Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:
134
796
155
6,580
888,347
Chanwoo Park retweeted
We got scooped by this great paper, and I’m not even mad: they did a wonderful job and chose a much better title than I had in mind. We have StopGrads everywhere and casually gloss over them as if we’re still doing gradient descent. Meanwhile, our parameters follow a vector field that may not even be the gradient of any loss function. For flow-map learning, they explicitly solve the idealized training dynamics and show convergence to the right solution, despite the absence of an underlying loss. Really great work by @maxwshen, @marikgoldstein and the team! arxiv.org/abs/2609.16222
6
20
2
273
28,112
Chanwoo Park retweeted
1/9 Current optimizers are static during training. What if the optimizer could think and react while the network trains? We let an agent watch the training dynamics and rewrite the optimizer's source code as training goes: a reactive optimizer. In every language and image setting we tried that has no tuned recipe, it matches or beats the best recipe one can one-shot, and it guards against loss spikes and overfitting by itself. In the near future, optimization decisions may belong to agents, not humans. 📝 xingjianbai.com/reactive-opt… 💻 Try it on your task: github.com/xingjian-bai/opal 🧵
79
208
72
1,604
224,895
Previously, Stepfun's tech report for Step 3.5-Flash impressed me with their telemetry culture. Very underappreciated paper, model, company.
Replying to @eliebakouch
quite fun that stepfun also release a similar activation gradient cliping in their blog today (also love how they use it as a diagnostic tool, very cool)
2
7
92
5,494
mHC and AttnRes demonstrate that richer depth mixing improves model performance at scale. Is there a unifying perspective? 🤔 Meet our Depth Memory framework: each layer reads from a set of streams to form its input, then writes back its output. Linear Depth Memory (LDM) equips models with a matrix-valued memory.
8
35
16
239
52,439
Chanwoo Park retweeted
here is an overview of the architecture of all the major american open weight models lots of similarities: sliding window (except mamba for nvidia), latent moe or a variant from muse, and a LOT of norms everywhere (muse has 9 norms per layer if i counted right!!) let's focus on muse, it's the only model in this list without shared experts. they are also doing different residuals with some form of gate (still single stream), and are more conservative in the hybrid ratio (3:1) and larger sliding window (2k). there are not a lot of details on the training procedure but you can notice there are a bunch of "muP like" multipliers including one for the router with a 1/sqrt(nb of experts) factor 👀 super interesting to see this quite different approach to scaling between american and chinese open weight labs, different constraints lead to different architectures
1/ muse spark open is the most capable US open-weight model to date. 816B total params, 42B active. 1M-token context. natively multimodal. built for coding and agentic work like long-horizon tasks, hard software engineering, multi-step reasoning. america! 🇺🇲
15
59
9
642
30,215
Chanwoo Park retweeted
JUST IN: Elon Musk's Tesla $TSLA officially unveils new flying Roadster.
1,462
3,288
2,879
34,883
18,922,934