Wrapping up my internship at @Apple MLR. It was a great experience to work with the team on agent training.
We have explored how MoE architectures behave in agentic post-training, and a general framework to steer the expert selection, to align with agents' operations. Specifically, we find that off-the-shelf MoE models tend to re-use experts for the same operation, while different operations are more likely to be accomplished by diverging experts. Our framework strengthens this operation-expert association, and achieves significant improvement on both task performance and inference efficiency.
arxiv.org/abs/2610.07332
Huge thanks to my mentors @cydhsieh and Ting-Yao Hu, as well as all collaborators in the team: Raviteja Vemulapalli, Sanjoy Chowdhury, @OncelTuzel.
Right now, I am going to defend my PhD by the end of 2026, and am searching for full-time research scientist roles in the industry. Feel free to message me if you find me to be a good fit for your team.
Bolian Li retweeted
Researchers working on vision-language-action models eventually run into the same deeply frustrating wall.
GRPO is great for ditching the critic, but binary rewards waste a ridiculous amount of data.
When early rollouts all fail, GRPO gets zero advantage spread and just throws the entire batch in the garbage, lighting expensive compute and simulation budget on fire.
The team at Purdue and AWS AI dropped a paper addressing this exact bottleneck with Prism-GRPO, and i feel the fix is brilliant.
Instead of dumping degenerate, same outcome batches, they rank attempts using execution quality signals like motion smoothness and contact dynamics.
Even in a batch of total failures, the model learns which mistake was less catastrophic, all while maintaining strict mathematical boundaries so actual successes still outrank any failure.
They achieved up to a 56% reduction in rollouts needed to reach target success rates, with noticeably cleaner policy transfer onto physical hardware.
It’s a very practical and high-signal idea that makes us wonder why we are letting so much rollout data go to waste in the first place.
Definitely worth a full read. Read the full paper here: arxiv.org/pdf/2608.17423v1
Bolian Li retweeted
🚨Registration is now open for MMLS!
Register here (Deadline June 14): web.cvent.com/event/c740da3e…
We also offer:
🏠 Free on-campus housing (Deadline June 5): forms.gle/2eEbsn6gGkgcKRXMA
✈️ Student travel award (Deadline June 5): forms.gle/6MUJg4U5V2caDMKu9
The Midwest Machine Learning Symposium (MMLS) 2026 will happen at Purdue University!
📍 West Lafayette, IN
📅 June 24–25, 2026
🔗 midwest-ml.org/2026/
📌 Poster submission deadline: May 24
We have an amazing lineup of plenary speakers: Tong Zhang, Jennifer Neville @ProfJenNeville, Mohit Bansal @mohitban47, Joyce Chai.
Looking forward to seeing you there!
@PurdueCS @PurdueECE @PurdueStats
Scaling up RL training with more data often encounters the performance saturation, which wastes compute.
We find that a precisely crafted entropy curve is all you need to avoid performance saturation, and we achieve this purely by rejection sampling.
huggingface.co/papers/2604.2…
Bolian Li retweeted
Self-correction is an essential ingredient for robust reasoning on complex multimodal tasks. Yet in reinforcement learning, the learning signals of effective self-correction are notoriously sparse.
Octopus breaks the signal bottleneck:
Where signals are sparse, we synthesize.
👀 We indicate that self-correction behaviors can be synthesized by exploiting contrastive signals already present in standard RL rollouts through pairing correct and incorrect reasoning trajectories.
🐙 We introduce Octopus, a rollout augmentation framework that constructs dense, explicit self-correction examples in RL. It improves sample efficiency via rollout reuse and stabilizes optimization by balancing positive and negative examples.
🚀 Octopus-8B achieves SOTA performance among models of comparable size across 7 benchmarks. It surpasses the strongest RLVR baseline by 1.0 average accuracy points while requiring only 0.72× training time per step.
We fully open-source Octopus, including RLVR baselines' checkpoints initialized from Qwen3-VL-8B across different rollout numbers.
Paper: arxiv.org/pdf/2602.08503
Project Page: dripnowhy.github.io/Octopus/
Checkpoints: huggingface.co/collections/T…
Bolian Li retweeted
Excited to share our ICLR 2026 paper.
👉 Stop asking users for feedback. Learn from the data you already have.
Explicit feedback (👍👎, surveys) is noisy and rare.
But implicit dissatisfaction is everywhere: retries, corrections, “no that’s wrong”.
Bolian Li retweeted
🚀Excited to present VERA at #NeurIPS2025!
We show that variational inference can systematically jailbreak aligned LLMs, uncovering failure modes that heuristic attacks miss and offering a deeper view into model vulnerabilities.
📍Thu Dec 4, 11am–2pm
VERA: arxiv.org/abs/2506.22666
I will be at San Diego during @NeurIPSConf . Actively looking for research internship for 2026.
linkedin.com/in/bolian-li-55…
4
203
Bolian Li retweeted
Continuous diffusion dominates images but fails on discrete data—despite learning continuous gradients that should enable coordinated updates.
"CANDI: Hybrid Discrete-Continuous Diffusion Models" explains why and how why hybrid diffusion fixes it! (1/8)
Can we accelerate test-time alignment?
YES!
📃paper: Reward-Shifted Speculative Sampling Is An Efficient Test-Time Weak-to-Strong Aligner
🔗arXiv: arxiv.org/abs/2508.15044
📌EMNLP 2025
Bolian Li retweeted
Excited to give a talk on Oct 14 about Gradient-Based Discrete Sampling!
How can we bring the power of Langevin dynamics to discrete spaces?
I’ll discuss algorithms like Discrete Langevin and its extensions for multimodal distributions and combinatorial optimization, with applications to graphical models, LLMs, and drug design.
Come join if you’re interested in sampling, inference, or generative modeling!
🎙️ Monte Carlo Seminar — Tue, Oct 14, 2025
Speaker: Ruqi Zhang (Purdue University)
Title: Gradient-Based Discrete Sampling: Algorithms and Applications
Time: 8:30 AM PT / 11:30 AM ET / 4:30 PM London / 5:30 PM Paris
Zoom: us06web.zoom.us/s/8238833260…
Welcome to our poster on efficient LLM alignment #COLM2025 #64. My advisor @ruqi_zhang is here.
Bolian Li retweeted
Purdue IPAI is hiring postdocs in AI! If you're interested in statistical machine learning or trustworthy AI, and would like to work with me, please get in touch! Applications are due by Sept 1, 2025.
purdue.edu/computes/institut…
Finally, CARDS is accepted by @COLM_conf – at Seattle, WA
Can we be MUCH FASTER and still get well-aligned text in decoding-time alignment? YES, WE CAN!
Thrilled to introduce CARDS, a segment-level sampling algorithm to efficiently generate aligned text. 5x faster and 99% win-ties compared to previous methods.
arxiv.org/abs/2406.16306
2
121
Bolian Li retweeted
Excited to see our chapter out! A concise and accessible introduction to Bayes Compute in deep neural networks and deep generative models. Great for statisticians curious about diving in!
A little chapter that we (@ruqi_zhang and awesome students and yours truly) wrote a while ago to give a brief intro of this nice field to statisticians 😊
Bolian Li retweeted
🚀 Thrilled our work is accepted at ICLR2025!
We mitigate safety challenges in VLMs with ETA, a plug-and-play inference-time method that minimizes trade-off between safety and general performance (helpfulness and inference time).
Big thanks to @lblaoke and @ruqi_zhang !
Bolian Li retweeted
Introducing ETA, an inference-time alignment method for VLM safety:
✨Improve safety without reducing VLM power—reducing unsafe rate by 87.5% in cross-modality attacks and achieving 96.6% win-ties in GPT-4 helpfulness evaluation
✨Induce only a slight increase in inference time
✨Requires no additional data or training, offering a plug-and-play solution for VLMs
