Zihui (Ray) Wu retweeted
🚨Our new #ICML2025 paper is featured by MIT News!
news.mit.edu/2025/unpacking-…
Why are LLMs lost in the middle? Why do transformers systematically favor tokens at certain positions in a sequence? We had vague intuitions but no rigorous understandings, until now!
We develop a *graph-theoretic framework* to characterize the individual contributions of 1) attention mask and 2) position encoding to the rise of position bias in attention.
All together, we are able to provably derive many well-known position bias phenomena in LLMs: lost-in-the-middle, attention sinks, how NoPE learns position, and more.
Here is the intuition behind. 🧵
Zihui (Ray) Wu retweeted
💻 Code & Data
Modular, extensible codebase—plug in your own PnPDP methods or new inverse problems!
👉 GitHub: github.com/devzhk/InverseBen….
Zihui (Ray) Wu retweeted
TLDR: if you can train video diffusion priors, then solving video inverse problems can be done much simpler and more accurately than heavily engineered approaches based on image diffusion priors.
Examples below for black hole video reconstruction and dynamic MRI.
Zihui (Ray) Wu retweeted
It may be necessary to align human behavior with AI feedback.
#NeurIPS2024
Zihui (Ray) Wu retweeted
"Hopfield joined Caltech as faculty in 1980 and, two years later, published his seminal paper in which he applied principles of the brain to computer circuits, creating a neural network able to hold memory and recognize patterns."
latimes.com/science/story/20…
As Yu’s mentee and close friend, I first met him at Prof. @ukmlv’s class; he was a TA. His sharp insights on complicated math concepts deeply impressed me. I am sure that he has demonstrated his academic excellence throughout his PhD as well. Huge congrats on Dr. Sun @YuSunMark!
This quoted post is unavailable.
2
9
Yu was my PhD mentor for my two great years at CIG. Prof. Kamilov and Yu led me from a rookie for computation imaging to a passionate researcher in the field, for which I have always been grateful. I am very glad that Yu and I will soon have a chance to work together again!
1
1
Zihui (Ray) Wu retweeted
Check out the fun projects we are working on in my computational cameras group at @Caltech! computationalcameras.org/
The best paper of #ML4H2021 was awarded to SeqMRI, which learns to sequentially select future measurements based on previous samples to improve accelerated MRI.
Authors: Tianwei Yin, @ZihuiRayWu, @_sun_he, @AdrianDalca, @yisongyue, @klbouman
Website: imaging.cms.caltech.edu/seq-…
Zihui (Ray) Wu retweeted
This year’s #ML4H2021 best paper was awarded to SeqMRI, which learns how to sequentially select future measurements based on previously collected samples to improve accelerated MRI.
Authors: Tianwei Yin, @ZihuiRayWu, @_sun_he, @AdrianDalca, @yisongyue, and @klbouman









