Zihui (Ray) Wu (@ZihuiRayWu) header
Zihui (Ray) Wu (@ZihuiRayWu) profile picture

Ph.D. student at Caltech working with Prof. Katie Bouman

Joined March 2020
Photo by Zihui (Ray) Wu (@ZihuiRayWu)
Zihui (Ray) Wu retweeted
🚨Our new #ICML2025 paper is featured by MIT News! news.mit.edu/2025/unpacking-… Why are LLMs lost in the middle? Why do transformers systematically favor tokens at certain positions in a sequence? We had vague intuitions but no rigorous understandings, until now! We develop a *graph-theoretic framework* to characterize the individual contributions of 1) attention mask and 2) position encoding to the rise of position bias in attention. All together, we are able to provably derive many well-known position bias phenomena in LLMs: lost-in-the-middle, attention sinks, how NoPE learns position, and more. Here is the intuition behind. 🧵
7
26
3
77
25,801
Zihui (Ray) Wu retweeted
TLDR: if you can train video diffusion priors, then solving video inverse problems can be done much simpler and more accurately than heavily engineered approaches based on image diffusion priors. Examples below for black hole video reconstruction and dynamic MRI.
4
13
86
8,727
Zihui (Ray) Wu retweeted
It may be necessary to align human behavior with AI feedback. #NeurIPS2024
6
46
10
565
37,074
Zihui (Ray) Wu retweeted
"Hopfield joined Caltech as faculty in 1980 and, two years later, published his seminal paper in which he applied principles of the brain to computer circuits, creating a neural network able to hold memory and recognize patterns." latimes.com/science/story/20…
30
1
78
7,901
As Yu’s mentee and close friend, I first met him at Prof. @ukmlv’s class; he was a TA. His sharp insights on complicated math concepts deeply impressed me. I am sure that he has demonstrated his academic excellence throughout his PhD as well. Huge congrats on Dr. Sun @YuSunMark!
This quoted post is unavailable.
2
9
Yu was my PhD mentor for my two great years at CIG. Prof. Kamilov and Yu led me from a rookie for computation imaging to a passionate researcher in the field, for which I have always been grateful. I am very glad that Yu and I will soon have a chance to work together again!
1
1
I feel fortunate for his future students and I am convinced that he will be as great a mentor as Prof. Kamilov has been for our CIG members. Congratulations on this milestone for CIG as well!
1
Zihui (Ray) Wu retweeted
Check out the fun projects we are working on in my computational cameras group at @Caltech! computationalcameras.org/
5
25
1
384
The best paper of #ML4H2021 was awarded to SeqMRI, which learns to sequentially select future measurements based on previous samples to improve accelerated MRI. Authors: Tianwei Yin, @ZihuiRayWu, @_sun_he, @AdrianDalca, @yisongyue, @klbouman Website: imaging.cms.caltech.edu/seq-…
2
5
1
27
Zihui (Ray) Wu retweeted
This year’s #ML4H2021 best paper was awarded to SeqMRI, which learns how to sequentially select future measurements based on previously collected samples to improve accelerated MRI. Authors: Tianwei Yin, @ZihuiRayWu, @_sun_he, @AdrianDalca, @yisongyue, and @klbouman
1
2
2
9