Research Scientist @AdobeResearch | Ph.D. @ CUHK | Prev. Research Intern @AdobeResearch | Photographer | Opinions are my own
California, USA
Joined November 2019
- Tweets262
- Following527
- Followers794
- Likes2.1K
Pinned Tweet
Excited to share Chimera!
As Kimi K3 brings hybrid linear attention into the spotlight, our work explores the same architectural transition from the visual side: adapting KDA-based hybrid attention to token-extensive visual generation, together with a systematic scaling recipe.
Interested in how frontier labs pre-train image/video generation models?
We were too.
Since those recipes are rarely made public in full, we started from the most mature pretraining playbook available in the open: how modern LLMs are built.
Introducing Chimera: a visual generation model family that brings LLM-style hybrid linear attention and scaling co-design to visual generation.
In large-scale pretraining, nearly every design choice eventually shows up.
That means solving architecture design and scaling as one coupled problem: every architectural choice changes how the model scales, and scaling behavior determines which choices actually survive.
Chimera approaches both jointly, building a model family that remains predictable as model size, compute budget, and data distribution change.
Our key architectural observation is a simple division of labor: a single raster-ordered KDA stream carries long-range state, while modality-aware short convolutions preserve native local geometry.
Together, they form an effective and elegant linear-attention backbone for multidimensional visual data, with periodic MLA providing direct global interaction and sparse MoE expanding capacity at controlled activated compute.
This design comes with a useful effect: NoPE.
In Chimera, position is represented by the computation itself. KDA’s ordered recurrence and learned state decay encode order and recency, while the short convolutions encode local spatial-temporal structure.
Explicit positional embeddings are not needed in our design.
Because these mechanisms are not tied to a fixed training grid or sequence length, Chimera shows strong zero-shot extrapolation in both space and time.
Trained exclusively on 1K images and 5-second videos, it directly generates coherent 4K images with little visible quality degradation and 30-second videos with only 6.5% FID degradation over the final five seconds, all zero-shot, without resolution- or length-specific finetuning.
But architecture alone is not a pretraining recipe unless it scales predictably.
Thus, scaling should not be treated as an afterthought: fitting a curve over model sizes is easy; making that curve meaningful is much harder.
If every model size is differently under-tuned, your scaling law may simply be measuring optimization error.
We propose HeteroP to transfer proxy-tuned hyperparameters module by module across width and depth, giving us a consistently tuned model family. This allows us to fit Chinchilla-style laws over activated model size, training tokens, and the image-video data mixture.
The laws not only provide the reference for compute-optimal model and data size, but also suggest that visual generation may be more model-hungry than we tend to assume.
Under the same parametric loss-fitting method used in Chinchilla, compute-optimal model size grows as FLOPs^0.516 for images, compared with FLOPs^0.46 for language. Video is even more model-hungry, with the exponent rising from 0.516 for images to 0.544 for video.
Guided by these laws, we trained an 11B-parameter Chimera that activates only 2B parameters per token, using ~600 H100 days.
- It matches Wan-2.1 2B pretraining loss with 7.3x fewer FLOPs, and runs 2.14x faster than full attention at 255K tokens.
- It matches FLUX.1-dev and Z-Image-Turbo on GenEval and outperforms both on DPG-Bench, using roughly 20× less training compute than Z-Image-Turbo.
Ultimately, Chimera indicates that once you pretrain at scale, every decision shows up.
And if the Kimi K3 recipe caught your attention, Chimera may look oddly familiar, except the tokens are pixels and frames with diffusion models.
A team effort from team @ChongjianG30781 , me, @VisionSteve , Jiuxiang Gu, @Xu_Arthas , @chenziwee , @ShaotengLiu , @Jingorz , @YicongHong , @Zefan_Cai , @HaoTan5 ; supported by Hailin Jin and @kalyank_s at @Adobe @AdobeResearch
Tianyu (Steve) Wang retweeted
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
When an image-editing agent fails, does it really need another try?
Introducing our new paper: EditAgent: Rethinking Agentic Image Editing.
We argue that editing agents are bounded by two spaces:
🔧 Action space: what the agent can do.
🔍 Observation space: what its judge can see.
Three findings:
1️⃣ Expanding the action space matters.
Across 600 expert-annotated candidate sets, FLUX.2-klein-9B produced no acceptable candidate in 42.7%. Specialized methods rescued 175 of those 256 failures.
2️⃣ Expand what the judge can observe.
EditAgent enriches the observation space with parallel candidates for comparison, task-specific verification evidence, and earlier images and requests as historical context—supporting better candidate selection and detecting when a new edit undoes an earlier one.
3️⃣ Explore diverse alternatives through parallel search.
Different specialists, generalists, and repeated samples offer different ways to fulfill a request. Parallel search lets the judge compare these alternatives, while generating each candidate from the same source avoids carrying errors from one attempt into the next.
Even with a single editor and the same four-generation budget, experts preferred parallel search over sequential refinement in 78.5% of non-tied comparisons—with at most half as many judge calls.
EditAgent expands both spaces and connects them through parallel candidate generation and comparative judging. It improves two editing backbones on three benchmarks, and its accepted sessions provide training data that strengthens standalone editors.
Our takeaway: building reliable editing agents requires expanding both what they can do and what their judges can observe.
For more details, please check our paper or project page:
📄 Paper: arxiv.org/abs/2610.39664
🌐 Project: editagent.github.io/
Special thanks to our intern @HaiyangXu3110 for the creativity, hard work, and ability to turn ideas into results—always bringing fresh perspectives and following through with strong execution. Huge thanks as well to all our colleagues @JialuLi96 @JitengMu, Mingze Xu for the insightful discussions and support across so many aspects of this work, from brainstorming ideas to building the benchmark and evaluation.
Tianyu (Steve) Wang retweeted
We build a hierarchy in semantic space: text → semantic → pixel. AR learns what to draw; diffusion learns how to draw it.
Like LLMs: pretrain for knowledge, post-train for behavior.
Know, then Draw: A New Learning Paradigm for Visual Generation 🧵
arxiv.org/abs/2610.37626
Tianyu (Steve) Wang retweeted
[1/n] 作为这场训练“投毒”事件最直接的受害者，2024年那几个月漫长的训练排查、对峙与质疑，对我而言是一段炼狱般的经历，也是一场对意志和技术的锤炼。
感谢好友们最艰难的时候支持陪伴。也感谢当时所有并肩作战的同事们。
Tianyu (Steve) Wang retweeted
🪄 Introducing OmniXRef: A Unified Framework for Multimodal Multi-Reference Video Editing
🧙 Any number of references, any modality, any attribute, any combination. OmniXRef transfers each into one coherent edit. (1/6)
Tianyu (Steve) Wang retweeted
HERE IT IS. The Next Generation Roadster developed by SpaceX and Tesla!
There is a fan-suction system at the bottom of the car that can pull up to 2 gs allowing it to stick upside down.
It just flipped UPSIDE DOWN with cold gas thrusters and landed. HOLY SHIT.
Tianyu (Steve) Wang retweeted
新时代做research太累了， 任何解决”简单“问题的paper都会在两个月内过时。祝愿大家都能找到重要问题 把时间花在重要的问题上
What is the role of academic computer vision research in the age of increasingly powerful large models? Is GPT-6 Astra a step change? How can a researcher have an impact today in academia?
These are the questions I ask myself as I head off to ECCV 2026, a conference I’ve attended since 1992. One of my papers this year is VIGA, a method that takes an image as input and outputs a 3D Blender scene that represents that image. This is a classical inverse-graphics task and VIGA was the first method to solve it using an agentic approach.
The idea is now several years old and the first version of the paper was rejected. This delayed publication significantly. After it was accepted at ECCV, it was quickly surpassed by people using Claude Code for the same purpose. Today GPT-6 Astra blows away all previous results. But we still head off to ECCV to tell the community about our invention that is now fully out of date.
The way academic work often progresses is that one reads recent papers, notices that they have limitations, comes up with a new idea, explores this, publishes it, etc. Any published paper I read today is based on ideas that are at least a year old. And those ideas were based on the literature of the time, which was also a year old. That means that any paper I see at ECCV is likely two years out of date. In AI today, two years means your work is likely irrelevant.
At CVPR this summer I noticed that many authors have not gotten the message. They continue to work on “old” problems that have a long history. This history is based on assumptions about how the “vision problem” will be “solved”. The truth is that it is being solved in a very different way and many of these problems are no longer relevant. Another group of papers focuses on very niche problems where large models likely fail because of insufficient data or lack of business interest. The impactful papers were largely from industry and had long author lists and massive data+compute behind them. These papers were also out of data, describing systems that had been released months before, but at least they served to provide the community with more complete documentation and analysis of commercial systems.
So what should academics do? First, we need to put aside the tools we’ve used for years and start from scratch. Every project should start by trying really hard to solve the problem with existing tools. I would like to see every paper begin with a detailed experimental analysis of how existing models perform and why they fail (if they do). This gives the kind of insight we need today. Then, assuming current models fail, the solution should provide some fundamental insight that will outlive the next release of such models.
Reviewers today still focus on technical novelty. This pushes people to focus on tweaking architectures rather than clearly moving the field forward. Papers need to be judged based on their novel insight and not their novel technical contribution. This is a real shift in thinking but it focuses us on what matters - progress of the field.
If we want there to be a “field” of computer vision, then it can’t become a marginal backwater, focusing on esoteric problems. If you haven’t tried using Astra (or whatever comes next) to solve your problem, then you have not done your homework. This omission should be seen as negatively as not having a previous work section.
Concretely, I think papers should include a new section analogous to “Related Work” where that related work is current models and how they perform on the task. Reviewers should start asking for this and expecting authors to be able to articulate their insights about the limitations of existing large models.
I'm interested in your thoughts.
Tianyu (Steve) Wang retweeted
I recently completed a research project through a highly automated collaboration with Codex. The division of labor was simple: Codex maximized exploration, while I focused on judgment.
In detail, Codex proposed hypotheses, implemented and debugged methods, ran experiments, analyzed results, and iterated from feedback. I set the research direction, chose the questions, inspected the evidence, and decided which ideas were worth pursuing.
This is also a concrete example of the research workflow I discussed in my previous post. I believe it is particularly effective when a problem is close to purely logical and each hypothesis can be verified with modest compute. Agents can then explore at high throughput, while cheap verification enables rapid iteration.
The result is my new paper, which asks a question: what limits length extrapolation in recurrent models, and how can we learn recurrent dynamics that remain effective beyond the training horizon?
More specifically, recurrent models are coming back not only because they are efficient. Their shared update rule provides a structurally natural path to length extrapolation. Yet learning recurrent dynamics that remain reliable over substantially longer horizons is still an open problem.
To study this problem, I build a controlled task suite that separates state tracking, memory retrieval, and their interaction. My diagnostics show that vanishing parameter gradients (previous focus) alone do not explain extrapolation failure. A fixed transition rule can still be learned from dense local supervision despite severe gradient decay. The harder failure occurs when future losses cannot transmit sufficient credit to earlier recurrent states that must preserve useful information.
This suggests intervening before the parameter gradient is formed. I call the gradient of future losses with respect to an earlier recurrent state as state credit. It is the upstream learning signal: a future loss must first reach the state before it can train the recurrent update.
This motivates Credit Stabilization through Time (CST) to act directly on this signal. During backward propagation, it restores the norm of state credit when contraction accumulates across sequence chunks, while preserving its direction and leaving the forward computation unchanged.
The results support this hypothesis across two regimes:
Synthetic tasks
• CST significantly improves length extrapolation across all task families and evaluated settings, with gains verified at up to 128× the training length.
Real language modeling (still toy)
• I train a 123M-parameter recurrent language model at 4K and evaluate it up to 128K on both in-domain and out-of-domain data.
• CST produces a modest improvement over all tokens, but a substantially larger gain on key tokens that require long-range context.
• The benefit generally increases with extrapolation length.
Learning Length-Extrapolatable Recurrent Models
Paper: arxiv.org/abs/2609.09157
This is my first paper centered on model architecture design and language modeling, and also my first attempt at this kind of highly automated research workflow. So comments and corrections are genuinely welcome. Hope you find both the paper and workflow make sense :)
Tianyu (Steve) Wang retweeted
Thanks so much for sharing! @_akhaliq 🥰
The core idea is to learn how the world evolves, rather than directly predict what the future looks like. LDR maps past frames into structured latent states and rolls them forward with kinematic integration, learning only the higher-order motion residuals.
To our knowledge, this is the first video world model that learns the underlying dynamics purely from pixels and extrapolates them beyond the training distribution.
- Paper: huggingface.co/papers/2608.0…
- Code: github.com/adobe-research/LD…
- Model: huggingface.co/haodongli/LDR
- Data: huggingface.co/datasets/haod…
Learning How the World Evolves
Extrapolative Video World Models via Latent Dynamics Reasoning
paper: huggingface.co/papers/2608.0…
Tianyu (Steve) Wang retweeted
Learning How the World Evolves
Extrapolative Video World Models via Latent Dynamics Reasoning
paper: huggingface.co/papers/2608.0…
Tianyu (Steve) Wang retweeted
Thanks so much for sharing! @_akhaliq 🥰
Paper / code / models / data are now public, check them out! 🚀
- Paper: huggingface.co/papers/2608.0…
- Code: github.com/adobe-research/LD…
- Model: huggingface.co/haodongli/LDR
- Data: huggingface.co/datasets/haod…
Learning How the World Evolves
Extrapolative Video World Models via Latent Dynamics Reasoning
paper: huggingface.co/papers/2608.0…
Tianyu (Steve) Wang retweeted
Excited to share Chimera! My favorite part is mShortConv: a simple way to tell the model that text is 1D, images are 2D, and videos are 3D—even after they are flattened into the same sequence. Together with KDA, this enables a single scan and NoPE. Proud of the team!
Interested in how frontier labs pre-train image/video generation models?
We were too.
Since those recipes are rarely made public in full, we started from the most mature pretraining playbook available in the open: how modern LLMs are built.
Introducing Chimera: a visual generation model family that brings LLM-style hybrid linear attention and scaling co-design to visual generation.
In large-scale pretraining, nearly every design choice eventually shows up.
That means solving architecture design and scaling as one coupled problem: every architectural choice changes how the model scales, and scaling behavior determines which choices actually survive.
Chimera approaches both jointly, building a model family that remains predictable as model size, compute budget, and data distribution change.
Our key architectural observation is a simple division of labor: a single raster-ordered KDA stream carries long-range state, while modality-aware short convolutions preserve native local geometry.
Together, they form an effective and elegant linear-attention backbone for multidimensional visual data, with periodic MLA providing direct global interaction and sparse MoE expanding capacity at controlled activated compute.
This design comes with a useful effect: NoPE.
In Chimera, position is represented by the computation itself. KDA’s ordered recurrence and learned state decay encode order and recency, while the short convolutions encode local spatial-temporal structure.
Explicit positional embeddings are not needed in our design.
Because these mechanisms are not tied to a fixed training grid or sequence length, Chimera shows strong zero-shot extrapolation in both space and time.
Trained exclusively on 1K images and 5-second videos, it directly generates coherent 4K images with little visible quality degradation and 30-second videos with only 6.5% FID degradation over the final five seconds, all zero-shot, without resolution- or length-specific finetuning.
But architecture alone is not a pretraining recipe unless it scales predictably.
Thus, scaling should not be treated as an afterthought: fitting a curve over model sizes is easy; making that curve meaningful is much harder.
If every model size is differently under-tuned, your scaling law may simply be measuring optimization error.
We propose HeteroP to transfer proxy-tuned hyperparameters module by module across width and depth, giving us a consistently tuned model family. This allows us to fit Chinchilla-style laws over activated model size, training tokens, and the image-video data mixture.
The laws not only provide the reference for compute-optimal model and data size, but also suggest that visual generation may be more model-hungry than we tend to assume.
Under the same parametric loss-fitting method used in Chinchilla, compute-optimal model size grows as FLOPs^0.516 for images, compared with FLOPs^0.46 for language. Video is even more model-hungry, with the exponent rising from 0.516 for images to 0.544 for video.
Guided by these laws, we trained an 11B-parameter Chimera that activates only 2B parameters per token, using ~600 H100 days.
- It matches Wan-2.1 2B pretraining loss with 7.3x fewer FLOPs, and runs 2.14x faster than full attention at 255K tokens.
- It matches FLUX.1-dev and Z-Image-Turbo on GenEval and outperforms both on DPG-Bench, using roughly 20× less training compute than Z-Image-Turbo.
Ultimately, Chimera indicates that once you pretrain at scale, every decision shows up.
And if the Kimi K3 recipe caught your attention, Chimera may look oddly familiar, except the tokens are pixels and frames with diffusion models.
A team effort from team @ChongjianG30781 , me, @VisionSteve , Jiuxiang Gu, @Xu_Arthas , @chenziwee , @ShaotengLiu , @Jingorz , @YicongHong , @Zefan_Cai , @HaoTan5 ; supported by Hailin Jin and @kalyank_s at @Adobe @AdobeResearch
Paper: arxiv.org/abs/2607.28611
Tianyu (Steve) Wang retweeted
Interested in how frontier labs pre-train image/video generation models?
We were too.
Since those recipes are rarely made public in full, we started from the most mature pretraining playbook available in the open: how modern LLMs are built.
Introducing Chimera: a visual generation model family that brings LLM-style hybrid linear attention and scaling co-design to visual generation.
In large-scale pretraining, nearly every design choice eventually shows up.
That means solving architecture design and scaling as one coupled problem: every architectural choice changes how the model scales, and scaling behavior determines which choices actually survive.
Chimera approaches both jointly, building a model family that remains predictable as model size, compute budget, and data distribution change.
Our key architectural observation is a simple division of labor: a single raster-ordered KDA stream carries long-range state, while modality-aware short convolutions preserve native local geometry.
Together, they form an effective and elegant linear-attention backbone for multidimensional visual data, with periodic MLA providing direct global interaction and sparse MoE expanding capacity at controlled activated compute.
This design comes with a useful effect: NoPE.
In Chimera, position is represented by the computation itself. KDA’s ordered recurrence and learned state decay encode order and recency, while the short convolutions encode local spatial-temporal structure.
Explicit positional embeddings are not needed in our design.
Because these mechanisms are not tied to a fixed training grid or sequence length, Chimera shows strong zero-shot extrapolation in both space and time.
Trained exclusively on 1K images and 5-second videos, it directly generates coherent 4K images with little visible quality degradation and 30-second videos with only 6.5% FID degradation over the final five seconds, all zero-shot, without resolution- or length-specific finetuning.
But architecture alone is not a pretraining recipe unless it scales predictably.
Thus, scaling should not be treated as an afterthought: fitting a curve over model sizes is easy; making that curve meaningful is much harder.
If every model size is differently under-tuned, your scaling law may simply be measuring optimization error.
We propose HeteroP to transfer proxy-tuned hyperparameters module by module across width and depth, giving us a consistently tuned model family. This allows us to fit Chinchilla-style laws over activated model size, training tokens, and the image-video data mixture.
The laws not only provide the reference for compute-optimal model and data size, but also suggest that visual generation may be more model-hungry than we tend to assume.
Under the same parametric loss-fitting method used in Chinchilla, compute-optimal model size grows as FLOPs^0.516 for images, compared with FLOPs^0.46 for language. Video is even more model-hungry, with the exponent rising from 0.516 for images to 0.544 for video.
Guided by these laws, we trained an 11B-parameter Chimera that activates only 2B parameters per token, using ~600 H100 days.
- It matches Wan-2.1 2B pretraining loss with 7.3x fewer FLOPs, and runs 2.14x faster than full attention at 255K tokens.
- It matches FLUX.1-dev and Z-Image-Turbo on GenEval and outperforms both on DPG-Bench, using roughly 20× less training compute than Z-Image-Turbo.
Ultimately, Chimera indicates that once you pretrain at scale, every decision shows up.
And if the Kimi K3 recipe caught your attention, Chimera may look oddly familiar, except the tokens are pixels and frames with diffusion models.
A team effort from team @ChongjianG30781 , me, @VisionSteve , Jiuxiang Gu, @Xu_Arthas , @chenziwee , @ShaotengLiu , @Jingorz , @YicongHong , @Zefan_Cai , @HaoTan5 ; supported by Hailin Jin and @kalyank_s at @Adobe @AdobeResearch