Research scientist in NVIDIA. Working in creating a world you cannot distinguish from reality. DLSS5 is my first step. Opinions are my own.
Joined July 2015
- Tweets182
- Following1.8K
- Followers604
- Likes869
I feel incredibly lucky that my first project after finishing my PhD became #DLSS5.
In my first 1:1 with @edliu1105, I created this slide to show what I believed next-generation rendering could become. For me, this was the beginning of the journey.
My North Star for rendering has always been a world indistinguishable from footage captured by a real camera. The realism upper bound for existing #DLSS is path tracing (PT). PT can faithfully simulate the scene we describe—but that scene remains an approximation of the real world. As a result, it still looks fake, plastic, far from visually realistic in most of the current gaming.
The rise of generative models opened a complementary path. Early results gave us enough signal to pursue a belief: learned real-world appearance priors could work with physically based rendering—not replace it—to move beyond this ceiling.
DLSS 5 is our first step: a generative video model that operates in real time at up to 4K while remaining grounded in the authored scene.
Getting here took years. Skepticism pushed us to clarify the idea, strengthen the system, and make the evidence more rigorous. I’m deeply grateful for the leadership of @edliu1105, @drewtao and @ctnzr, and for our team’s extraordinary execution.
I’m proud that we have demonstrated a new research and design space in graphics rendering—one that deserves much more exploration. If you work in graphics, vision, or generative models, I hope you’ll read our technical report and join us in exploring it.
We’re publishing a deeper technical report on why generation is needed for the next leap toward photorealism.
VRAM and compute limit both the scene abstraction and the number of rays we can afford. Previous DLSS reconstructs from that representation—so even perfect reconstruction is still bounded by what the scene contains.
DLSS 5 is the first DLSS to generate beyond that abstraction and break through that ceiling, while staying grounded in the game and controlled by the artist.
We also compare it with offline generative models and renderers at similar scaling cost.
research.nvidia.com/labs/adl…
If you work in #3D generation or #worldmodel, definitely take a look at this:
I-Scene is an image to 3D scene model, achieves better 3D scene-gen than #SAM3D just using limited amount of random dataset.
Do we really need massive curated 3D scene data for interactive world generation?
#SAM3D, #WorldGen say yes.
We say no.
I-Scene learns better spatial knowlesge using only 25K randomly composed instances.
🔑 Key insight:
We reprogram the instance generator to infer support, proximity, and symmetry from purely geometric cues for generating interactive scenes.
🧠 Scene-context attention
👁️ View-centric space
🧱 Random composition beats expensive curation
🌐 luling06.github.io/I-Scene-p…
💻 github.com/LuLing06/I-Scene-…
🧵 Details below [1/6]
Heading to #ICCV2025 ?
Do not miss tomorrow's DLSS4 keynote talk by @edliu1105
Time: 1:30~2:15 pm, 20th Oct
ROOM: B317
Our neural renderer learns the relationship between the low-level 3D geometries and the soft shadow, so it is agnostic to the cutout categories and can generalize to many different scenarios! (5/6)🧵
We further propose 3D-aware buffer channels to improve soft shadow rendering given only image cutouts. (4/6)🧵
This equation implies that pixel height can be used for 3D reconstruction and 3D-related lighting effects rendering, including reflections, refractions, and soft shadow rendering. (3/6)🧵
This paper presents an equation that connects the pixel height space (shengcn.github.io/SSG/) to 3D space. (2/6)🧵
Happy to share that our work PixHt-Lab was accepted by #CVPR2023. @PurdueCS
PixHt-Lab (shengcn.github.io/PixHtLab/) renders controllable realistic reflections, soft shadows, and reflections for image compositing.
Want to know how? (1/6)🧵
