ELLIS PhD Student @ ETH / Google
Zurich, Switzerland
Joined July 2022
- Tweets31
- Following172
- Followers83
- Likes80
Pinned Tweet
Our recent finding on Diffusion Alignment: a reward model in pixel space can be easily transferred to score noisy diffusion latents directly — at small finetuning cost, via stitching.
This makes Faster & Better for both Training & Inference Alignment.
Meet StitchVM👇
1/
Hyojun Go retweeted
🤖 Abundant videos capture how the world works, but robot policies still rely on costly action labels to learn.
🔥 Excited to share our work at @Meta:
📢 Native Action-Prior Learning from Videos for World Action Models 📢
📄arxiv.org/pdf/2610.03391
🌐zhaochongan.github.io/NAVA-W…
🧵
Hyojun Go retweeted
I proudly share this new paper "𝗧𝗲𝘅𝘁𝘂𝗿𝗲 𝗦𝗽𝗮𝗰𝗲 𝗠𝗮𝘁𝗲𝗿𝗶𝗮𝗹 𝗗𝗶𝗳𝗳𝘂𝘀𝗶𝗼𝗻" from my NVIDIA mentors: Jacob Munkberg, Peter Kocsis (@Peter4AI), and Jon Hasselgren!
The idea is elegant: instead of generating material views in image space and fusing them, run diffusion directly in PBR material texture space. That makes the result view-consistent by design, while reusing strong pre-trained video diffusion priors to generate complete PBR materials that generalize across arbitrary geometries and UV parameterizations.
Honestly, the 8K-res PBR texture generation is the most impressive I've seen for this task. Enjoy the super-detailed lion statue in the attached video!
I'm also excited about its extension to neural materials, which connects beautifully to my own 𝙉𝙚𝙪𝙈𝙖𝙩𝙀𝙭 work with the same team on extracting neural materials from images. Two sides of the same story — pushing the boundaries of what we can do within PBR and with new representations beyond PBR.
Don't miss the stunning visuals on the project page!
🔗 Page: nvlabs.github.io/texdiffusio…
Also check out the updated 𝙉𝙚𝙪𝙈𝙖𝙩𝙀𝙭 results 🤗
🔗 Page: nvlabs.github.io/neumatex/
📢 Texture Space Material Diffusion
We generate relightable materials from photos or text using video diffusion priors directly in texture space, scaling to 8K textures.
Great project with Jacob Munkberg (@yoslber) and Jon Hasselgren!
nvlabs.github.io/texdiffusio…
Hyojun Go retweeted
Glad to see VGGRPO accepted to ECCV’26! Exciting times ahead🔥. See you all in Sweden!
🎥 Video diffusion models can create beautiful videos, but their worlds often fall apart.
🔥 Excited to share our latest work at @Google:
📢 VGGRPO: Towards World-Consistent Video Generation with 4D Latent Reward 📢
📄 arxiv.org/pdf/2603.26599
🌐 zhaochongan.github.io/projec…
🧵
Hyojun Go retweeted
based on our results under DiffusionBench, LDM converges much faster than pixel-space diffusion at 100k steps. curious why so many people are switching to pixel space, am i missing anything?
arxiv.org/abs/2606.24888
Hyojun Go retweeted
Very excited to be in Zurich and Lausanne next week - hit me up if you want to catch up!
ZurichCV#14 will be on June 30th at the
@ETH_AI_Center!
@Haithem_Turki (NVIDIA) on photorealistic world models and @osanseviero (Google DeepMind) on the development of Google's open source model family Gemma.
RSVP:
zurichai.ch/events/zurichcv-…
Hyojun Go retweeted
📢 Excited to share our new paper: SierpinskiCam!
SierpinskiCam retakes videos from new camera trajectories, even under large viewpoint changes, using Sierpinski-patterned camera-motion cues!
Project: hyelinnam.github.io/Sierpins…
Paper: arxiv.org/abs/2606.17310
🧵 More below
Hyojun Go retweeted
🚀 Excited to share my @GoogleDeepMind student researcher project: Dual-Rate Diffusion✨
⚡ A simple construction that speeds up both regular diffusion and distilled models by interleaving a heavy context encoder with a light conditional denoiser.
🧵👇
Hyojun Go retweeted
For alignment you need V, but is hard to compute. Most methods try to approximate with
1) Tweedie, which is biased
2) MC roll-outs, which is slow with high var.
Training V was often neglected since it's hard.
We beg to differ. StitchVM enables this! Led by @gohyojun3
👇
Result 4️⃣ — Training-time alignment with DRaFT & DiffusionNFT
No need for full rollouts. Just stop denoising at an intermediate step and use StitchVM's inference as the reward signal.
Now we have much faster convergence
7/
For more, check out our work 👇
@hyungjin_chung , @prunetruong , Goutam Bhat, @ZhaochongAn , @zixiangzhao_ , @DNarnhofer , @fedassa , @SergeBelongie , Konrad Schindler
paper: arxiv.org/abs/2605.19804
Page: gohyojun15.github.io/StitchV…
Hyojun Go retweeted
Want a lighter yet stronger Point Transformer? Meet LitePT ✨
LitePT is a lightweight, high-performance 3D point cloud architecture for a wide range of point cloud processing tasks. Our smallest variant LitePT-S, features 3.6× fewer parameters, 2× faster runtime and 2× lower memory footprint than PTv3, while already matching or outperforming it across a range of benchmarks.
💻Code: github.com/prs-eth/LitePT
🌐Project page: litept.github.io/
📰Paper: arxiv.org/abs/2512.13689
with Damien Robert, @jianyuan_wang , Sunghwan Hong, Jan Dirk Wegner, Christian Rupprecht, and Konrad Schindler
Hyojun Go retweeted
Combining video diffusion and 3D feedforward models by simply stiching them together in latent space - very cool idea! Make sure to check out this novel work from my collagues at Google and ETH!
🎺Meet VIST3A — Text-to-3D by Stitching a Multi-view Reconstruction Network to a Video Generator.
➡️ Paper: arxiv.org/abs/2510.13454
➡️ Website: gohyojun15.github.io/VIST3A/
Collaboration between ETH & Google with Hyojun Go, @DNarnhofer, Goutam Bhat, @fedassa, and Konrad Schindler.
Hyojun Go retweeted
Want to leverage the power of SOTA 3D models like VGGT & Video LDMs for 3D generation? Now you can! 🚀
Introducing VIST3A — we stitch pretrained video generators to 3D foundation models and align them via reward finetuning.
📄 arxiv.org/abs/2510.13454
🌐 gohyojun15.github.io/VIST3A