miru (@miru_why) header
miru (@miru_why) profile picture

3e-4x engineer, unswizzled wagmi. specialization is for warps

Joined January 2024
Photo by miru (@miru_why)Photo by miru (@miru_why)Photo by miru (@miru_why)Photo by miru (@miru_why)Photo by miru (@miru_why)Photo by miru (@miru_why)Photo by miru (@miru_why)Photo by miru (@miru_why)Photo by miru (@miru_why)Photo by miru (@miru_why)
know the difference
8
30
4
399
18,728
gudok.xyz/transpose/ how to transpose a matrix, efficiently, on x86_64 cpu
14
927
Reviving ConvNeXt for Efficient Convolutional Diffusion Models github.com/star-kwon/FCDM arxiv.org/abs/2603.09408v1 the authors propose an improved convnext-based diffusion model architecture that reportedly matches DiT-XL/2 quality with 7x fewer training steps
2
16
4
140
21,106
LoL: Longer than Longer, Scaling Video Generation to Hour the authors fix the 'video resets to initial frame' (sink collapse) problem in long streaming video generation, by jittering RoPE phases across heads so they never unintentionally sync up arxiv.org/abs/2601.16914v1
1
7
445
Scalable GANs with Transformers arxiv.org/abs/2509.24935v1 hse1032.github.io/GAT authors train latent-space transformer GANs up to XL/2 scale, and report SotA 1-step class-conditional image generation results on ImageNet-256 after 40 epochs (*with REPA in discriminator)
2
34
2
216
17,473
Weighted Conditional Flow Matching the authors improve flow-matching training by downweighting poorly-matched noise/image pairs, and show that this cheap reweighting produces clean, straight flow paths (like what you'd get from optimal transport) arxiv.org/pdf/2507.22270
1
1
24
1,586
We'd love our flow-based generative models to learn the optimal transport from noise to data... but they rarely do ❌. Mini-batch Optimal Transport methods aim to fix this — but they're costly and require large batch sizes to work well... Can we approximate this behaviour cheaply and effectively? 💡 In our new preprint, we introduce 𝐖𝐞𝐢𝐠𝐡𝐭𝐞𝐝 𝐂𝐨𝐧𝐝𝐢𝐭𝐢𝐨𝐧𝐚𝐥 𝐅𝐥𝐨𝐰 𝐌𝐚𝐭𝐜𝐡𝐢𝐧𝐠 🌀 — a simple and intuitive way to approximate 𝑒𝑛𝑡𝑟𝑜𝑝𝑖𝑐 𝑜𝑝𝑡𝑖𝑚𝑎𝑙 𝑡𝑟𝑎𝑛𝑠𝑝𝑜𝑟𝑡 behaviour without ever explicitly solving OT. w/ M. Meunier*, @AlvaroCartea, @C__Reisinger, @yaringal, @jmhernandez233 🔗 arxiv.org/pdf/2507.22270 🧵(1/🧵)
3
1,011
anyone know a paper where they do RLVR strictly on next token prediction task? like ilya's detective novel analogy, doing reasoning rollouts and rewarding the reasoning chains that correctly deduce the criminal's name?
2
9
1,343
anonymous whistleblower from noah's ark lab has posted a writeup detailing the sad saga of pangu - they support honestagi's claim that pangu MoE 72B was plagiarized from qwen 2.5 14B - they also witnessed similar plagiarism from qwen 1.5 110B, deepseek-v3 github.com/HW-whistleblower/…
9
20
7
212
77,934
english translation of huawei whistleblower's pangu writeup, 1/2
2
1
30
4,772
english translation of huawei whistleblower's pangu writeup, 2/2. all translation thanks to gemini, with minor edits and [annotations] from discussion, hopefully we did it justice
1
22
3,958
one day in and @giffmana is already fixing annoyances in pytorch main
Rumor has it you hated pytorch so much you joined meta to fix it from the source yourself, LOL
5
1
2
141
24,123
pytorch transpose vs numpy transpose. baffling
6
5
4
101
35,534
μTransfer in action
1
1
21
1,987
interesting paper on ‘any-subset’ auto-regressive modeling without the standard product-rule factorization arxiv.org/abs/2504.20456v1 github.com/gabeguo/any-order… their model can sample from the true joint distribution with 10% fewer NFEs (i.e. speculative decoding with no extra model)
9
711
Boosting Generative Image Modeling via Joint Image-Feature Synthesis arxiv.org/pdf/2504.16064v1 the authors concatenate normal VAE latents with PCA’d DINOv2 embeddings, and find that diffusion models trained on this joint distribution achieve lower FID than VAE-only or VAE+REPA
4
22
3
166
22,870
ReDi author thread with more information nitter.gq/thkouz/status/19156639…
1/n Introducing ReDi (Representation Diffusion): a new generative approach that leverages a diffusion model to jointly capture – Low-level image details (via VAE latents) – High-level semantic features (via DINOv2)🧵
2
748
PixelFlow: Pixel-Space Generative Models with Flow github.com/ShoufaChen/PixelF… arxiv.org/abs/2504.07963 the authors train a pixel space image generator with gradually-increasing spatial resolution across timesteps, and release 1B-scale class- and text-conditional checkpoints
21
77
6,222
turns out the AI CUDA Engineer achieved 100x speedup by… hacking the eval script
This Post is from an account that no longer exists. Learn more
75
231
104
3,015
509,139
sakana have updated their leaderboard to address the memory-reuse exploit sakana.ai/ai-cuda-engineer/#… there is only one >100x speedup left, on task 23_Conv3d_GroupNorm_Mean in this task, the AI CUDA Engineer forgot the entire conv part and the eval script didn’t catch it
6
13
12
251
56,929
sakana is now working on a more comprehensive effort to fix all eval script exploits/loopholes discovered by the AI CUDA Engineer and reevaluate their technique. happy to see it and hope they succeed nitter.gq/sakanaailabs/status/18…
Replying to @SakanaAILabs
Update: Combining evolutionary optimization with LLMs is powerful but can also find ways to trick the verification sandbox. We are fortunate to have readers, like @main_horse test our CUDA kernels, to identify that the system had found a way to “cheat”. For example, the system had found a memory exploit in the evaluation code which, in a number of cases, allowed it to avoid checking for correctness. Furthermore, we find the system could also find other novel exploits in the benchmark’s tasks. We have since made the evaluation and runtime profiling harness more robust to eliminate many of such loopholes. We are in the process of revising our paper, and our results, to reflect and discuss the effects, and mitigation of LLM reward hacking for CUDA kernel optimization. We deeply apologize for our oversight to our readers. We will provide a revision of this work soon, and discuss our learnings.
1
1
43
5,489