YJW

Congratulations to the authors of this lovely work that just won a best paper award at @siggraph #siggraph2023. They achieve state-of-the-art visual quality with real-time (≥ 100 fps) novel-view synthesis at 1080p resolution. Far exceeding NeRF approaches on both quality and speed. How? They use anisotropic Gaussian kernel interpolation/splatting. Such kernels also have a history being very useful in computational imaging (e.g. super-resolution, noise reduction, etc.) The literature on Radiance Field Rendering has finally come full circle: in the original NeRF formulation, the “neural” part referred to a small multi-layer perceptron (MLP) which, in turn, corresponds to an underlying interpolation kernel. This aspect hasn’t been discussed much in the NeRF literature - namely, that NeRFs aren’t much of a neural network, so much as they are kernel (interpolation) machines. So how come NeRFs haven’t been this fast and high quality so far? Because to get here, NeRFs would have to eventually stumble upon an MLP design that would yield exactly the same anisotropic Gaussians that this paper has nailed down. And these Gaussians are ideal because they are both efficient to compute and can provide interpolation quality guarantees. This paper has done a lot of things well. But perhaps most importantly, it has leapfrogged many generations of NeRF trial and error on NeRF architectures, and landed on what’s arguably the most effective underlying interpolator. That’s what progress in science and engineering is all about. repo-sam.inria.fr/fungraph/3…