Neural Radiance Fields

Definition

Neural Radiance Fields (NeRF) is a 3D scene representation and rendering model.

Architecture

NeRF represents a 3D scene as a continuous volumetric function, learned by a Neural Network. The function maps 3D coordinates and 2D viewing directions to color and density values. $F_{\Theta }:(\mathbf{x},\mathbf{d})\to (\mathbf{c},\sigma )$ where $\mathbf{x}:=(x,y,z)$ is the 3D location, $\mathbf{d}:=(\theta ,\varphi )$ is the viewing direction, $\mathbf{c}:=(r,g,b)$ is the color, and $\sigma$ is the volume density.

Volume Rendering

$C(\mathbf{r})={\int }_{t_n}^{t_f}T(t)\sigma (\mathbf{r}(t))\mathbf{c}(\mathbf{r}(t),\mathbf{d})dt$ where:

• $t_n$ and $t_f$ are near and far bounds.
• $\mathbf{r}(t)=\mathbf{o}+t\mathbf{d}$ is the camera ray through a pixel.
• $T(t)=\exp \left(-{\int }_{t_n}^t\sigma (\mathbf{r}(s))ds\right)$ denotes the accumulated transmittance along the ray from $t_n$ to $t$.

Loss Function

The loss function of NeRF model is defined as $L=\sum _{\mathbf{r}\in \mathbb{R}}||\hat{\mathbf{C}}(\mathbf{r})-\mathbf{C}(\mathbf{r})|{|}_2^2$ where $R$ is the set of rays in the training images, $\widehat{\mathbf{C}(\mathbf{r})}$ is the predicted color for poins on the ray $\mathbf{r}$, and $\mathbf{C}(\mathbf{r})$ is the gound truth color for ray $\mathbf{r}$.