NeRF-Neural Radiance Field

2020년 8월에 arxiv 투고. ECCV oral. project page, github(pytorch-facebook, tensorflow-colab있음-official)

1. Abstract

연속적인 volumetric scene을 만든다. sparse input views를 이용하여. FC layer만 사용. input이 5차원(공간 3차원($x, y, z$) + 보는 방향 2차원($\theta$, $\phi$)), output : volume density and view-dependent emitted radiance at that spatial location.(4차원$RGB\sigma$) Volume rendering은 differentiable해야 한다. 외형적으로 복잡한것도 잘 된다고 함.

3. Neural Radiance Field Scene Representation

• MLP :

$$
F_{\Theta} : (\boldsymbol{x},\boldsymbol{d})\rightarrow(\boldsymbol{c},\sigma)
$$

논문 인용 :

To accomplish this, the MLP FΘ first processes the input
3D coordinate x with 8 fully-connected layers (using ReLU activations and 256
channels per layer), and outputs σ and a 256-dimensional feature vector. This
feature vector is then concatenated with the camera ray’s viewing direction and
passed to one additional fully-connected layer (using a ReLU activation and 128
channels) that output the view-dependent RGB color.

4. Volume Rendering with Radiance Fields.

논문에 integration을 discrete sum으로 만드는 방법이 자세하게 되어있음. 그것보다 그냥 아래 그림 보는게 더 좋을듯.

5. Optimizing a Neural Radiance Field

$xyz\theta\phi$를 그냥 바로 쓰지 않음. $F_{\Theta}=F'_{\Theta}\circ\gamma$. 아래를 써서 매우매우 큰 향상을 얻음.

$$
\gamma(p) = (\sin(2^0 \pi p),\cos(2^0 \pi p), \sin(2^1 \pi p),\cos(2^1 \pi p), \cdots, \sin(2^L \pi p),\cos(2^L \pi p))
$$

where $p\in {x,y,z,\theta,\phi}$. For $x,y,z$, $L=10$. For $\theta, \phi$, $L=4$.
위 $\gamma$를 통해 $[-1,1]$로 normalize함.

위와 관련된 내용, 논문 인용 :

This is consistent with recent work by Rahaman
et al. [35], which shows that deep networks are biased towards learning lower frequency functions. They additionally show that mapping the inputs to a higher
dimensional space using high frequency functions before passing them to the
network enables better fitting of data that contains high frequency variation.

---

내 발표자료 : 링크

• 해야할 일 : 논문 속 [35] 인용 논문 읽기.