He Initialization

Definition

He initialization is designed for Neural Network using the ReLU activation function.

For Normal Distribution: $W\sim N\left(0,\frac{2}{n_{\text{in}}}\right)$ For Uniform Distribution $W\sim U\left(-\sqrt{\frac{6}{n_{\text{in}}}},\sqrt{\frac{6}{n_{\text{in}}}}\right)$ where $n_{\text{in}}$ is the number of input nodes, and $n_{\text{out}}$ is the number of output nodes.