Xavier Initialization

Definition

Xavier initialization is designed for Neural Network using the sigmoid or Tanh activation function.

For Normal Distribution: $W\sim N\left(0,\frac{2}{n_{\text{in}}+n_{\text{out}}}\right)$ For Uniform Distribution $W\sim U\left(-\sqrt{\frac{6}{n_{\text{in}+n_{\text{out}}}}},\sqrt{\frac{6}{n_{\text{in}+n_{\text{out}}}}}\right)$ where $n_{\text{in}}$ is the number of input nodes, and $n_{\text{out}}$ is the number of output nodes.