LeCun Initialization

Definition

LeCun initialization is designed for Neural Network using the Tanh activation function.

For Normal Distribution: $W\sim N\left(0,\frac{1}{n_{\text{in}}}\right)$ For Uniform Distribution $W\sim U\left(-\sqrt{\frac{1}{n_{\text{in}}}},\sqrt{\frac{1}{n_{\text{in}}}}\right)$ where $n_{\text{in}}$ is the number of input nodes.