Logistic Function

Definition

$\sigma (x)=\mathrm{logistic}(x)={\mathrm{logit}}^{-1}(x)=\frac{1}{1+\exp (-x)}$

The logistic function is inverse function of Logit.

Facts

Sigmoid activation function is vulnerable to vanishing gradient problem. $\frac{d}{dx}\sigma (x)=\sigma (x)(1-\sigma (x))$ The image of the derivative of the sigmoid function is $(0,0.25]$. For this reason, after passing node with sigmoid Activation Function, the gradient is decreased

Also, with the sigmoid Activation Function, if all the inputs are positive, then all the gradients also positive.