Parametric ReLU

Definition

$f(x)=\max (\alpha x,x)$ where $\alpha \le 1$ is a hyperparameter

Facts

If $\alpha =0.01$, it is called a Leaky ReLU