Bias-Variance Decomposition

Definition

Assume that $Y=f(X)+ϵ$ where $E(ϵ)=0$ and $\mathrm{Var}(ϵ)={\sigma }_{ϵ}^2$. Then, under squared error loss. $\begin{array}{rl}\mathrm{Err}(x_0)& =E[(Y-\hat{f}(x_0){)}^2|X=x_0]\\ & ={\sigma }_{ϵ}^2+{\mathrm{Bias}}^2(\hat{f}(x_0))+\mathrm{Var}(\hat{f}(x_0))\\ & =\text{Irreducible error}+{\text{Bias}}^2+\text{Variance}\end{array}$