Cross Validation

Definition

Partition a set of data into $K$ sets $I_j,j=1,2,\dots ,K$ and denote a function $\kappa :\{1,2,\dots ,n\}\to \{1,2,\dots ,K\}$ such that $\kappa (i)=j\iff i\in I_j$. For the dataset $\{({\mathbf{X}}_1,Y_1),({\mathbf{X}}_2,Y_2),\dots ,({\mathbf{X}}_n,Y_n)\}$, let ${\hat{f}}_{-j}$ be the estimator based on observations except $I_j$, then the cross validation estimation for the prediction error is defined as ${\widehat{PE}}_{CV}(\hat{f})=\frac{1}{n}\sum _{i=1}^{n}L(Y_i,{\hat{f}}_{-\kappa (i)}({\mathbf{X}}_i))$ where $L$ is a loss function

Facts

If the data is partitioned into $k$ group with equal size, then it is called a $k$-fold cross validation.