Principal Component Regression

Definition

${\hat{\mathbf{y}}}_{\text{PCR}}=\hat{\beta }\mathbf{Z}$ where ${\mathbf{Z}}_{n\times M}(M\le p)$ is a matrix whose columns are ${\mathbf{z}}_i:=\mathbf{X}{\mathbf{v}}_i=d_i{\mathbf{u}}_i$. In PCR, instead of regressing the dependent variables on explanatory variables directly, principal components(PC) of the explanatory variables are used as regressors. One typically uses only first a few PCs for regression, making PCR a kind of regularized procedure.

Facts

Since ${\mathbf{z}}_m$ are linear combination of the original ${\mathbf{x}}_j$, we can express the solution in therms of coefficients of the ${\mathbf{x}}_j$.