Partial Least Squares Regression

Definition

${\hat{\phi }}_m=\underset{\alpha }{\mathrm{argmax}}{\mathrm{Corr}}^2(\mathbf{y},\mathbf{X}a)\mathrm{Var}(\mathbf{X}\alpha )\text{subject to}||\alpha ||=1,{\alpha }^{⊺}\mathbf{S}{\hat{\phi }}_l=0,l=1,\dots ,m-1$ where $\mathbf{S}=\frac{1}{n}{\mathbf{X}}^{⊺}\mathbf{X}$ is the sample covariance matrix.

Unlike PCR maximizing $\mathrm{Var}(\mathbf{X}a)$ only, PLS finds directions maximizing both $\mathrm{Var}(\mathbf{X}a)$ and ${\mathrm{Corr}}^2(\mathbf{y},\mathbf{X}a)$