Projection Matrix

Definition

$P=A(A^{⊺}A{)}^{-1}{A}^{⊺}$

For some vector $\mathbf{b}$, $P\mathbf{b}$ is the projection of $\mathbf{b}$ onto $A$

Facts

The projection matrix $P$ is symmetric Idempotent Matrix

Consider a Symmetric Matrix $P$, then $P$ is idempotent with rank $r$ if and only if $r$ eigenvalues are $1$ and $n-r$ eigenvalues are $0$.

$\mathrm{tr}(P)=\mathrm{rank}(P)$

The projection matrix is Positive Semi-Definite Matrix

If $P_1$ and $P_2$ are projection matrices, and $P_1-P_2$ is Positive Semi-Definite Matrix, then

• $P_1{P}_2=P_2{P}_1=P_2$
• $P_1-P_2$ is a projection matrix.