Eigendecomposition

Definition

$\mathbf{Av}=\lambda \mathbf{v}$ where $\mathbf{A}$ is a $n\times n$ matrix and $\mathbf{v}\ne 0$

If $\mathbf{Av}$ is a scalar multiple of non-zero vector $\mathbf{v}$, then $\lambda$ is the eigenvalue and $\mathbf{v}$ is the eigenvector.

Characteristic Polynomial

$|\mathbf{A}-\lambda \mathbf{I}|=0$ The values ${\lambda }_i$ satisfying the characteristic polynomial, are the eigenvalues of the matrix $\mathbf{A}$

Eigenspace

$E=\{\mathbf{v}:(\mathbf{A}-\lambda \mathbf{I})\mathbf{v}=0\}$

The set of all eigenvectors of $A$ corresponding to the same eigenvalue, together with the zero vector.

The Kernel of the matrix $(\mathbf{A}-\lambda \mathbf{I})$

Eigenvector: non-zero vector in the eigenspace

Algebraic Multiplicity

${\mu }_A({\lambda }_i)$

Let ${\lambda }_i$ be an eigenvalue of an $n\times n$ matrix $A$. The algebraic multiplicity ${\mu }_A({\lambda }_i)$ of the eigenvalue is its multiplicity as a root of a Characteristic Polynomial, that is, the largest k such that $(\lambda -{\lambda }_i{)}^k$

Geometric Multiplicity

${\gamma }_A({\lambda }_i)$ Let ${\lambda }_i$ be an eigenvalue of an $n\times n$ matrix $A$. The geometric multiplicity ${\gamma }_A({\lambda }_i)$ of the eigenvalue is the dimension of the Eigenspace associated with the eigenvalue.

Computation

1. Find the solution^[eigenvalues] of the Characteristic Polynomial.
2. Find the solution^[eigenvectors] of the Under-Constrained System $(\mathbf{A}-{\lambda }_i\mathbf{I}){\mathbf{v}}_i=0$ using the found eigenvalue.

Facts

There exists at least one eigenvector corresponding to the eigenvalue $\lambda$

Eigenvectors corresponding to different eigenvalues are always linearly independent.

When $\mathbf{A}$ is a normal or real Symmetric Matrix, the eigendecomposition is called Spectral Decomposition

$\mathrm{tr}(\mathbf{A})={\sum }_{i=1}^n{\lambda }_i$ The Trace of the matrix is equal to the sum of the eigenvalues of the matrix.

Proof Since the matrix $\mathbf{A}-\lambda \mathbf{I}$ only has $\lambda$ term on diagonal, and the calculation of cofactor deletes a row and a column, the coefficient of ${\lambda }^{n-1}$ of $\text{det}(\mathbf{A}-\lambda \mathbf{I})$ is always $-(a_{11}+a_{22}+\cdots +a_{nn})$. Also, since ${\lambda }_i$ are the solution of the Characteristic Polynomial, the expression is factorized as $\text{det}(\mathbf{A}-\lambda \mathbf{I})=(\lambda -{\lambda }_1)(\lambda -{\lambda }_2)\dots (\lambda -{\lambda }_n)$ and the coefficient of ${\lambda }^{n-1}$ become a $-({\lambda }_1+{\lambda }_2+\cdots +{\lambda }_n)$ Therefore, $-(a_{11}+a_{22}+\cdots +a_{nn})=-({\lambda }_1+{\lambda }_2+\cdots +{\lambda }_n)$ and $\text{Trace}(\mathbf{A})={\sum }_{i=1}^n{\lambda }_i$.

$|\mathbf{A}|={\prod }_{i=1}^n{\lambda }_i$ The Determinant of the matrix is equal to the product of the eigenvalues of the matrix.

Not all matrices have $n$ linearly independent eigenvectors.

When $\mathbf{Av}=\lambda \mathbf{v}$ holds,

• the eigenvalues of ${\mathbf{A}}^{-1}$ are $\frac{1}{\lambda }$ and the eigenvectors of the matrix ${\mathbf{A}}^{-1}$ are the same as $\mathbf{A}$.
• the eigenvalues of $A^k$ is ${\lambda }^k$
• the eigenvalues of $c\mathbf{A}$ is $c\lambda$
• the eigenvalues of $\mathbf{A}+c\mathbf{I}$ is $\lambda +c$
• the eigenvalues of $(\mathbf{A}+c\mathbf{I}{)}^{-1}$ is $1/(\lambda +c)$

$1\le {\gamma }_A({\lambda }_i)\le {\mu }_A({\lambda }_i)\le n$

An eigenvalue’s Geometric Multiplicity cannot exceed its Algebraic Multiplicity

$\exists P\text{ s.t. }A=PD{P}^{-1}\iff \forall {\lambda }_i:{\gamma }_A({\lambda }_i)={\mu }_A({\lambda }_i)$

the matrix is diagonalizable $\iff$ the Geometric Multiplicity is equal to the Algebraic Multiplicity for all eigenvalues

Let $A$ be a symmetric matrix. Then, $A$ has $r$ eigenvalues equal to $1$ and the rest zero $\iff A^2=A,\mathrm{rank}(A)=r$.

Link to original

The non-zero eigenvalues of $AB$ are the same as those of $BA$.