Definite Matrix

Kinds

Positive-Definite Matrix

Definition

$M\succ 0\iff \forall \mathbf{z}\ne 0,{\mathbf{z}}^{⊺}M\mathbf{z}>0$

Matrix $M$, in which ${\mathbf{z}}^{⊺}M\mathbf{z}$ is positive for every non-zero column vector $\mathbf{z}$ is a positive-definite matrix

Facts

Let $M\succ 0$, then

• $A^{-1}\succ 0$
• $\mathrm{rank}(CA{C}^{⊺})=\mathrm{rank}(C)$ where $C$ is a $p\times n$ matrix.
• $\mathrm{rank}(C)=p\Rightarrow CM{C}^{⊺}\succ 0$
• The diagonal elements of $M$ are positive.
• For a Symmetric Matrix $B$, $\exists t,M-tB\succ 0$ for sufficiently small $|t|$
• $\forall \mathbf{b},\underset{\mathbf{h}:\mathbf{h}\ne 0}{\sup }\frac{({\mathbf{h}}^{⊺}\mathbf{b}{)}^2}{{\mathbf{h}}^{⊺}M\mathbf{h}}={\mathbf{b}}^{⊺}{A}^{-1}\mathbf{b}$
• $\exists M^{1/2}\succ 0\text{s.t.}(M^{1/2}{)}^2=M$

$M\succ 0\iff \exists R\text{s.t.}A=R{R}^{⊺}$ where $R$ is a non-singular matrix.

$M\succ 0\iff$ all the leading minor determinants of $M$ are positive.

$\mathrm{rank}(X)=p\Rightarrow X^{⊺}X\succ 0$

Link to original

Positive Semi-Definite Matrix

Definition

$M⪰0\iff \forall \mathbf{z}\ne 0,{\mathbf{z}}^{⊺}M\mathbf{z}\ge 0$

Matrix $M$, in which ${\mathbf{z}}^{⊺}M\mathbf{z}$ is positive or zero for every non-zero column vector $\mathbf{z}$ is a positive semi-definite matrix

Facts

Let $M⪰0$, then

• $\mathrm{tr}(A)\ge 0$
• $\mathrm{rank}(A)=r\iff \exists R_{n\times n},\mathrm{rank}(R)=r,A=R{R}^{⊺}$
• $\mathrm{rank}(A)=r\Rightarrow \exists S_{n\times r},\mathrm{rank}(s)=r,S^{⊺}AS=I_r$
• $X^{⊺}AX=O\Rightarrow AX=O$ where $X$ is $n\times p$ matrix.

Link to original

Negative-Definite Matrix

Definition

$M\prec 0\iff \forall \mathbf{z}\ne 0,{\mathbf{z}}^{⊺}M\mathbf{z}<0$

Matrix $M$, in which ${\mathbf{z}}^{⊺}M\mathbf{z}$ is negative for every non-zero column vector $\mathbf{z}$ is a negative-definite matrix

Link to original

Facts

Every positive (semi)-definite matrix is Hermitian Matrix

Positive (Semi)-definite and Symmetric Matrix

• $M$ is positive definite $\iff$ all eigenvalues of $M$ are real and positive.
• $M$ is positive semi definite $\iff$ all eigenvalues of $M$ are real and positive or zero.

where $M$ is Hermitian Matrix

Proof

$(\Leftarrow )$ A Hermitian Matrix $\mathbf{M}$ is orthogonally diagonalizable as $\mathbf{P\Lambda }{\mathbf{P}}^{⊺}$¹ Therefore, ${\mathbf{z}}^{⊺}\mathbf{Mz}={\mathbf{z}}^{⊺}\mathbf{P\Lambda }{\mathbf{P}}^{⊺}\mathbf{z}={\mathbf{x}}^{⊺}\mathbf{\Lambda x}={\sum }_{i=1}^n{x}_i^2{\lambda }_i>0$

$(\Rightarrow )$ A Hermitian Matrix $\mathbf{M}$ is orthogonally diagonalizable as $\mathbf{p\Lambda }{\mathbf{p}}^{⊺}$¹ where $\mathbf{p}$ is one of the eigenvectors of $\mathbf{M}$

The $\mathbf{M}$ can be replaced by the corresponding eigenvalue $\lambda$^[By definition of Eigendecomposition] $\mathbf{pM}{\mathbf{p}}^{⊺}=\mathbf{p}\lambda {\mathbf{p}}^{⊺}$

Since matrix of eigenvectors $\mathbf{P}$ is orthogonal¹, $\mathbf{p}\lambda {\mathbf{p}}^{⊺}=\lambda$ $\mathbf{p}\lambda {\mathbf{p}}^{⊺}>0$ for every $\mathbf{p}$^[By definition of positive definite matrix] Therefore, $\mathbf{pM}{\mathbf{p}}^{⊺}=\mathbf{p}\lambda {\mathbf{p}}^{⊺}=\lambda >0$ holds. In other words, every eigenvalue is non-zero.

Footnotes

1. By Spectral Theorem ↩ ↩² ↩³