Multivariate Normal Distribution

Definition

$\mathbf{X}\sim N_n(\mathbf{\mu },\Sigma )$ where $n$ is the number of dimensions, $\mathbf{\mu }$ is the vector of location parameters, and $\mathbf{\Sigma }$ is the vector of scale parameters

Standard Multivariate Normal Distribution

$\mathbf{X}\sim N_n(0,I_n)$

PDF

$f(\mathbf{z})=(2\pi {)}^{-\frac{n}{2}}\exp \left(-\frac{1}{2}{\mathbf{z}}^{⊺}\mathbf{z}\right)$

MGF

$M_{\mathbf{z}}(\mathbf{t})=\exp \left(\frac{1}{2}{\mathbf{t}}^{⊺}\mathbf{t}\right)$

Properties

PDF

$f(\mathbf{x})=(2\pi {)}^{-\frac{n}{2}}|\Sigma {|}^{-\frac{1}{2}}\exp \left(-\frac{1}{2}(\mathbf{x}-\mathbf{\mu }{)}^{⊺}{\Sigma }^{-1}(\mathbf{x}-\mathbf{\mu })\right)$

Mean

$E(\mathbf{X})=\mu$

Variance

$\mathrm{Var}(\mathbf{X})=\Sigma$

MGF

$M_{\mathbf{x}}(\mathbf{t})=\exp \left({\mathbf{t}}^{⊺}\mathbf{\mu }+\frac{1}{2}{\mathbf{t}}^{⊺}\Sigma \mathbf{t}\right)$

Affine Transformation

Let $\mathbf{X}\sim N_n(\mathbf{\mu },\Sigma )$ be a Random Variable following multivariate normal distribution, $A$ be a $m\times n$ matrix, and $\mathbf{b}$ be a $m$ dimensional vector, then $A\mathbf{X}+\mathbf{b}\sim N_m(A\mathbf{\mu }+\mathbf{b},A\Sigma A^{⊺})$

Relationship with Chi-squared Distribution

Suppose $\mathbf{X}\sim N_n(\mathbf{\mu },\Sigma )$ be a Random Variable following multivariate normal distribution, then ${\Sigma }^{-\frac{1}{2}}(\mathbf{X}-\mathbf{\mu })\sim N_n(0,I_n)$ $(\mathbf{X}-\mathbf{\mu }{)}^{⊺}{\Sigma }^{-1}(\mathbf{X}-\mathbf{\mu })\sim {\chi }^2(n)$

Facts

Let $\mathbf{X}\sim N_n(\mathbf{\mu },\Sigma )$, $C$ be an $m\times n$, and $\mathbf{d}$ be a $m$-dimensional vector, then $C\mathbf{X}+\mathbf{D}\sim N_m(C\mu +\mathbf{d},C\Sigma C^{⊺})$

Let $\mathbf{Y}\sim N_n(\mathbf{\mu },\Sigma )$, $\mathbf{Y}=(Y_1,Y_2{)}^{⊺}$, $\mathbf{\mu }=({\mu }_1,{\mu }_2{)}^{⊺}$, $\mathbf{\Sigma }=\left(\begin{array}{cc}{\Sigma }_{11}& {\Sigma }_{12}\\ {\Sigma }_{21}& {\Sigma }_{22}\end{array}\right)$, where $Y_1$ is $p\times 1$ and $Y_2$ is $(n-p)\times 1$ vectors. Then, $Y_1$ and $Y_2$ are independent $\iff {\Sigma }_{12}=O_{p\times (n-p)}$

Let $\mathbf{Y}\sim N_n(\mathbf{\mu },\Sigma )$, $\mathbf{u}=A\mathbf{Y}$, and $\mathbf{v}=B\mathbf{Y}$, where $A$ is $m\times n$, $B$ is $l\times n$ matrices. $\mathbf{u}$ and $\mathbf{v}$ are independent $\iff \mathrm{Cov}(\mathbf{u},\mathbf{v})=A\Sigma B^{⊺}=O_{m\times l}$