Asymptotics for Multivariate Distributions

Definition

Convergence in Probability

${\mathbf{X}}_n\stackrel{P}{\to }\mathbf{X}\iff (\forall ϵ>0,\underset{n\to \infty }{\lim }P(||{\mathbf{X}}_n-\mathbf{X}||\ge ϵ)=0)$

Let $\{{\mathbf{X}}_n\}$ be a Sequence of random vectors and $\mathbf{X}$ be a Random Vector, then ${\mathbf{X}}_n$ converges in probability to $\mathbf{X}$ if $\forall ϵ>0,\underset{n\to \infty }{\lim }P(||{\mathbf{X}}_n-\mathbf{X}||\ge ϵ)=0$, and denoted by ${\mathbf{X}}_n\stackrel{P}{\to }\mathbf{X}$

Let $\{{\mathbf{X}}_n\}$ be a Sequence of random vectors and $\mathbf{X}$ be a Random Vector, then $(\forall j\in \{1,2,\dots ,p\},X_{nj}\stackrel{P}{\to }{X}_j)\iff {\mathbf{X}}_n\stackrel{P}{\to }\mathbf{X}$

Convergence in Distribution

${\mathbf{X}}_n\stackrel{D}{\to }\mathbf{X}\iff (\forall \mathbf{x}\in C(F_{\mathbf{X}}),\underset{n\to \infty }{\lim }{F}_{{\mathbf{X}}_n}(\mathbf{x})=F_{\mathbf{X}}(\mathbf{x}))$

Let $\{{\mathbf{X}}_n\}$ be a Sequence of random vectors with CDF $F_{{\mathbf{X}}_n}$, $\mathbf{X}$ be a Random Variable with cdf $F_{\mathbf{X}}$, and $C(F_{\mathbf{X}})$ be a set of every point at which $F_X$ is continuous, then $\{{\mathbf{X}}_n\}$ converges in distribution to $\mathbf{X}$ if $\forall \mathbf{x}\in C(F_{\mathbf{X}}),\underset{n\to \infty }{\lim }{F}_{{\mathbf{X}}_n}(\mathbf{x})=F_{\mathbf{X}}(\mathbf{x})$, and denoted by ${\mathbf{X}}_n\stackrel{D}{\to }\mathbf{X}$. $\mathbf{X}$ is also called limiting distribution of ${\mathbf{X}}_n$ or asymptotic distribution of ${\mathbf{X}}_n$

Continuous Mapping Theorem

Let $g$ be a Continuous Function ${\mathbf{X}}_n\stackrel{D}{\to }\mathbf{X}\Rightarrow g({\mathbf{X}}_n)\stackrel{D}{\to }g(\mathbf{X})$

MGF Technique

Let $\{{\mathbf{X}}_n\}$ be a Sequence of random vectors with MGF $M_{{\mathbf{X}}_n}(t)$ and $\mathbf{X}$ be a Random Vector with MGF $M_{\mathbf{X}}(t)$, then $\underset{n\to \infty }{\lim }{M}_{{\mathbf{X}}_n}(t)=M_{\mathbf{X}}(t)\iff {\mathbf{X}}_n\stackrel{D}{\to }\mathbf{X}$

Central Limit Theorem

Let $\{{\mathbf{X}}_n\}$ be i.i.d. Sequence of random vectors from a distribution with mean $E({\mathbf{X}}_1)=\mathbf{\mu }$, variance-covariance matrix $\mathrm{Var}({\mathbf{X}}_1)=\Sigma$, then $\sqrt{n}(\overline{\mathbf{X}}-\mathbf{\mu })\stackrel{D}{\to }{N}_p(0,\Sigma )$

For i.i.d random variables that have finite variance, the sample mean converges to Normal Distribution.

Delta Method

Let $\{{\mathbf{X}}_n\}$ be a Sequence of p-dimensional random vectors with $\sqrt{n}({\mathbf{X}}_n-\mathbf{\mu })\stackrel{D}{\to }{N}_p(0,\Sigma )$, $\mathbf{g}:{\mathbb{R}}^p\to {\mathbb{R}}^k$, $B=\left(\frac{\partial g_i}{\partial {\mu }_i}\right)$ be a $k\times p$ matrix, and $B(\mathbf{\mu })\ne 0$, then $\sqrt{n}(\mathbf{g}({\mathbf{X}}_n)-\mathbf{g}(\mathbf{\mu }))\stackrel{D}{\to }{N}_k(0,B\Sigma B^{⊺})$

Facts

Let ${\mathbf{v}}^{⊺}=(v_1,v_2,\dots ,v_p)\in {\mathbb{R}}^p$, then $\forall j\in \{1,2,\dots p\},|v_j|\le ||\mathbf{v}||\le \sum _{i=1}^n|v_i|$

Let $\{{\mathbf{X}}_n\}$ be a Sequence of p-dimensional random vectors, ${\mathbf{X}}_n\stackrel{D}{\to }N(\mathbf{\mu },\Sigma )$, $A$ be a $m\times p$ constant matrix, and $\mathbf{b}$ be a $m$ dimensional constant vector, then $A{\mathbf{X}}_n+\mathbf{b}\stackrel{D}{\to }N(A\mathbf{\mu }+\mathbf{b},A\Sigma A^{⊺})$