Convergence in Distribution

Definition

$(\forall x\in C(F_X),\underset{n\to \infty }{\lim }{F}_{X_n}(x)=F_X(x))\Rightarrow (X_n\stackrel{D}{\to }X)$

Let $\{X_n\}$ be a Sequence of random variables with CDF $F_{X_n}$, $X$ be a Random Variable with cdf $F_X$, and $C(F_X)$ be a set of every point at which $F_X$ is continuous, then $X_n$ converges in distribution to $X$ if $\forall x\in C(F_X),\underset{n\to \infty }{\lim }{F}_{X_n}(x)=F_X(x)$, and denoted by $X_n\stackrel{D}{\to }X$. $X$ is called the limiting distribution of $X_n$ or asymptotic distribution of $X_n$

Facts

$(X_n\stackrel{P}{\to }X)\Rightarrow (X_n\stackrel{D}{\to }X)$

Convergence in Probability implies convergence in distribution

$(\forall c\in \mathbb{R},X_n\stackrel{P}{\to }c)\iff (X_n\stackrel{D}{\to }c)$

$(({\overline{X}}_n\stackrel{D}{\to }X)\wedge ({\overline{Y}}_n\stackrel{P}{\to }0))\Rightarrow (X_n+Y_n\stackrel{D}{\to }X)$

Continuous Mapping Theorem

Definition

Continuous functions preserve convergences (in probability, Almost Surely, or in distribution) of a Sequence of random variables to limits.

Consider a Sequence $\{X_n\}$ of random variables defined on same Probability Space, and a Continuous Function $g$ on the space. Then,

• Convergence in Probability to a Constant: $X_n\stackrel{P}{\to }a\in \mathbb{R}⟹g(X_n)\stackrel{P}{\to }g(a)$
• Convergence in Probability: $X_n\stackrel{P}{\to }X⟹g(X_n)\stackrel{P}{\to }g(X)$
• Almost Sure Convergence: $X_n\stackrel{a.s.}{\to }X⟹g(X_n)\stackrel{a.s.}{\to }g(X)$
• Convergence in Distribution: $X_n\stackrel{D}{\to }X⟹g(X_n)\stackrel{D}{\to }g(X)$

Link to original

Slutzky Theorem

Definition

Let $\{X_n\},\{Y_n\},\{Z_n\}$ be Sequence of random variables, $X$ be a Random Variable, $k_1,k_2$ be constants, then $((X_n\stackrel{D}{\to }X)\wedge (Y_n\stackrel{D}{\to }{k}_1)\wedge (Z_n\stackrel{D}{\to }{k}_2))\Rightarrow (Y_n+Z_n{X}_n\stackrel{D}{\to }{k}_1+k_{2}X)$

Link to original

Let $\{P_n{\}}_{n\in \mathbb{N}}$ be a sequence of distributions. Then, $D_{KL}(P_n||P)\to 0⟹D_{JS}(P_n,P)\to 0⟺\delta (P_n,P)\to 0⟹W(P_n,P)\to 0⟺P_n\stackrel{D}{\to }P$ The convergence of the KL-Divergence to zero implies that the JS-Divergence also converges to zero. The convergence of the JS-Divergence to zero is equivalent to the convergence of the Total Variation Distance to zero. The convergence of the Total Variation Distance to zero implies that the Wasserstein Distance also converges to zero. The convergence of the Wasserstein Distance to zero is equivalent to the Convergence in Distribution of the sequence.

Link to original