Statistical Independence

Definition

Statistical Independence of Events

Suppose a Probability Space $(\Omega ,\mathcal{F},P)$. The events $(E_{\alpha }{)}_{\alpha \in I}\subset \mathcal{F}$ are independent if $P(\bigcap _{j=1}^n{E}_{{\alpha }_j})=\prod _{j=1}^{n}P(E_{{\alpha }_j}),\forall ({\alpha }_j{)}_{j=1}^n\subset I$

Statistical Independence of Two Random Variables

Rigorous

Suppose a Probability Space $(\Omega ,\mathcal{F},P)$. random variables $(X_{\alpha }:(\Omega ,\mathcal{F})\to (\mathbb{R},\mathcal{R}){)}_{\alpha \in I}$ are independent if $(E_{\alpha }:=X_{\alpha }^{-1}(B_{\alpha }){)}_{\alpha \in I}$ are independent, $\forall B_{\alpha }\in \mathcal{R}$

Casual

Two random variables $X,Y$ are independent if and only if $\forall x,\forall y,F_{X,Y}(x,y)=F_X(x)F_Y(y)$

Statistical Independence of Random Variables

Mutually Independent

A collection of random variables $X_1,X_2,\dots ,X_n$ are mutually independent if and only if $F_{\mathbf{X}}(\mathbf{x})={\prod }_{i=1}^n{F}_{X_i}(x_i),\forall x_i$ where $F_{X_i}(x_i)$ is marginal CDF of $X_i$

Or equivalently, a collection of random variables $X_1,X_2,\dots ,X_n$ are mutually independent if and only if $f_{\mathbf{X}}(\mathbf{x})={\prod }_{i=1}^n{f}_{X_i}(x_i),\forall x_i$ where $f_{X_i}(x_i)$ is marginal PDF of $X_i$

Pointwise Independent

A collection of random variables $X_1,X_2,\dots ,X_n$ are Pointwise independent if and only if $\forall i\ne j,F_{X_i,X_j}(x_i,x_j)=F_{X_i}(x_i)F_{X_j}(x_j),\forall x_i,x_j$ where $F_{X_i}(x_i)$ is marginal CDF of $X_i$ and $F_{X_i,X_j}(x_i,x_j)$ is joint pdf of $X_i$ and $X_j$.

Statistical Independence of Stochastic Processes

Facts

Random variables $X,Y$ are independent $\iff \forall x,y,\exists f,g\text{s.t.}{f}_{X,Y}(x,y)=g(x)h(y)$

where $g(x)$ is a function of $x$ only, and $h(y)$ is a function of $y$ only.

Random variables $X,Y$ are independent $\iff \forall x,\forall y,F_{X,Y}(x,y)=F_X(x)F_Y(y)$

Random variables $X,Y$ are independent $\iff \forall x,\forall y,P(a<X\le b,c<Y\le d)=P(a<X\le b)P(c<Y\le d)$

Random variables $X,Y$ are independent $\Rightarrow E[u(X)v(Y)]=E[u(X)]E[v(Y)]$

If Moment Generating Function $M(t_1,t_2)$ exists and Random variables $X,Y$ are independent $\iff M(t_1,t_2)=M(t_1,0)M(0,t_2)$

Mutually independence $\underset{\times }{\stackrel{\circ }{\rightleftharpoons }}$ pairwise independence

If $\mathbf{X}$ is mutually independent, then $E\left[{\prod }_{i=1}^n{g}_i(X_i)\right]={\prod }_{i=1}^{n}E[g_i(X_i)]$

If random variables $(X_{\alpha }{)}_{\alpha \in I}$ are independent, $f_{\alpha }:\mathbb{R}\to \mathbb{R}$ are Borel measurable, so are $(f_{\alpha }(X_{\alpha }))$

Let Random variables $X,Y$. $X$ and $Y$ are independent $\underset{\times }{\stackrel{\circ }{\rightleftharpoons }}\mathrm{Cov}(X,Y)=0$

If $X$ and $Y$ follow normal distributions, $X$ and $Y$ are independent $\iff \mathrm{Cov}(X,Y)=0$