Stochastic Process

Definition

$\{X_i|i\in I\}$ where each $X_i$ is a Random Variable.

Stochastic process is a collection of random variables defined on a common probability space $(\Omega ,\mathcal{F},P)$, and whose index represents time.

Notation

Stochastic Process whose Initial Distribution is a Delta Distribution

If a random variable $X$ has a positive probability only at $X=x$ and otherwise $0$, then the distribution $\mathbf{\mu }$ is expressed as ${\mathbf{\delta }}_x$. Also, the probability and expectation of $X$ are expressed as $P_{{\mathbf{\delta }}_x}(X=x)=1$, $P_{{\mathbf{\delta }}_x}(X\ne x)=0$ and $E_{{\mathbf{\delta }}_x}(X)=x$ respectively.

The probability of $X_t$ given some initial distribution is expressed as following

• $P_i(X_t=k):=P_{{\mathbf{\delta }}_i}(X_t=k)=P(X_t=k|X_0=i)$ where $i$ is the initial state
• $P_{\mathbf{\pi }}(X_t=k)={\pi }_k$ where $\mathbf{\pi }$ is the initial distribution

where $\{X_t\}$ is a HMC defined on the state space $E$

Averages

For a sequence of random variables $X_1,X_2,X_3,\dots ,X_i,\dots ,X_n$

Ensemble Average

$E(X_t)$

Time Average

$\frac{1}{T}{\sum }_{t=1}^T{X}_t$

Facts

If the random variables of a stochastic process satisfy i.i.d condition, then the ensemble average is equal to time average

Summary

The Distributions of Stochastic Process

	Empirical distribution	Stationary Distribution	Limiting Distribution
Used for	Every stochastic process	Every stochastic process	Markov chain
Expression	$\bar{\mathbf{\pi }}=\frac{1}{T}\left[\begin{array}{c}\mathtt{sum}(X_t=0)\\ \mathtt{sum}(X_t=1)\end{array}\right]$	$\mathbf{\pi }=\left[\begin{array}{c}E(I(X=0))\\ E(I(X=1))\end{array}\right]$	${\mathbf{p}}_{\star }={\lim }_{t\to \infty }\left[\begin{array}{c}{p}_{?0}^{(t)}\\ p_{?1}^{(t)}\end{array}\right]$
Consideration of $\omega$	use single $\omega$	use every $\omega$	don’t consider $\omega$
Statistical distribution?	O	O	X
Analytic value?	X	O	O
Related to data?	O	X	X
Related to limit?	O	X	O
Related to LLN?	O	O	X
Description	Averaged calculated by the data	Theoretical expectation	Theoretical convergence value

${\bar{\mathbf{\pi }}}^{⊺}$: the empirical distribution of $\{X_t\}$ with consideration of single $\omega$ (use time average)
${\mathbf{\pi }}^{⊺}$: the Stationary Distribution of $\{X_t\}$ with consideration of every $\omega$ ${\mathbf{p}}_{\star }^{⊺}$: the Limiting Distribution of $\{X_t\}$ obtained by the limit of $\mathbf{P}$ without consideration of $\omega$. Used with Markov Chain

where $\{X_t\}$ is a sequence of random variables