Recurrence

Definition

Continuous Version

Let $\{X_t\}$ be a HMC defined on the state space $E$, and $T_i$ be the return time for the state $i\in E$

Recurrent

$P_i(T_i<\infty )=1$

Positive recurrent

$P_i(T_i<\infty )=1\wedge E_i[T_i]<\infty$

Null recurrent

$P_i(T_i<\infty )=1\wedge E_i[T_i]=\infty$

Transient

$P_i(T_i<\infty )<1$

Incomplete Discrete Version

Let $\{X_t\}$ be a HMC defined on the state space $E$, $i\in E$, and $p$ be the element of the transition matrix of $\{X_t\}$

Recurrent

${\sum }_{t=0}^{\infty }{p}_{ii}^{(t)}=\infty$

Transient

${\sum }_{t=0}^{\infty }{p}_{ii}^{(t)}<\infty$

Notation

$T_y:=\min \{t|X_t=y,t\ge 1\}=\min \{t\ge 1|X_t=y\}$

The first time to return to state $y$

$P_y(T_y<\infty )$

The probability of returning to $y$ from $y$ within a finite time.

Example

Let the $\{X_t\}$ be a HMC and $\mathbf{P}$ be a transition matrix.

$\mathbf{P}=\left[\begin{array}{cccccc}1-p& p& 0& 0& 0& \dots \\ 1-p& 0& p& 0& 0& \dots \\ 0& 1-p& 0& p& 0& \dots \\ 0& 0& 1-p& 0& p& \dots \\ \dots & \dots & \dots & \dots & \dots & \dots \end{array}\right]$

$\mathbf{P}$ is an Irreducible Markov Chain. Whether recurrent or transient is determined by $p$

• $p>1/2\Rightarrow {\sum }_{t=0}^{\infty }{p}_{00}^{(t)}<\infty$: transient
• $p<1/2\Rightarrow {\sum }_{t=0}^{\infty }{p}_{00}^{(t)}=\infty$ with $p_{00}^{(t)}\to c>0$: positive recurrent
• $p=1/2\Rightarrow {\sum }_{t=0}^{\infty }{p}_{00}^{(t)}<\infty$ with $p_{00}^{(t)}\to 0$: null recurrent

Facts

If a HMC $\{X_t\}$ is Irreducible Markov Chain, then every state $i$ has the same nature. In other words, it has one among the following

• every state is transient
• every state is null recurrent
• every state is positive recurrent

$\frac{1}{E(T_i)}\approx \frac{1}{T}{\sum }_{t=1}^T\#1(X_t=i)$ The number of times that returning to oneself for a time $T$ approximates the inverse of $E(T_i)$.