Transition Probability

Definition

$p_{ij}=P(j|i)=P(X_{t+1}=j|X_t=i)$ where $\{X_t{\}}_{t\ge 0}$ is HMC that has countable state space $E$

The conditional probability of the Transition from state $i$ to $j$

Facts

$p_{ij}\ge 0$