(2) d is the largest integer with property (1).
If d = 1, state i is called periodic. The Markov chain X is periodic if and only if Vi e E
are periodic.
Theorem A2: If X is irreducible and if 3i e E 3 p(i, i) > 0, then X is periodic.
Definition A10: Any probability vector q over the state space of the time homogeneous
Markov chain X and satisfying
is called a stationary distribution of X. It is not necessarily unique.
Theorem A3: If the Markov chain X is time-homogeneous, irreducible, periodic and has
a finite state space, then a stationary distribution exists for X. Further, the stationary dis-
tribution is unique and is determined by
and
qI 1.
Theorem A4: If the time-homogeneous Markov chain X possesses a unique stationary
distribution, q, then for every probability vector x with compatible dimensions, it follows
that
l -t= -
lim k ? =q.
k -+ -