\documentclass[a4paper,12pt]{article}
\newcommand{\ds}{\displaystyle}
\newcommand{\pl}{\partial}
\parindent=0pt
\begin{document}


{\bf Question}

A man's smoking habits are as follows.  If he smokes filter
cigarettes one week, he switches to nonfilter cigarettes the next
week with probability $p.$  On the other hand, if he smokes
nonfilter cigarettes one week there is a probability $\alpha$ that
he switches to filter  in the following week.  Classify the states
of this Markov chain for different values of $p$ and $\alpha$ as
transient, null-recurrent or positive recurrent.

\vspace{.25in}

{\bf Answer}

This is a 2-state Markov chain with state 1 \lq \lq smokes filter
\rq \rq and state 2 \lq \lq smokes non-filter \rq \rq.

The transition matrix is $P = \left( \begin{array}{cc} 1-p & p\\
\alpha & 1-\alpha \end{array} \right) $

\setlength{\unitlength}{.75in}
\begin{picture}(7,2)

\put(2,1){\circle{.5}} \put(1.55,1){\circle{.4}}
\put(1.36,1.08){\vector(0,1){0}}\put(.8,1){$1-p$}
\put(1.95,.95){1}

\put(4,1){\circle{.5}} \put(4.45,1){\circle{.4}}
\put(4.63,.95){\vector(0,-1){0}}\put(4.7,1){$1-\alpha$}
\put(3.95,.95){2}

 \put(2.15,1.2){\line(1,0){1.7}}
\put(3.15,1.204){\vector(1,0){0}} \put(3,1.3){$p$}

\put(2.15,0.8){\line(1,0){1.7}} \put(3,.804){\vector(-1,0){0}}
\put(3,0.6){$\alpha$}

\end{picture}

If $p =0$ state 1 is absorbing

If $\alpha =0$ state 2 is absorbing

If $p =0$ and $\alpha > 0 $ state 2 is transient

If $\alpha =0$ and $p > 0$ state 1 is transient

Suppose $p \not= 0$ and $\alpha \not= 0$

P(returning to state 1) = $f_{11}$

$= (i-p) + p \alpha + p(1 - \alpha) \alpha + p(1 - \alpha)^2\alpha
+ ...$

$= (1-p + p \alpha (1+ (1 - \alpha) + (1 - \alpha)^2 + ... )$

$\ds = 1 - p + \frac{p \alpha}{1 - (1 - \alpha)} = 1$

so state 1 is recurrent.  Similarly state 2 is recurrent.

\bigskip

Mean  recurrence time for state 1:

\begin{eqnarray*}\mu_1 & = & 1 \cdot (1 -p) + 2pa + 3 p(1 - \alpha
)\alpha + 4p(1 - \alpha)^2 \alpha +... \\ & = & 1 +
\frac{p}{\alpha} \end{eqnarray*} using arithmetic - geometric
series.

Similarly $\ds\mu_2 = 1 + \frac{\alpha}{p}$




\end{document}