\documentclass[a4paper,12pt]{article}
\newcommand{\ds}{\displaystyle}
\newcommand{\pl}{\partial}
\parindent=0pt
\begin{document}
{\bf Question}
A conversation involving a person $A$ was observed at 5-second
intervals. If $A$ was speaking a 1 was recorded; if $A$ was
silent a 0 was recorded. The following results were obtained:
\bigskip
$1111111110001111111111111111111111111111$
$1111111111111111111111111111111110000000$
$0000000000011111111111111111111111111111$
$1111100111111111111111110011111111111111$
$11111111110100000001111111$
\bigskip
Describe the assumption which must be made in order to model this
process as a two-state Markov chain.
Estimate the transition probability matrix.
If the person is observed talking estimate the probability that he
will be taking.
\begin{description}
\item[(i)] at the next observation of the conservation,
\item[(ii)] when the conservation is observed 500 seconds later.
\end{description}
Estimate the mean recurrence time for state 1 from the data and
explain how this relates to the proportion of time spent in state
1.
\vspace{.25in}
{\bf Answer}
The frequency matrix is as follows:
\bigskip
\hspace{1.1in} To \hspace{.7in} Total
\medskip
From \hspace{.3in} $\bordermatrix{ & 0 & 1 \cr 0 & 25 & 6 \cr 1 &
6 & 161}$ \hspace{.3in} $\begin{array}{r} 31 \\ 167 \end{array}$
\hspace{2.2in} ${\overline{198}}$
\bigskip
So an estimated transition matrix will be $$P =
\left(\begin{array}{cc} \ds\frac{25}{31} & \ds\frac{6}{31} \\
\ds\frac{6}{167} & \ds\frac{161}{167}
\end{array}\right)$$
The assumptions are that the probabilities of transition depend
only on the starting state, and not on past history.
\begin{description}
\item[(i)] If ${\bf p}_0 = (0,1)$ then at the next observation the
distribution is
$\ds {\bf p}_0P = \left( \frac{6}{167}, \frac{161}{167} \right)$
so the probability that he is talking is $\ds \frac{161}{167}$
\item[(ii)] This is a finite Markov chain so it has equilibrium
distribution given by the stationary distribution. So $\pi P =
\pi$
i.e. $\ds \pi_0 \frac{25}{31} + \pi_1 \frac{6}{167} = \pi_0 \ \
{\rm and}\ \ \pi_0 + \pi_1 = 1$
These give $\pi_0 = \frac{31}{198} \approx 0.157 \hspace{.2in}
\pi_1 = \frac{167}{198} \approx 0.843$
OR use the standard formula for a $2\times2$ Markov chain to
obtain:$$P^n \to \left(
\begin{array}{cc} \frac{31}{198} & \frac{167}{198} \\
\frac{31}{198} & \frac{167}{198} \end{array} \right)$$
From the data, recurrence times for state 1 are as follows
$$\begin{array}{lccccccc} {\rm Time\ till\ return} & 1 & 2 & 3 & 4
& 8 & 17 & {\rm Total} \\ {\rm Frequency} & 161 & 1 & 2 & 1 & 1 &
1 & 167 \end{array}$$
The sample mean recurrence time is $\ds \frac{198}{167} \approx
1.186$ The expected proportion of the time spent in state 1
should be $\pi_1$ for a long sequence, and should satisfy $\pi_1 =
\frac{1}{\mu} $ where $\mu_1$ is the mean recurrence time.
This is borne out by this example.
\end{description}
\end{document}