\documentclass[a4paper,12pt]{article}
\newcommand{\ds}{\displaystyle}
\newcommand{\pl}{\partial}
\parindent=0pt
\begin{document}


{\bf Question}

A conversation involving a person $A$ was observed at 5-second
intervals.  If $A$ was speaking a 1 was recorded; if $A$ was
silent a 0 was recorded.  The following results were obtained:

\bigskip

$1111111110001111111111111111111111111111$

$1111111111111111111111111111111110000000$

$0000000000011111111111111111111111111111$

$1111100111111111111111110011111111111111$

$11111111110100000001111111$

\bigskip

Describe the assumption which must be made in order to model this
process as a two-state Markov chain.

Estimate the transition probability matrix.

If the person is observed talking estimate the probability that he
will be taking.

\begin{description}
\item[(i)] at the next observation of the conservation,
\item[(ii)] when the conservation is observed 500 seconds later.
\end{description}

Estimate the mean recurrence time for state 1 from the data and
explain how this relates to the proportion of time spent in state
1.



\vspace{.25in}

{\bf Answer}

The frequency matrix is as follows:

\bigskip

\hspace{1.1in} To \hspace{.7in} Total

\medskip

From \hspace{.3in} $\bordermatrix{ & 0 & 1 \cr 0 & 25 & 6 \cr 1 &
6 & 161}$ \hspace{.3in} $\begin{array}{r} 31 \\ 167 \end{array}$

\hspace{2.2in} ${\overline{198}}$

\bigskip

So an estimated transition matrix will be $$P =
\left(\begin{array}{cc} \ds\frac{25}{31} & \ds\frac{6}{31} \\
\ds\frac{6}{167} & \ds\frac{161}{167}
\end{array}\right)$$

The assumptions are that the probabilities of transition depend
only on the starting state, and not on past history.
\begin{description}
\item[(i)] If ${\bf p}_0 = (0,1)$ then at the next observation the
distribution is

$\ds {\bf p}_0P = \left( \frac{6}{167}, \frac{161}{167} \right)$

so the probability that he is talking is $\ds \frac{161}{167}$

\item[(ii)] This is a finite Markov chain so it has equilibrium
distribution given by the stationary distribution.  So $\pi P =
\pi$

i.e. $\ds \pi_0 \frac{25}{31} + \pi_1 \frac{6}{167} = \pi_0 \ \
{\rm and}\ \  \pi_0 + \pi_1 = 1$

These give $\pi_0 = \frac{31}{198} \approx 0.157 \hspace{.2in}
\pi_1 = \frac{167}{198} \approx 0.843$

OR use the standard formula for a $2\times2$ Markov chain to
obtain:$$P^n \to \left(
\begin{array}{cc} \frac{31}{198} & \frac{167}{198} \\
\frac{31}{198} & \frac{167}{198} \end{array} \right)$$


From the data, recurrence times for state 1 are as follows
$$\begin{array}{lccccccc} {\rm Time\ till\ return} & 1 & 2 & 3 & 4
& 8 & 17 & {\rm Total} \\ {\rm Frequency} & 161 & 1 & 2 & 1 & 1 &
1 & 167 \end{array}$$

The sample mean recurrence time is $\ds \frac{198}{167} \approx
1.186$  The expected proportion of the time spent in state 1
should be $\pi_1$ for a long sequence, and should satisfy $\pi_1 =
\frac{1}{\mu} $ where $\mu_1$ is the mean recurrence time.

This is borne out by this example.
\end{description}



\end{document}