\documentclass[a4paper,12pt]{article}
\newcommand{\ds}{\displaystyle}
\newcommand{\un}{\underline}
\newcommand{\undb}{\underbrace}
\newcommand{\pl}{\partial}
\newcommand{\df}{\ds\frac}
\parindent=0pt
\begin{document}


{\bf Question}

Twelve patients suffering from asthma are given treatment each
evening to relieve their symptoms. They will assess the state of
their breathlessness as nil, mild, moderate or severe (coded 0,\
1,\ 2 and 3 respectively), on the morning before treatment starts
(morning 0) and on each of the four mornings during the treatment
period (1,\ 2,\ 3 and 4). The following data are obtained:

\begin{tabular}{c|cccccccccccc}
Morning & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 & 9 & 10 & 11 & 12\\
\hline\\ 0 & 2 & 2 & 3 & 2 & 1 & 2 & 2 & 3 & 3 & 1 & 2 & 2 \\ 1 &
2 & 2 & 3 & 1 & 2 & 2 & 1 & 3 & 2 & 1 & 2 & 3 \\ 2 & 1 & 2 & 2 & 0
& 2 & 1 & 1 & 2 & 2 & 1 & 1 & 2 \\ 3 & 1 & 2 & 2 & 0 & 1 & 1 & 0 &
1 & 1 & 1 & 0 & 2 \\ 4 & 1 & 1 & 2 & 0 & 0 & 1 & 1 & 1 & 0 & 0 & 0
& 1 \end{tabular}

\begin{description}
\item[(i)]
Using the above table, count the number of transitions between
each pair of states. From this information estimate the transition
probability ,matrix, assuming breathlessness can be modelled by a
homogeneous Markov chain. Explain the meaning of the assumption.

\item[(ii)]
If a patient with moderate breathlessness is given the treatment,
what does the Markov chain model give for the probability that his
degree of breathlessness will be reduced on morning 1?

\item[(iii)]
Obtain the distribution predicted by the model for the degree of
breathlessness on day 1. Compare this with the frequency
distribution observed on day 1. Explain, without performing any
calculations, how the same comparison could be made for morning 4.
Suggest another method of investigating the validity of the Markov
assumption.
\end{description}






{\bf Answer}

Counting transitions gives the following results:

\begin{tabular}{c|cccc|c}
{} & 0 & 1 & 2 & 3 \\ \hline 0 & 3 & 1 & 0 & 0 & 4\\ 1 & 6 & 9 & 1
& 0 & 16\\ 2 & 0 & 10 & 11 & 1 & 22\\ 3 & 0 & 0 & 4 & 2 & 6\\
\hline & & & & & 48
\end{tabular}
\newpage
\begin{description}
\item[(i)]
Estimated $P$:

$\left(\begin{array}{cccc} \frac{3}{4} & \frac{1}{4} & 0 & 0\\
\frac{6}{16} & \frac{9}{16} & \frac{1}{16} & 0\\ 0 & \frac{10}{22}
& \frac{11}{22} & \frac{1}{22}\\ 0 & 0 & \frac{4}{6} & \frac{2}{6}
\end{array} \right)$

The Markov property says that the probability of transition
depends only on the current state, and not on previous history.
Also it is time homogeneous i.e., it doesn't depend on which day
is considered.

\item[(ii)]
$P(<2\ \mbox{on morning}\ 1\ |\ 2\ \mbox{on morning}\
0)=p_{20}+p_{21}=\df{10}{22}$

\item[(iii)]
The initial distribution is

$\un{p}_0 =\df{1}{12}\ (0,2,7,3)$ from the table.

On morning 1 the predicted distribution is

$\begin{array}{rcl} \un{p}_0 P & = & \left(\df{1}{16},\
\df{379}{1056},\ \df{15}{32},\ \df{29}{264}\right)\\ & = &
(.0625,\ .3589,\ .4688,\ .1098) \end{array}$

the observed relative frequency on day 1 is

$$\df{1}{12}(0,3,6,3)=(0,\ .25,\ .5,\ .25)$$

On morning 4 the predicted distribution is $\un{p}_0p^4$, which
could be compared with the observed frequency.

Alternatively we could look at

$$rel\ freq\ (X_n=k\ |\ X_{n-2}=i+X_{n-1}=j)$$

to test the Markov assumption.

e.g.

$rel\ freq\ (X_2=2\ |\ X_1=2\ \rm{and}\ X_0=3)=1\ \
\rm{different}$

$rel\ freq\ (X_2=2\ |\ X_1=2\ \rm{and}\ X_0=2)=\df{1}{4}$
\end{description}



\end{document}