\documentclass[a4paper,12pt]{article}
\newcommand{\ds}{\displaystyle}
\newcommand{\pl}{\partial}
\parindent=0pt
\begin{document}


{\bf Question}

Customers arrive for a service taking one hour and provided by
only one server.  If there are any customers waiting for service
at the beginning of an hour, exactly one person will be served
during that hour.  If no customers are waiting for service at the
beginning of an hour, none will be served during the hour. Suppose
that $Z_n$ new customers arrive during the $n$-th period, where
$Z_1, Z_2, ...$ are independent, identically distributed random
variables with distribution $P(Z = k) = c_k,\ (k = 1, 2,\ldots).$
Let $X_n$ denote the number of customer present at the end of the
$n$-th period $(n \geq 1)$ and let $X_0$ be the initial number of
customers. Check that $\left\{X_n\right\}\ \ (n = 0,1,2,\ldots)$
is a Markov chain and show that its transition probability matrix
has the form $$\begin{array}{c} 0 \\ 1 \\ 2 \\ 3 \\ \vdots
\end{array} \left(
\begin{array}{ccccc} c_0 & c_1 &  c_2 & c_3 & \cdots \\ c_0 & c_1 &  c_2 & c_3 &
\cdots \\ 0 & c_0 &  c_1 & c_2 & \cdots \\ 0 & 0 &  c_0 & c_1 &
\cdots \\ \vdots & \vdots & \vdots & \vdots & \end{array}
\right)$$



\vspace{.25in}

{\bf Answer}

Let $X_n$ denote the number of customers waiting for service at
the end of the $n$-th hour, and let $Z_n$ denote the number of
customers who arrive to be served during the $n$-th hour.

If $X_n = 0$ then $X_{n+1} = Z_{n+1}$

If $X_n > 0$ then $X_{n+1} = X_n - 1 + Z_{n+1}$

$P(X_{n+1} = k| X_n = 0) = P(Z_{n+1} = k) = c_k$

and so this doesn't depend on previous history, but only on how
many customers arrive during the $n$-th hour.

$P(X_{n+1} = k| X_n = j) = P(Z_{n+1} = k-j+1) = c_{k-j+1}$ for $k
\geq j-1$ and $j \geq 1$

$P(X_{n+1} = k| X_n = j) = 0$ for $k < j-1$

Again these depend only on the number of customers arriving during
the $(n+1)$-th  period and the number at the beginning of the
period, and not on previous history.

so

$$P\ \ =\ \ \begin{array}{c} 0 \\ 1 \\ 2 \\ 3 \\ \vdots
\end{array} \left(
\begin{array}{ccccc} c_0 & c_1 &  c_2 & c_3 & \cdots \\ c_0 & c_1 &  c_2 & c_3 &
\cdots \\ 0 & c_0 &  c_1 & c_2 & \cdots \\ 0 & 0 &  c_0 & c_1 &
\cdots \\ \vdots & \vdots & \vdots & \vdots & \end{array}
\right)$$



\end{document}