Quality control for Markov chains and free boundary problems

Abstract:

A machine can manufacture any one of n Markov chains $P_x^{{\lambda _j}} \left ( {1 \leq j \leq n} \right )$; the $P_x^{{\lambda _j}}$ are defined on the space of all sequences $x = \left \{ {x\left ( m \right )} \right \} \left ( {1 \leq m \leq \infty } \right )$ and are absolutely continuous (in finite times) with respect to one another. It is assumed that chains $P_x^{{\lambda _j}}$ evolve in a random way, dictated by a Markov chain $\theta \left ( m \right )$ with n states, so that when $\theta \left ( m \right ) = j$ the machine is producing $P_x^{{\lambda _j}}$. One observes the $\sigma$-fields of $x\left ( m \right )$ in order to determine when to inspect $\theta \left ( m \right )$. With each product $P_x^{{\lambda _j}}$ there is associated a cost ${c_j}$. One inspects $\theta$ at a sequence of times (each inspection entails a certain cost) and stops production when the state $\theta = n$ is reached. The problem is to find an optimal sequence of inspections. This problem is reduced, in this paper, to solving a certain free boundary problem. In case $n = 2$ the latter problem is solved.