Combined Estimation and Control of HMMs

The principal contribution of this paper is the presentation of the potential theoretical results that are needed for an application of stochastic approximation theory to the problem of demonstrating asymptomatic stability for combined estimation and control of a plant described by a hidden Markov model. We motivate the results by briefly describing a combined estimation and control problem. We show how the problem translates to the stochastic approximation framework. We also show how the Markov chain that underlies the stochastic approximation problem can be decomposed into factors with discrete and continuous range. Finally, we use this decomposition to develop the results that are needed for an application of the ODE method to the stochastic control problem.