In this paper, the authors consider a class of stochastic systems described by Ito differential equations for which both controls and parameters are to be chosen optimally with respect to a certain performance index over a fixed time interval. The controls to be optimized depend only on partially observed current states as in a work of Fleming. However, he considered, instead, a problem of optimal control of systems governed by stochastic Ito differential equations with Markov terminal time. The fixed time problems usually give rise to the Cauchy problems (unbounded domain) whereas the Markov time problems give rise to the first boundary value problems (bounded domain). This fact makes the former problems relatively more involved than the latter. For the latter problems, Fleming has reported a necessary condition for optimality and an existence theorem of optimal controls. In this paper, a necessary condition for optimality for both controls and parameters combined together is presented for the former problems.

A criterion for ergodicity of lattice interactions has been given by Dobrushin [2], and improved by Harris [3] and Holley [5]. In this paper we present a simplified derivation of these results, and obtain stronger conditions for interactions on the one- and two-dimensional integer lattices, and the 3–Bethe lattice.

The familiar three theorems of Rényi concerning the record times in an i.i.d. sequence of random variables are extended to the record times and inter-record times of a sequence of dependent, non-identically distributed random variables defined on a finite Markov chain. These theorems are the Central Limit Theorem (C.L.T.), the Strong Law of Large Numbers (S.L.L.N.) and the Law of the Iterated Logarithm (L.I.L.). Similar results are also obtained for m-record times, inter-m-record times, and for the continuous parameter situation when observations are taken at the epochs of a Poisson process.

For the Ek/G/1 queue with finite waiting room the phase technique is used to analyse the Markov chain imbedded in the queueing process at successive instants at which customers complete service, and the distribution of queue length is obtained. The limit as the size of the waiting room becomes infinite is found.