The process generated by the crossings of a fixed level, u, by the process Pn(t) is considered, where

and the Xi(t) are identical, independent, separable, stationary, zero mean, Gaussian processes. A simple formula is obtained for the expected number of upcrossings in a given time interval, sufficient conditions are given for the upcrossings process to tend to a Poisson process as u→∞, and it is shown that under suitable scaling the distribution of the length of an excursion of Pn(t) above u tends to a Rayleigh distribution as u→ ∞.

A stationary process yt, t ∈ R1 is considered which is Markov between points of changeover from a stationary point process, and at these points it changes over according to a distribution dependent only on the value of yt just before the change is analysed. An explicit form of a rate-conservative principle is stated, and its relationship with formulae relating the distribution of the process at an instant t and the distribution at a point of changeover is shown. The theory is applied to discrete state processes and to processes which are generalizations of the Takács processes, and is also applied in the theory of G/M/l and G/G/k queues to obtain relations between distributions of a process and some process imbedded in it.

A class of stationary multivariate point processes is considered in which the events of one of the point processes act as regeneration points for the entire multivariate process. Some important properties of such processes are derived including the joint probability generating function for numbers of events in an interval of fixed length and the asymptotic behaviour of such processes. The general theory is then applied in three bivariate examples. Finally, some simple monotonicity results for stationary and renewal point processes (which are used in the second example) are proved in two appendices.

Replacement policies based on measurements of some increasing state variable, e.g. wear, accumulated damage or accumulated stress, are studied in this paper. It is assumed that the state measurements may be regarded as realizations of some stochastic process and that the proneness to failure of an active unit may be described by an increasing state-dependent failure rate function. Average long-run cost per unit time is considered. The optimal replacement rule is shown to be a control limit rule, i.e. it is optimal to replace either at failure or when the state variable has reached some threshold value, whichever occurs first. The optimal rule is determined. Some generalizations and special cases are given.

This paper considers non-cooperative N-person stochastic games with a countable state space and compact metric action spaces. We concentrate upon the average return per unit time criterion for which the existence of an equilibrium policy is established under a number of recurrency conditions with respect to the transition probability matrices associated with the stationary policies. These results are obtained by establishing the existence of total discounted return equilibrium policies, for each discount factor α ∈ [0, 1) and by showing that under each one of the aforementioned recurrency conditions, average return equilibrium policies appear as limit policies of sequences of discounted return equilibrium policies, with discount factor tending to one.

Finally, we review and extend the results that are known for the case where both the state space and the action spaces are finite.

Discounted dynamic programming problems whose transition mechanism depends only on the action taken and does not depend on the current state are considered. A value determination operation and method of obtaining optimal policies for the case of finite action space (and arbitrary state space) are presented.

The solution of other problems is reduced to this special case by a suitable transformation. Results are illustrated by examples.