In this paper we extend the results of Meyn and Tweedie (1992b) from discrete-time parameter to continuous-parameter Markovian processes Φ evolving on a topological space.

We consider a number of stability concepts for such processes in terms of the topology of the space, and prove connections between these and standard probabilistic recurrence concepts. We show that these structural results hold for a major class of processes (processes with continuous components) in a manner analogous to discrete-time results, and that complex operations research models such as storage models with state-dependent release rules, or diffusion models such as those with hypoelliptic generators, have this property. Also analogous to discrete time, ‘petite sets', which are known to provide test sets for stability, are here also shown to provide conditions for continuous components to exist.

New ergodic theorems for processes with irreducible and countably reducible skeleton chains are derived, and we show that when these conditions do not hold, then the process may be decomposed into an uncountable orbit of skeleton chains.

In Part I we developed stability concepts for discrete chains, together with Foster–Lyapunov criteria for them to hold. Part II was devoted to developing related stability concepts for continuous-time processes. In this paper we develop criteria for these forms of stability for continuous-parameter Markovian processes on general state spaces, based on Foster-Lyapunov inequalities for the extended generator.

Such test function criteria are found for non-explosivity, non-evanescence, Harris recurrence, and positive Harris recurrence. These results are proved by systematic application of Dynkin's formula.

We also strengthen known ergodic theorems, and especially exponential ergodic results, for continuous-time processes. In particular we are able to show that the test function approach provides a criterion for f-norm convergence, and bounding constants for such convergence in the exponential ergodic case.

We apply the criteria to several specific processes, including linear stochastic systems under non-linear feedback, work-modulated queues, general release storage processes and risk processes.

In this paper we introduce a multilevel birth-death particle system and consider its diffusion approximation which can be characterized as a M([R+)-valued process. The tightness of rescaled processes is proved and we show that the limiting M(R+)-valued process is the unique solution of the M([R+)-valued martingale problem for the limiting generator. We also study the moment structures of the limiting diffusion process.

During the last few years, several variants of P. Lévy's formula for the stochastic area of complex Brownian motion have been obtained. These are of interest in various domains of applied probability, particularly in relation to polymer studies. The method used by most authors is the diagonalization procedure of Paul Lévy.

Here we derive one such variant of Lévy's formula, due to Chan, Dean, Jansons and Rogers, via a change of probability method, which reduces the computation of Laplace transforms of Brownian quadratic functionals to the computations of the means and variances of some adequate Gaussian variables.

We then show that with the help of linear algebra and invariance properties of the distribution of Brownian motion, we are able to derive simply three other variants of Lévy's formula.

In this paper, we study the superposition of finitely many Markov renewal processes with countable state spaces. We define the S-Markov renewal equations associated with the superposed process. The solutions of the S-Markov renewal equations are derived and the asymptotic behaviors of these solutions are studied. These results are applied to calculate various characteristics of queueing systems with superposition semi-Markovian arrivals, queueing networks with bulk service, system availability, and continuous superposition remaining and current life processes.

The two-point Markov chain boundary-value problem discussed in this paper is a finite-time version of the quasi-stationary behaviour of Markov chains. Specifically, for a Markov chain {Xt:t = 0, 1, ·· ·}, given the time interval (0, n), the interest is in describing the chain at some intermediate time point r conditional on knowing both the behaviour of the chain at the initial time point 0 and that over the interval (0, n) it has avoided some subset B of the state space. The paper considers both ‘real time' estimates for r = n (i.e. the chain has avoided B since 0), and a posteriori estimates for r < n with at least partial knowledge of the behaviour of Xn. Algorithms to evaluate the distribution of Xr can be as small as O(n3) (and, for practical purposes, even O(n2 log n)). The estimates may be stochastically ordered, and the process (and hence, the estimates) may be spatially homogeneous in a certain sense. Maximum likelihood estimates of the sample path are furnished, but by example we note that these ML paths may differ markedly from the path consisting of the expected or average states. The scope for two-point boundary-value problems to have solutions in a Markovian setting is noted.

Several examples are given, together with a discussion and examples of the analogous problem in continuous time. These examples include the basic M/G/k queue and variants that include a finite waiting room, reneging, balking, and Bernoulli feedback, a pure birth process and the Yule process. The queueing examples include Larson's (1990) ‘queue inference engine'.

For two-dimensional spatial data, a spatial unilateral autoregressive moving average (ARMA) model of first order is defined and its properties studied. The spatial correlation properties for these models are explicitly obtained, as well as simple conditions for stationarity and conditional expectation (interpolation) properties of the model. The multiplicative or linear-by-linear first-order spatial models are seen to be a special case which have proved to be of practical use in modeling of two-dimensional spatial lattice data, and hence the more general models should prove to be useful in applications. These unilateral models possess a convenient computational form for the exact likelihood function, which gives proper treatment to the border cell values in the lattice that have a substantial effect in estimation of parameters. Some simulation results to examine properties of the maximum likelihood estimator and a numerical example to illustrate the methods are briefly presented.

Non-linear stochastic systems driven by white noise are analysed from the viewpoint of non-linear oscillation theory. Under various familiar hypotheses concerning dissipative and restorative dynamical forces, the existence and uniqueness, asymptotic growth, and oscillatory behavior of the solutions are demonstrated.

When a subsystem goes into an infrequent state, how does the remainder of the system behave? We show how to calculate the relevant distributions using the notions of reversed time for Markov processes and large deviations. For ease of exposition, most of the work deals with a specific queueing model due to Flatto, Hahn, and Wright. However, we show how the theorems may be applied to much more general jump-Markov systems.

We also show how the tools of time-reversal and large deviations complement each other to yield general theorems. We show that the way a constant coefficient process approaches a rare event is roughly by following the path of another constant coefficient process. We also obtain some properties, including apriori bounds, for the change of measure associated with some large deviations functionals; these are useful for accelerating simulations.

A single-server retrial queue consists of a primary queue, an orbit and a single server. Assume the primary queue capacity is 1 and the orbit capacity is infinite. Customers can arrive at the primary queue either from outside the system or from the orbit. If the server is busy, the arriving customer joins the orbit and conducts a retrial later. Otherwise, he receives service and leaves the system.

We investigate the stability condition for a single-server retrial queue. Let λ be the arrival rate and 1/μ be the mean service time. It has been proved that λ/μ < 1 is a sufficient stability condition for the M/G/1/1 retrial queue with exponential retrial times. We give a counterexample to show that this stability condition is not valid for general single-server retrial queues. Next we show that λ /μ < 1 is a sufficient stability condition for the stability of a single-server retrial queue when the interarrival times and retrial times are finite mixtures of Erlangs.

A new approach is used to obtain the transient probabilities of the M/M/1 queueing system. The first step of this approach deals with the generating function of the transient probabilities of the uniformized Markov chain associated with this queue. The second step consists of the inversion of this generating function. A new analytical expression of the transient probabilities of the M/M/1 queue is then obtained.

We propose a two-parameter family of conjugate prior distributions for the number of undiscovered objects in a class of Bayesian search models. The family contains the one-parameter Euler and Heine families as special cases. The two parameters may be interpreted respectively as an overall success rate and a rate of depletion of the source of objects. The new family gives enhanced flexibility in modelling.