In this paper, we study two-person nonzero-sum games for continuous-time Markov chains with discounted payoff criteria and Borel action spaces. The transition rates are possibly unbounded, and the payoff functions might have neither upper nor lower bounds. We give conditions that ensure the existence of Nash equilibria in stationary strategies. For the zero-sum case, we prove the existence of the value of the game, and also provide a recursive way to compute it, or at least to approximate it. Our results are applied to a controlled queueing system. We also show that if the transition rates are uniformly bounded, then a continuous-time game is equivalent, in a suitable sense, to a discrete-time Markov game.

Suppose that a gambler starts with a fortune in (0,1) and wishes to attain a fortune of 1 by making a sequence of bets. Assume that whenever the gambler stakes an amount s, the gambler's fortune increases by s with probability w and decreases by s with probability 1 − w, where w < ½. Dubins and Savage showed that the optimal strategy, which they called ‘bold play’, is always to bet min{f, 1 − f}, where f is the gambler's current fortune. Here we consider the problem in which the gambler may stake no more than ℓ at one time. We show that the bold strategy of always betting min{ℓ, f, 1 − f} is not optimal if ℓ is irrational, extending a result of Heath, Pruitt, and Sudderth.

We consider epidemics in populations that are partitioned into small groups known as households. Whilst infectious, a typical infective makes global and local contact with individuals chosen independently and uniformly from the whole population or their own household, as appropriate. Previously, the classical Poisson approximation for the number of survivors of a severe epidemic has been extended to the household model. However, in the current work we exploit a Sellke-type construction of the epidemic process, which enables the derivation of sufficient conditions for the existence of a compound Poisson limit theorem for the survivors of the epidemic. The results are specialised to the Reed-Frost and general stochastic epidemic models.

We consider a heterogeneous population of identical particles divided into a finite number of classes according to their level of health. The partition can change over time, and a suitable exchangeability assumption is made to allow for having identical items of different types. The partition is not observed; we only observe the cardinality of a particular class. We discuss the problem of finding the conditional distribution of particle lifetimes, given such observations, using stochastic filtering techniques. In particular, a discrete-time approximation is given.

We study portfolio optimization problems in which the drift rate of the stock is Markov modulated and the driving factors cannot be observed by the investor. Using results from filter theory, we reduce this problem to one with complete observation. In the cases of logarithmic and power utility, we solve the problem explicitly with the help of stochastic control methods. It turns out that the value function is a classical solution of the corresponding Hamilton-Jacobi-Bellman equation. As a special case, we investigate the so-called Bayesian case, i.e. where the drift rate is unknown but does not change over time. In this case, we prove a number of interesting properties of the optimal portfolio strategy. In particular, using the likelihood-ratio ordering, we can compare the optimal investment in the case of observable drift rate to that in the case of unobservable drift rate. Thus, we also obtain the sign of the drift risk.

For general risk processes, we introduce and study the expected time-integrated negative part of the process on a fixed time interval. Differentiation theorems are stated and proved. They make it possible to derive the expected value of this risk measure, and to link it with the average total time below 0, studied by Dos Reis, and the probability of ruin. We carry out differentiation of other functionals of one-dimensional and multidimensional risk processes with respect to the initial reserve level. Applications to ruin theory, and to the determination of the optimal allocation of the global initial reserve that minimizes one of these risk measures, illustrate the variety of fields of application and the benefits deriving from an efficient and effective use of such tools.

We generalize a theorem due to Keilson on the approximate exponentiality of waiting times for rare events in regenerative processes. We use the result to investigate the limit distribution for a family of first entrance times in a sequence of Ehrenfest urn models. As a second application, we consider approximate pattern matching, a problem arising in molecular biology and other areas.

In this work, we study some geometrical properties of a stationary Gaussian field modeling the sea surface, using the energy spectrum. We consider the length of a crest and the mean speed of contours, which can be expressed as integrals over level sets. We also give central limit theorems for some of these quantities, using chaos expansions.

We study the asymptotic tail behavior of the conditional probability distributions of rt+k and rt+1+⋯+rt+k when (rt)t∈ℕ is a GARCH(1, 1) process. As an application, we examine the relation between the extreme lower quantiles of these random variables.

A multitype branching process is presented in the framework of marked trees and its structure is studied by applying the strong branching property. In particular, the Markov property and the expression for the generator are derived for the process whose components are the numbers of particles of each type. The filtering of the whole population, observing the number of particles of a given type, is discussed. Weak uniqueness for the filtering equation and a recursive structure for the linearized filtering equation are proved under a suitable assumption on the reproduction law.

In this paper, we investigate the asymptotic behaviour of controlled branching processes with random control functions. In a critical case, we establish sufficient conditions for both their almost-sure extinction and for their nonextinction with a positive probability. For some suitably chosen norming constants, we also determine different kinds of limiting behaviour for this class of processes.

We consider single-server queues with exponentially distributed service times, in which the arrival process is governed by a semi-Markov process (SMP). Two service disciplines, processor sharing (PS) and random service (RS), are investigated. We note that the sojourn time distribution of a type-l customer who, upon his arrival, meets k customers already present in the SMP/M/1/PS queue is identical to the waiting time distribution of a type-l customer who, upon his arrival, meets k+1 customers already present in the SMP/M/1/RS queue. Two sets of system equations, one for the joint transform of the sojourn time and queue size distributions in the SMP/M/1/PS queue, and the other for the joint transform of the waiting time and queue size distributions in the SMP/M/1/RS queue, are derived. Using these equations, the mean sojourn time in the SMP/M/1/PS queue and the mean waiting time in the SMP/M/1/RS queue are obtained. We also consider a special case of the SMP in which the interarrival time distribution is determined only by the type of the customer who has most recently arrived. Numerical examples are also presented.

We use a sample-path technique to derive asymptotics of generalized Jackson queueing networks in the fluid scale; that is, when space and time are scaled by the same factor n. The analysis only presupposes the existence of long-run averages and is based on some monotonicity and concavity arguments for the fluid processes. The results provide a functional strong law of large numbers for stochastic Jackson queueing networks, since they apply to their sample paths with probability 1. The fluid processes are shown to be piecewise linear and an explicit formulation of the different drifts is computed. A few applications of this fluid limit are given. In particular, a new computation of the constant that appears in the stability condition for such networks is given. In a certain context of a rare event, the fluid limit of the network is also derived explicitly.

We give the exact asymptotics of the tail of the stationary maximal dater in generalized Jackson networks with subexponential service times. This maximal dater, which is an analogue of the workload in an isolated queue, gives the time taken to clear all customers present at some time t when stopping all arrivals that take place later than t. We use the property that a large deviation of the maximal dater is caused by a single large service time at a single station at some time in the distant past of t, in conjunction with fluid limits of generalized Jackson networks, to derive the relevant asymptotics in closed form.

We derive several algorithms for the busy period distribution of the canonical Markovian fluid flow model. One of them is similar to the Latouche-Ramaswami algorithm for quasi-birth-death models and is shown to be quadratically convergent. These algorithms significantly increase the efficiency of the matrix-geometric procedures developed earlier by the authors for the transient and steady-state analyses of fluid flow models.

We consider solutions of Burgers' equation with linear or quadratic external potential and stationary random initial conditions of Ornstein-Uhlenbeck type. We study a class of limit laws that correspond to a scale renormalization of the solutions.

We consider the lifetimes of systems that can be modeled as particles that move within a bounded region in ℝn. Particles move within the set according to a random walk, and particles that leave the set are lost. We divide the set into equal cells and define the lifetime of the set as the time required for the number of particles in one of the cells to fall below a predetermined threshold. We show that the lifetime of the system, given a sufficiently large number of particles, is Weibull distributed.

A random variable Y is branching stable (B-stable) for a nonnegative integer-valued random variable J with E(J)>1 if Y*J∿cY for some scalar c, where Y*J is the sum of J independent copies of Y. We explore some aspects of this notion of stability and show that, for any Y0 with finite nonzero mean, if we define Yn+1=Yn*J/E(J) then the sequence Yn converges in law to a random variable Y∞ that is B-stable for J. Also Y∞ is the unique B-stable law with mean E(Y0). We also present results relating to random variables Y0 with zero means and infinite means. The notion of B-stability arose in a scheme for cataloguing a large network of computers.

We consider a finite dam under the policy, where the input of water is formed by a Wiener process subject to random jumps arriving according to a Poisson process. The long-run average cost per unit time is obtained after assigning costs to the changes of release rate, a reward to each unit of output, and a penalty that is a function of the level of water in the reservoir.