We investigate the asymptotic behaviour of empirical processes truncated outside an interval about the (1 – s(n)/n)-quantile where s(n) → ∞ and s(n)/n → 0 as the sample size n tends to ∞. It is shown that extreme value (Poisson) processes and, alternatively, the homogeneous Poisson process may serve as approximations if certain von Mises conditions hold.

Semi-Markov games are investigated under discounted and limiting average payoff criteria. The issue of the existence of the value and a pair of stationary optimal strategies are settled; the optimality equation is studied and under a natural ergodic condition the existence of a solution to the optimality equation is proved for the limiting average case. Semi-Markov games provide useful flexibility in constructing recursive game models. All the work on Markov/semi-Markov decision processes and Markov (stochastic) games can be viewed as special cases of the developments in this paper.

Let Qn denote the number of times where a simple random walk reaches its maximum, where the random walk starts at the origin and returns to the origin after 2n steps. Such random walks play an important role in probability and statistics. In this paper the distribution and the moments of Qn, are considered and their asymptotic behavior is studied.

Let X(1) ≦ X(2) ≦ ·· ·≦ X(N(t)) be the order statistics of the first N(t) elements from a sequence of independent identically distributed random variables, where {N(t); t ≧ 0} is a renewal counting process independent of the sequence of X's. We give a complete description of the asymptotic distribution of sums made from the top kt extreme values, for any sequence kt such that kt → ∞, kt/t → 0 as t → ∞. We discuss applications to reinsurance policies based on large claims.

Let {Xn, n ≧ 1} be an i.i.d. sequence of positive random variables with a continuous distribution function having a regularly varying upper tail. Denote by {X(n), n ≧ 1} the corresponding sequence of record values. We introduce two statistics based on the sequence of successive record values and investigate their asymptotic behaviour. We also give some numerical results.

This paper deals with a class of discrete-time Markov chains for which the invariant measures can be expressed in terms of generalized continued fractions. The representation covers a wide class of stochastic models and is well suited for numerical applications. The results obtained can easily be extended to continuous-time Markov chains.

We construct a risk process, where the law of the next jump time or jump size can depend on the past through earlier jump times and jump sizes. Some distributional properties of this process are established. The compensator is found and some martingale properties are discussed.

We derive conditions under which the increments of a vector process are associated — i.e. under which all pairs of increasing functions of the increments are positively correlated. The process itself is associated if it is generated by a family of associated and monotone kernels. We show that the increments are associated if the kernels are associated and, in a suitable sense, convex. In the Markov case, we note a connection between associated increments and temporal stochastic convexity.

Our analysis is motivated by a question in variance reduction: assuming that a normalized process and its normalized compensator converge to the same value, which is the better estimator of that limit? Under some additional hypotheses we show that, for processes with conditionally associated increments, the compensator has smaller asymptotic variance.

Discrete minification processes are introduced and it is proved that the discrete first-order autoregression of McKenzie (1986) and the discrete minification process are mutually time-reversible if and only if they have common marginal geometric distribution, corresponding to a result for continuous processes given by Chernick et al. (1988). It is also proved that a discrete minification process is time-reversible if and only if it has marginal Bernoulli distribution.

We show that under some assumptions a diffusion process satisfying a one-dimensional Itô's equation has a transition probability density concentrated on a finite spatial interval. We give a formula for this density.

In this paper we show that to each distance d defined on the finite state space S of a strongly ergodic Markov chain there corresponds a coefficient ρd of ergodicity based on the Wasserstein metric. For a class of stochastically monotone transition matrices P, the infimum over all such coefficients is given by the spectral radius of P – R, where R = limkPk and is attained. This result has a probabilistic interpretation of a control of the speed of convergence of by the metric d and is linked to the second eigenvalue of P.

Let Y0, Y1, Y2, ··· be an i.i.d. sequence of random variables with absolutely continuous distribution function F, and let {N(t), t ≧ 0} be a Poisson process with rate λ (t) and mean Λ(t), independent of the Yj's. We associate Y0 with the point t = 0, and Yj with the jth point of N(·), j ≧ 1. The first Yj (j ≧ 1) to exceed all previous ones is the first record value, and the time of its occurrence is the first record time; subsequent record values and times are defined analogously. For general Λ, we give the joint distribution of the values and times of the first n records to occur after a fixed time T, 0 ≦ T < ∞. Assuming that F satisfies Von Mises regularity conditions, and that λ (t)/Λ (t) → c ∈ (0, ∞) as t → ∞, we find the limiting joint p.d.f. of the values and times of the first n records after T, as T → ∞. In the course of this we correct a result of Gaver and Jacobs (1978). We also consider limiting marginal and conditional distributions. In addition, we extend a known result for the limit as the number of recordsK → ∞, and we compare the results for the limit as T → ∞ with those for the limit as K → ∞.

We consider a Wiener process between a reflecting and an absorbing barrier and derive a series solution for the transition density of the process and for the density of the time to absorption. If the drift is towards the reflecting barrier, the variance is not too large, and the distance of the barriers is not too small, the leading term of the series derives from imaginary solutions of the basic eigenvalue equation of this problem. It is shown that these leading terms often make the dominant contribution to the complete series. Finally, we consider previous attempts by Fürth [3], Cox and Miller [1], and by Goel and Richter-Dyn [5] to solve the stated problem and point out, in some detail, why their solutions are wrong.

We characterise the classes of continuous and discrete phase-type distributions in the following way. They are known to be closed under convolutions, mixtures, and the unary ‘geometric mixture' operation. We show that the continuous class is the smallest family of distributions that is closed under these operations and contains all exponential distributions and the point mass at zero. An analogous result holds for the discrete class.

We also show that discrete phase-type distributions can be regarded as ℝ+-rational sequences, in the sense of automata theory. This allows us to view our characterisation of them as a corollary of the Kleene–Schützenberger theorem on the behavior of finite automata. We prove moreover that any summable ℝ+-rational sequence is proportional to a discrete phase-type distribution.

In this paper, we consider kth-order two-state Markov chains {Xi} with stationary transition probabilities. For k = 1, we construct in detail an upper bound for the total variation d(Sn, Y) = Σx |𝐏(Sn = x) − 𝐏(Y = x)|, where Sn = X1 + · ··+ Xn and Y is a compound Poisson random variable. We also show that, under certain conditions, d(Sn, Y) converges to 0 as n tends to ∞. For k = 2, the corresponding results are given without derivation. For general k ≧ 3, a conjecture is proposed.

The paper deals with a problem which arises in the Monte Carlo optimization of steady state or ergodic systems which can be modelled by Markov chains. The transition probability depends on a parameter, and one wishes to find the parameter value at which some performance function is minimum. The only available data are obtained from either simulation or actual operating information. For such a problem one needs good statistical estimates of the derivatives. Conditions are given for the existence of the derivative of the stationary measure with respect to the parameter, in the sense that the derivative is a signed measure, and is the limit of the natural approximating sequence. Some properties and a useful characterization of the derivative are obtained. It is also shown that, under appropriate conditions, the derivative of the n-step transition function converges to the derivative of the stationary measure as n tends to ∞. This latter result is of particular importance whether one is simply estimating or is actually optimizing via some sequential Monte Carlo procedure, since the basic observations are always taken over a finite time interval.

Assume that we want to estimate – σ, the abscissa of convergence of the Laplace transform. We show that no non-parametric estimator of σ can converge at a faster rate than (log n)–1, where n is the sample size. An optimal convergence rate is achieved by an estimator of the form where xn = O(log n) and is the mean of the sample values overshooting xn. Under further parametric restrictions this (log n)–1 phenomenon is also illustrated by a weak convergence result.

The paper puts forward steady-state Markov chain models for the Heine and Euler distributions. The models for oil exploration strategies that were discussed by Benkherouf and Bather (1988) are reinterpreted as current-age models for discrete renewal processes. Steady-state success-runs processes with non-zero probabilities that a trial is abandoned, Foster processes, and equilibrium random walks corresponding to elective M/M/1 queues are also examined.

Mathematical models have been proposed for oil exploration and other kinds of search. They can be used to estimate the amount of undiscovered resources or to investigate optimal stopping times for the search. Here we consider a continuous search for hidden objects using a model which represents the number and values of the objects by mixtures of Poisson processes. The flexibility of the model and its complexity depend on the number of components in the mixture. In simple cases, optimal stopping rules can be found explicitly and more general qualitative results can sometimes be obtained.

This paper introduces several versions of starting-stopping problem for the diffusion model defined in terms of a stochastic differential equation. The problem could be regarded as a stochastic differential game in which the player can only decide when to start the game and when to quit the game in order to maximize his fortune. Nested variational inequalities arise in studying such a problem, with which we are able to characterize the value function and to obtain optimal strategies.