We consider two random sequential packing processes in which spheres of unit radius are randomly attached to the surface of a fixed unit sphere. Independent random spheres are generated and added successively, provided there is no overlap with previous spheres. In model 1, the process stops when a trial sphere intersects one of the previously-accepted spheres. In model 2, random sequential packing, any such overlapping trial sphere is discarded and the next random sphere is tried, until it is impossible to add any further spheres.

Previous workers have conjectured convincingly that no exact analytical solution is possible for this type of problem. We use Monte Carlo simulation methods to estimate transition probabilities for the two models. Because some probabilities are extremely small, a simulation using independent repetitions of the model would be inefficient. We designed a branching process of conditionally binomial trials, and performed over 108 trials on a supercomputer.

This paper extends the rate conservation principle to cadlag processes whose jumps form a non-stationary point process with a time-dependent intensity. It is shown that this is a direct consequence of path integration and the strong law of large numbers for local martingales. When specialized to mean rates a non-stationary version of Miyazawa's result is obtained which is recovered in the stationary case. Some applications of the result are also given.

This paper presents a unified treatment of the convergence properties of nonhomogeneous Markov systems under different sets of assumptions. First the periodic case is studied and the limiting evolution of the individual cyclically moving subclasses of the state space of the associated Markov replacement chain is completely determined. A special case of the above result is the aperiodic or strongly ergodic convergence. Two numerical examples from the literature on manpower planning highlight the practical aspect of the theoretical results.

For a given transition rate, i.e., a Q-matrix Q = (qij) on a countable state space, the uniqueness of the Q-semigroup P(t) = (Pij(t)), the recurrence and the positive recurrence of the corresponding Markov chain are three fundamental and classical problems, treated in many textbooks. As an addition, this paper introduces some practical results motivated from the study of a type of interacting particle systems, reaction diffusion processes. The main results are theorems (1.11), (1.17) and (1.18). Their proofs are quite straightforward.

The heaps process (also known as a Tsetlin library) provides a model for a self-regulating filing system. Items are requested from time to time according to their popularity and returned to the top of the heap after use. The size-biased permutation of a collection of popularities is a particular random permutation of those popularities, which arises naturally in a number of applications and is of independent interest. For a slightly non-standard formulation of the heaps process we prove that it converges to the size-biased permutation of its initial distribution. This leads to a number of new characterizations of the property of invariance under size-biased permutation, notably what might be described as invariance under ‘partial size-biasing' of any order. Finally we consider in detail the heaps process with Poisson–Dirichlet initial distribution, exhibiting the tractable nature of its equilibrium distribution and explicitly calculating a number of quantities of interest.

We study a search model in which offers of random size are received randomly over time. The arrival times form a point process of a certain type, and the offer size distribution may depend on the corresponding arrival time. The search costs have a time-dependent cost rate. The objective is to stop the search process such that the expected discounted net reward (associated with the maximal offer received so far) is maximized. A stopping time σ is suggested, and conditions are specified under which σ turns out to be optimal.

Intuitively, the effect of conditioning a one-dimensional process to remain below a certain (possibly time-dependent) boundary is to ‘push' the process downwards. This paper investigates the effect of such conditioning, and finds the class of processes for which our intuition is accurate. It is found that ordinary stochastic inequalities are in general unsuitable for making statements about such conditioned processes, and that a stronger type of inequality is more appropriate.

The investigation is motivated by applications in estimation of boundary hitting time distributions.

We introduce multivariate partial orderings related with the Palm and time-stationary probabilities of a point process. Using these orderings, we give conditions for the monotonicity of a random sequence, with respect to some integral stochastic ordering, to be inherited with a continuous time process in which this sequence is imbedded. This type of inheritance is also discussed for the property of association.

In this paper we consider the Markov chain formed by the operation of the move-to-front scheme. We show that the eigenvalues of the transition probability matrix are of the form pi, pi + pj, ···, where pi is the probability of selecting the ith item and N is the number of items; further, that the multiplicity of the eigenvalues of the form Σpi where the summation is over m items is equal to the number of permutations of N – m objects, ordered in some way, such that no object is in its natural position. Finally, we show that the Markov chain is lumpable – many times over.

We consider the adaptive control of Markov chains under the weak accessibility condition with a view to minimizing the learning loss. A certainty equivalence control with a forcing scheme is constructed. We use a stationary randomized control scheme for forcing and compute a maximum likelihood estimate of the unknown parameter from the resulting observations. We obtain an exponential upper bound on the rate of decay of the probability of error. This allows us to choose the rate of forcing appropriately, whereby we achieve a o(f(n) log n) learning loss for any function as .

Stochastic processes {X(t)} of the form X(t) = Σ n f(t – Tn) are considered, where {Tn} is a stationary Poisson point process with intensity λ and f: R → R is an unknown response function. Conditions are obtained for weak consistency and asymptotic normality of estimators of λ based on long-run observations of {X(t)}.

In a finite, aperiodic, irreducible Markov chain, a visit to some states generates a one; to some others, a zero, while the remaining states are considered silent, in that neither symbol is generated. States during which either a one or a zero is generated are called active states, and sojourns in the set of active states correspond to messages. The output process is called a Markovian packet stream. Informally, a stream is called thin, if the steady-state fraction of time spent in the active states is small and messages are separated by silent periods of long durations. A limit theorem for the superposition of a large number of independent, stochastically identical and appropriately thin Markovian packet streams is obtained. The class of limit processes consists of a family of stationary, integer-valued, discrete-parameter processes of dependent Poisson random variables. Some properties of the limit processes are established.

We consider the SETAR(l; 1, ···, 1) model: where – ∞= r0 < r1, < · ·· < rl = ∞ and for each j {ε t(j)} forms an i.i.d. zero-mean error sequence independent of {ε t(i)} for i ≠ j and having a density positive on the real line. Chan et al. (1985) obtained the region of the parameter space on which the process is ergodic, and showed the process to be transient on a subset of the remainder. They conjectured that the process was null recurrent everywhere else. In this paper we show that conjecture to be incorrect and under the assumption of finite variance of the error distributions we resolve the remaining questions of transience or null recurrence for this process.

We find the first-hit distributions of symmetric randomized random walks on ℝ with exponential lifetime using prediction formulas for Gaussian stationary autoregressive processes, associated with the random walks.

Suppose that π is a policy for resource allocation in a stochastic environment and π ∗ is an optimal policy. Two existing procedures for policy evaluation are described and compared. Both of these evaluate π by means of upper bounds on R(π ∗) – R(π), the total reward lost when making resource allocations according to π rather than π∗. The bounds developed by these two methods are called Type 1 and Type 2. We demonstrate by example that neither of these procedures dominates the other in the sense of always yielding tighter bounds. A modification to Type 2 bounds is proposed resulting in an improved procedure which always dominates the Type 1 approach.

We are given a Markov chain with states 0, 1, 2, ···. We want to get a numerical approximation of the steady-state balance equations. To do this, we truncate the chain, keeping the first n states, make the resulting matrix stochastic in some convenient way, and solve the finite system. The purpose of this paper is to provide some sufficient conditions that imply that as n tends to infinity, the stationary distributions of the truncated chains converge to the stationary distribution of the given chain. Our approach is completely probabilistic, and our conditions are given in probabilistic terms. We illustrate how to verify these conditions with five examples.

Beurling's inequality is proved to have an interpretation in terms of ingredients that do not depend on the effective numerical contribution of transition probabilities of circuit processes (those processes whose transition law is expressed by a denumerable class of directed weighted circuits). Connections with reversibility for Sk-state, k≧2, circuit processes, where S is a denumerable set, are revealed.

The classical proofs for the existence of a stationary (s, S) inventory policy that minimizes the total discounted or average cost over an infinite horizon are lengthy because they depend heavily on the optimality results for corresponding finite-horizon models. This note presents a simpler alternative. Since optimal stationary (s, S) policies are relatively simple to characterize, it is easy to construct a solution to the optimality equation which is satisfied by an (s, S) policy or an equivalent variant thereof. For the discounted model, the proof characterizes an (s, S) policy that is optimal for all initial inventory positions. This policy can be generated by a simple existing algorithm. For the average-cost model, the optimality proof is completed with some additional arguments, which are simple but novel, to overcome the normal difficulties encountered in models with unbounded one-step expected costs.

We deal with the distribution of the first zero Rn of the real part of the empirical characteristic process related to a random variable X. Depending on the behaviour of the theoretical real part of the underlying characteristic function, cases with a slow exponential decrease to zero are considered. We derive the limit distribution of Rn in this case, which clarifies some recent results on Rn in relation to the behaviour of the characteristic function.

A multivariate reward process defined on a semi-Markov process is studied. Transform results for the distributions of the multivariate reward and related processes are derived through the method of supplementary variables and the Markov renewal equations. These transform results enable the asymptotic behavior to be analyzed. A class of first-passage time distributions of the multivariate reward processes is also investigated.