A semi-Markov decision process, with a denumerable multidimensional state space, is considered. At any given state only a finite number of actions can be taken to control the process. The immediate reward earned in one transition period is merely assumed to be bounded by a polynomial and a bound is imposed on a weighted moment of the next state reached in one transition. It is shown that under an ergodicity assumption there is a stationary optimal policy for the long-run average reward criterion. A queueing network scheduling problem, for which previous criteria are inapplicable, is given as an application.

The paper characterizes point processes with the order statistic property without the unnecessary condition of finiteness of the first moment of the process, a condition imposed by previous researchers. It shows that the class of these processes is composed only of mixed Poisson processes up to a time-scale transformation and of the mixed sample processes. It also introduces a multivariate analog of the order statistic property and characterizes completely the class of multivariate point processes with this property.

Random digits are collected one at a time until a given k -digit sequence is obtained, or, more generally, until one of several k -digit sequences is obtained. In the former case, a recursive formula is given, which determines the distribution of the waiting time until the sequence is obtained and leads to an expression for the probability generating function. In the latter case, the mean waiting time is given until one of the given sequences is obtained, or, more generally, until a fixed number of sequences have been obtained, either different sequences or not necessarily different ones. Several results are known before, but the methods of proof seem to be new.

The species-area problem in biology and the type-token problem in literary studies are analogues of one another but have nearly disjoint literatures. Here their relationship is treated, a critique of models used in each is made and Markovian models are provided for two well-known empirical models from the species-area literature. Finally, some of the assumptions underlying each of these models are considered in the light of literary data.

We study the situation in which individuals occur in ‘families' or similar groups, individuals within a ‘family' being correlated with one another, as for example a biological population. In such a population, the number of individuals will usually vary from one family to another. We assume that a sample chosen from the population consists of whole families rather than unrelated individuals. A similar situation might occur in experimental design, if individuals occur in ‘blocks' (of varying sizes) within which they share a common environment. In the simplest case considered here two measurements are made on each individual, namely y, the character of interest, and x (not necessarily a random variable), some other character which is believed to influence y. We discuss how to estimate the regression of y on x, and the within and between family variance components for y (and hence the intrafamily correlation) when the effect of x is eliminated. Generalizations of this are briefly discussed.

Formulae are presented for the variance and higher moments of the present value of single-stage rewards in a finite Markov decision process. Similar formulae are exhibited for a semi-Markov decision process. There is a short discussion of the obstacles to using the variance formula in algorithms to maximize the mean minus a multiple of the standard deviation.

The optimal control of dynamic models which are not completely known to the controller often requires some kind of estimation of the unknown parameters. We present conditions under which a minimum contrast estimator will be strongly consistent independently of the control used. This kind of estimator is appropriate for the adaptive or ‘estimation and control' approach in dynamic programming under uncertainty. We consider a countable-state Markov renewal model and we impose bounding and recurrence conditions of the so-called Liapunov type.

Two distributions, F and G, are said be ordered in dispersion if F-1(β)-F-1(α)≦G-1(β)-G-1(α) whenever 0<α <β <1. This relation has been studied by Saunders and Moran (1978). The purpose of this paper is to study this partial ordering in detail. Few characterizations of this concept are given. These characterizations are then used to show that some particular pairs of distributions are ordered by dispersion. In addition to it some proofs of results of Saunders and Moran (1978) are simplified. Furthermore, the characterizations of this paper can be used to throw a new light on the meaning of the underlying partial ordering and also to derive various inequalities. Several examples illustrate the methods of this paper.

It is shown that, for one model of genetic evolution, the times at which mutations at nucleotide sites become fixed in a population tend to cluster together rather than behave as a Poisson process.

We propose an estimator for the number of sources, N, which transmit signals according to a Poisson process with a common rate μ. This estimation problem was originated by Starr and pursued by Vardi. Our procedure is a ‘double sampling’ one and has the advantage that it can be used when μ is unknown.

The decision-maker drives a car along a straight highway towards his destination and looks for a parking place. When he finds a parking place, he can either park there and walk the distance to his destination or continue driving. Parking places are assumed to occur in accordance with a Poisson process along the highway. The decision-maker does not know the distance Y to his destination exactly in advance. Only an a priori distribution is assumed for Y and cases of typically important distribution are examined. When we take as loss the distance the decision-maker must walk and wish to minimize the expected loss, the optimal stopping rule and the minimum expected loss are obtained. In Section 3 a generalization to the cases of a non-homogeneous Poisson process and a renewal process is considered.

Consider the placement of a finite number of arcs on the circle of circumference 2π where the midpoint and length of each arc follows an arbitrary bivariate distribution. In the case where each arc has lengthπ, the probability that the circle is completely covered is equal to the probability that the convex hull of a finite random sample of points, chosen according to a certain bivariate distribution in the plane contains the origin. In general, we show that evaluating the probability that the random convex hull contains a fixed disc is equivalent to solving the general coverage problem where the midpoint and length of each arc follows an arbitrary bivariate distribution. Exact formulae for the above probabilities are obtained and some examples are considered.

Cars entering and leaving a parking place in a forest are registered automatically in connection with an investigation of forest recreational use. By subtraction the number of cars parked is computed. Due to erroneous and missing counts, errors occur and may for instance indicate a negative number. The theory of incompletely observed Markov chains is used to obtain an estimate of the number of cars parked.

A second guesser, G2, proposes to guess ‘smaller' or ‘larger' on each of n contests, i, according as , or where Xi, i = 1, ···, n are the revealed guesses of the first guesser, G1. G2 wins contest i if his assertion about the size of the target quantity θi, is more accurate than that of G1, Xi, i = 1, ···, n. Otherwise G 1 is the winner. Laws of large numbers are derived for G2's win totals for arbitrary configurations of θi. Limiting distributions are obtained for the first guesser's win totals in two special cases, the first where the θi are highly concentrated and the second where they are widely dispersed. G2 enjoys an enormous advantage over G1 in the former case, as intuition would suggest.

This article discusses certain aspects of determining heritability estimates from the genetic analyses of data. First, a brief review is given of the major statistical techniques for fitting models to quantitative family data. Then, by reference to a general schematic framework, the basic models that have been proposed for such data will be described. Some new environmental models are also developed. Finally, by reference to some data sets, the interactive approach to data analysis is proposed, advocating the determination of adequacy of fit of all reasonable models to available data.

The problem of optimal stopping in a Markov chain when there is imperfect state information is formulated as a partially observable Markov decision process. Properties of the optimal value function are developed. It is shown that under mild conditions the optimal policy is well structured. An efficient algorithm, which uses the structural information in the computation of the optimal policy, is presented.

The question whether a convex closed curve can be approximated by ellipses having a large number of foci is considered. It is shown that the limiting, convex figure of multifocal ellipses may have only one single straight segment. This happens only in the case, when the foci tend partly to infinity and partly to points of the line through the straight segment. The approximations of certain ‘distance integrals' are treated; the characterization of approximability remains an open problem.

Results and methods discussed in two previous papers are extended to other cases. Comparison is made with recent work by Pickard (1980) and an earlier conjecture is disproved.

The long-run average cost per unit time of operating a finite dam controlled by a PlM policy (Faddy (1974), Zuckerman (1977)) is determined when the cumulative input process is (a) a Wiener process with drift and (b) the integral of a Markov chain. It is shown how the cost for (a) can be obtained as the limit of the costs associated with a sequence of input processes of the type (b).

The coverage problem on the circle is considered from the shadowing process point of view. A random number of shadow arcs are distributed on a circle. The length of each arc is a random variable which depends on the random diameter of a shadowing disk and its random location. Formulae are derived for the numerical determination of the moments of the measure of vacancy of arcs on the circle, for a special example. An approximation to the distribution of the measure of vacancy is also provided.