One considers semi-Markov processes using phase-type distributions, such that the marginal distribution of the interevent times are identically distributed but not independent. By suitably choosing the phase-type distributions, one obtains an exponential marginal distribution.

A numerical method is proposed for computing the stationary probabilities of a Markovian system consisting of two subsystems with slight interrelation. It exploits the independent features of the subsystems for evaluating the effects of their interrelation. In a test with a simple example, the numerical method proposed required less computational burden than other common numerical methods.

A general method is developed for giving simulation estimates of the probability ψ(u, T) of ruin before time T. When the probability law P governing the given risk reserve process is imbedded in an exponential family (Pθ), one can write ψ(u, T) = EθRθ for certain random variables Rθ given by the fundamental identity of sequential analysis. Using this to simulate from Pθ rather than P, it is possible not only to overcome the difficulties connected with the case T = ∞, but also to obtain a considerable variance reduction. It is shown that the solution of the Lundberg equation determines the asymptotically optimal value of θ in heavy traffic when T = ∞, and some results guidelining the choice of θ when T < ∞ are also given. The potential of the method in complex models is illustrated by two examples.

Since the late 1960s Wiener's theory on the non-linear functionals of white noise has been widely applied to the construction of mathematical models of non-linear systems, especially in the field of biology. For such applications the main part is the measurement of Wiener's kernels, for which two methods have been proposed: one by Wiener himself and the other by Lee and Schetzen. The aim of this paper is to show that there is another method based on Hida's differential operator.

Consider the following situation: a computer program which presumably contains a certain number of errors is tested over a given period of time, in order to infer some conclusions about its behaviour when used by future customers.

This paper derives an algorithm for finding a scheduling discipline for scheduling N customers at M service stations in a closed queueing network so as to optimize a performance measure which depends on the configuration of customers at the service stations.

The ideas previously used (Beneš (1981)) to construct some finite-dimensional non-linear filters also yield related new filters of finite dimension with arbitrarily large bases; this is because the finite dimensionality is not destroyed by insertion of noiseless linear differential operations on the observations.

Piecewise-deterministic (PD) Markov processes consist of a mixture of deterministic motion and random jumps, the jumps occurring ‘spontanteously’ in a Poisson-like fashion and also ‘surely’ when the process hits certain boundaries. They form a flexible class of stochastic models, including as special cases almost all the non-diffusion models of applied probability. The construction of these processes will be outlined and precise characterization of the extended generator given.

We consider an analogue of classical calculus of variations, in which deterministic comparison curves are replaced by stochastic processes xt, of the form with ut a control applied at time t and wt a brownian motion. Connections with stochastic mechanics and non-linear filtering are discussed, as well as the deterministic limit as λ → 0.

The stochastic control problem of tracking a Brownian motion by a non-decreasing process (monotone follower) is related to a question of optimal stopping. Direct probabilistic arguments are employed to show that the two problems are equivalent and that both admit optimal solutions.

The stochastic variability is investigated in the experimental results from an acoustic grain loss monitor and a table auger torque sensor on a combine harvester. A bivariate first-order autoregressive first-order moving-average model is identified and used to specify optimal control systems for adjusting combine forward speed to maintain constant threshing losses. These provide upper bounds on the benefits of practical forward speed control systems.

A device is subject to a series of shocks which cause damage and eventually failure will occur at the time of arrival of one of the shocks. In between the shocks, the device is partially repaired as the cumulative damage decreases as some Markov process. The device must be replaced upon failure at some cost but it also can be replaced before failure at a lower cost. We consider the general case where the failure rate need not be increasing and replacement can be made at any time. The form of the optimal replacement policy is found and fairly general conditions are given for which a control limit policy is optimal.

The method of discrete-time approximation is widespread in control and decision theory. However, little attention has been paid to the conditions on parameters and control under which the discrete-time systems come close to the continuous-time system.

The stochastic control problem of tracking a Brownian motion by a process of bounded variation is reduced to a control problem with reflection at the origin, and the latter is related to a question of optimal stopping of Brownian motion absorbed at the origin. Direct probabilistic arguments can be used to show equivalences between the various problems.

We consider the optimal control of deterministic processes with countably many (non-accumulating) random iumps. A necessary and sufficient optimality condition can be given in the form of a Hamilton-jacobi-Bellman equation which is a functionaldifferential equation with boundary conditions in the case considered. Its solution, the value function, is continuously differentiable along the deterministic trajectories if. the random jumps only are controllable and it can be represented as a supremum of smooth subsolutions in the general case, i.e. when both the deterministic motion and the random jumps are controlled (cf. the survey by M. H. A. Davis (p.14)).

Most time series models in hydrology are used for river flow forecasting, for generation of synthetic data sequences or for the study of physical characteristics underlying the hydrological processes. The models are formulated as linear stochastic difference equations. Three phases are considered for the selection of a model based on a satisfactory representation of a given empirical time series: identification, estimation and validation. Several criteria have been proposed for the selection of the order of ARMA models. The Akaike information criterion (Ale) is popular among hydrologists, but the posterior probability criterion has the advantage of optimality and asymptotic consistency. There are numerous applications of AR or ARMA models to annual streamflow series which are stationary. Seasonal, monthly, weekly or daily streamflow series are cyclically stationary and generally exhibit periodicities in the mean and variance and possibly in the autocorrelation structure. Removal of the periodicity has been accomplished by fitting harmonic series or by subtracting the seasonal mean and dividing by the seasonal standard deviation, and a time series model is then fitted to the residual series. Alternatively, ARMA models with time-varying coefficients are also used. The multiplicative ARlMA model of Box and Jenkins is less frequent in hydrology because of the difficulty in the identification of the parameter structure. Multivariate models are used when river flows at different sites are considered. Parameter estimation in multivariate time series models can become cumbersome because of the dimensionality of the problem. Often the covariance matrix of the noise term is not known in advance and limited information estimates are used. Multivariate models have been used for annual and monthly series. Disaggregation models have been used to subdivide a yearly series into monthly or weekly series or to disaggregate a main river flow into tributary flows while maintaining certain space and time cross-correlations. The aggregation of monthly into yearly time series has been shown to improve the parameter estimation of the yearly series. Hydrologic time series occasionally exhibit changes in level due to natural or man-made causes such as forest fires, volcanic eruption, climatological change, urbanization etc. These situations can be treated making use of intervention analysis.

Consider a saturated porous m edium in which water is flowing slowly with a steady velocity. Suppose at some space-time scale the concentration C(x, r) of a non-reactive dilute solute is governed by the following Fokker-Planck differential equation: