Candidates arrive sequentially for an interview process which results in them being ranked relative to their predecessors. Based on the ranks available at each time, a decision mechanism must be developed that selects or dismisses the current candidate in an effort to maximize the chance of selecting the best. This classical version of the ‘secretary problem’ has been studied in depth, mostly using combinatorial approaches, along with numerous other variants. We consider a particular new version where, during reviewing, it is possible to query an external expert to improve the probability of making the correct decision. Unlike existing formulations, we consider experts that are not necessarily infallible and may provide suggestions that can be faulty. For the solution of our problem we adopt a probabilistic methodology and view the querying times as consecutive stopping times which we optimize with the help of optimal stopping theory. For each querying time we must also design a mechanism to decide whether or not we should terminate the search at the querying time. This decision is straightforward under the usual assumption of infallible experts, but when experts are faulty it has a far more intricate structure.

The myopic strategy is one of the most important strategies when studying bandit problems. In 2018, Nouiehed and Ross put forward a conjecture about Feldman’s bandit problem (J. Appl. Prob. (2018) 55, 318–324). They proposed that for Bernoulli two-armed bandit problems, the myopic strategy stochastically maximizes the number of wins. In this paper we consider the two-armed bandit problem with more general distributions and utility functions. We confirm this conjecture by proving a stronger result: if the agent playing the bandit has a general utility function, the myopic strategy is still optimal if and only if this utility function satisfies reasonable conditions.

The rich-get-richer rule reinforces actions that have been frequently chosen in the past. What happens to the evolution of individuals’ inclinations to choose an action when agents interact? Interaction tends to homogenize, while each individual dynamics tends to reinforce its own position. Interacting stochastic systems of reinforced processes have recently been considered in many papers, in which the asymptotic behavior is proven to exhibit almost sure synchronization. In this paper we consider models where, even if interaction among agents is present, absence of synchronization may happen because of the choice of an individual nonlinear reinforcement. We show how these systems can naturally be considered as models for coordination games or technological or opinion dynamics.

We study the Wiener disorder detection problem where each observation is associated with a positive cost. In this setting, a strategy is a pair consisting of a sequence of observation times and a stopping time corresponding to the declaration of disorder. We characterize the minimal cost of the disorder problem with costly observations as the unique fixed point of a certain jump operator, and we determine the optimal strategy.

We investigate the impact of Knightian uncertainty on the optimal timing policy of an ambiguity-averse decision-maker in the case where the underlying factor dynamics follow a multidimensional Brownian motion and the exercise payoff depends on either a linear combination of the factors or the radial part of the driving factor dynamics. We present a general characterization of the value of the optimal timing policy and the worst-case measure in terms of a family of explicitly identified excessive functions generating an appropriate class of supermartingales. In line with previous findings based on linear diffusions, we find that ambiguity accelerates timing in comparison with the unambiguous setting. Somewhat surprisingly, we find that ambiguity may lead to stationarity in models which typically do not possess stationary behavior. In this way, our results indicate that ambiguity may act as a stabilizing mechanism.

We consider a fractional Brownian motion with linear drift such that its unknown drift coefficient has a prior normal distribution and construct a sequential test for the hypothesis that the drift is positive versus the alternative that it is negative. We show that the problem of constructing the test reduces to an optimal stopping problem for a standard Brownian motion obtained by a transformation of the fractional Brownian motion. The solution is described as the first exit time from some set, and it is shown that its boundaries satisfy a certain integral equation, which is solved numerically.

We study a renewal theory approach to perpetual two-state switching problems with infinite value functions. Since the corresponding value functions are infinite, the problems fall outside the standard class of problems which can be analyzed using dynamic programming. Instead, we propose an alternative formulation of optimal switching theory in which optimality of a strategy is defined in terms of its long-term mean return, which can be determined using renewal theory. The approach is illustrated by examples in connection with trend-following strategies in finance.

The nonstationary Erlang-A queue is a fundamental queueing model that is used to describe the dynamic behavior of large-scale multiserver service systems that may experience customer abandonments, such as call centers, hospitals, and urban mobility systems. In this paper we develop novel approximations to all of its transient and steady state moments, the moment generating function, and the cumulant generating function. We also provide precise bounds for the difference of our approximations and the true model. More importantly, we show that our approximations have explicit stochastic representations as shifted Poisson random variables. Moreover, we are also able to show that our approximations and bounds also hold for nonstationary Erlang-B and Erlang-C queueing models under certain stability conditions.

In the literature on optimal stopping, the problem of maximizing the expected discounted reward over all stopping times has been explicitly solved for some special reward functions (including (x+)ν, (ex − K)+, (K − e− x)+, x ∈ ℝ, ν ∈ (0, ∞), and K > 0) under general random walks in discrete time and Lévy processes in continuous time (subject to mild integrability conditions). All such reward functions are continuous, increasing, and logconcave while the corresponding optimal stopping times are of threshold type (i.e. the solutions are one-sided). In this paper we show that all optimal stopping problems with increasing, logconcave, and right-continuous reward functions admit one-sided solutions for general random walks and Lévy processes, thereby generalizing the aforementioned results. We also investigate in detail the principle of smooth fit for Lévy processes when the reward function is increasing and logconcave.

We consider the Bernoulli bandit problem where one of the arms has win probability α and the others β, with the identity of the α arm specified by initial probabilities. With u = max(α, β), v = min(α, β), call an arm with win probability u a good arm. Whereas it is known that the strategy of always playing the arm with the largest probability of being a good arm maximizes the expected number of wins in the first n games for all n, we conjecture that it also stochastically maximizes the number of wins. That is, we conjecture that this strategy maximizes the probability of at least k wins in the first n games for all k, n. The conjecture is proven when k = 1, and k = n, and when there are only two arms and k = n - 1.

A classical Pólya urn scheme is a Markov process where the evolution is encoded by a replacement matrix (Ri, j)1 ≤ i, j ≤ d. At every discrete time-step, we draw a ball uniformly at random, denote its colour c, and replace it in the urn together with Rc, j balls of colour j (for all 1 ≤ j ≤ d). We study multiple drawing Pólya urns, where the replacement rule depends on the random drawing of a set of m balls from the urn (with or without replacement). Many particular examples of this situation have been studied in the literature, but the only general results are due to Kuba and Mahmoud (2017). These authors proved second-order asymptotic results in the two-colour case, under the so-called balance and affinity assumptions, the latter being somewhat artificial. The main idea of this work is to apply stochastic approximation methods to this problem, which enables us to prove analogous results to Kuba and Mahmoud, but without the artificial affinity hypothesis, and, for the first time in the literature, in the d-colour case (d ≥ 3). We also provide some partial results in the two-colour nonbalanced case, the novelty here being that the only results for this case currently in the literature are for particular examples.

In this paper we consider a variation of the full-information secretary problem where the random variables to be observed are independent but not necessary identically distributed. The main result is a sharp lower bound for the optimal win probability. Precisely, if X1,...,Xn are independent random variables with known continuous distributions and Vn(X1,...,Xn):=supτℙ(Xτ=Mn), where Mn≔max{X1,...,Xn} and the supremum is over all stopping times adapted to X1,...,Xn then Vn(X1,...,Xn)≥(1-1/n)n-1, and this bound is attained. The method of proof consists in reducing the problem to that of a sequence of random variables taking at most two possible values, and then applying Bruss' sum-the-odds theorem, Bruss (2000). In order to obtain a sharp bound for each n, we improve Bruss' lower bound, Bruss (2003), for the sum-the-odds problem.

The best-choice problem and the duration problem, known as versions of the secretary problem, are concerned with choosing an object from those that appear sequentially. Let (B,p) denote the best-choice problem and (D,p) the duration problem when the total number N of objects is a bounded random variable with prior p=(p1, p2,...,pn) for a known upper bound n. Gnedin (2005) discovered the correspondence relation between these two quite different optimal stopping problems. That is, for any given prior p, there exists another prior q such that (D,p) is equivalent to (B,q). In this paper, motivated by his discovery, we attempt to find the alternate correspondence {p(m),m≥0}, i.e. an infinite sequence of priors such that (D,p(m-1)) is equivalent to (B,p(m)) for all m≥1, starting with p(0)=(0,...,0,1). To be more precise, the duration problem is distinguished into (D1,p) or (D2,p), referred to as model 1 or model 2, depending on whether the planning horizon is N or n. The aforementioned problem is model 1. For model 2 as well, we can find the similar alternate correspondence {p[m],m≥ 0}. We treat both the no-information model and the full-information model and examine the limiting behaviors of their optimal rules and optimal values related to the alternate correspondences as n→∞. A generalization of the no-information model is given. It is worth mentioning that the alternate correspondences for model 1 and model 2 are respectively related to the urn sampling models without replacement and with replacement.

The Marcinkiewicz strong law, limn→∞(1 / n1/p)∑k=1n(Dk - D) = 0 almost surely with p ∈ (1, 2), is studied for outer products Dk = {XkX̅kT}, where {Xk} and {X̅k} are both two-sided (multivariate) linear processes (with coefficient matrices (Cl), (C̅l) and independent and identically distributed zero-mean innovations {Ξ} and {Ξ̅}). Matrix sequences Cl and C ̅l can decay slowly enough (as |l| → ∞) that {Xk,X ̅k} have long-range dependence, while {Dk} can have heavy tails. In particular, the heavy-tail and long-range-dependence phenomena for {Dk} are handled simultaneously and a new decoupling property is proved that shows the convergence rate is determined by the worst of the heavy tails or the long-range dependence, but not the combination. The main result is applied to obtain a Marcinkiewicz strong law of large numbers for stochastic approximation, nonlinear function forms, and autocovariances.

The full-information duration problem with a random number N of objects is considered. These objects appear sequentially and their values Xk are observed, where Xk, independent of N, are independent and identically distributed random variables from a known continuous distribution. The objective of the problem is to find a stopping rule that maximizes the duration of holding a relative maximum (e.g. the kth object is a relative maximum if Xk = max{X1, X2, . . ., Xk}). We assume that N is a random variable with a known upper bound n, so two models, Model 1 and Model 2, can be considered according to whether the planning horizon is N or n. The structure of the optimal rule, which depends on the prior distribution assumed on N, is examined. The monotone rule is defined and a necessary and sufficient condition for the optimal rule to be monotone is given for both models. Special attention is paid to the class of priors such that N / n converges, as n → ∞, to a random variable Vm having density fVm(v) = m(1 - v)m-1, 0 ≤ v ≤ 1 for a positive integer m. An interesting feature is that the optimal duration (relative to n) for Model 2 is just (m + 1) times as large as that for Model 1 asymptotically.

In an optimal variance stopping problem the goal is to determine the stopping time at which the variance of a sequentially observed stochastic process is maximized. A solution method for such a problem has been recently provided by Pedersen (2011). Using the methodology developed by Pedersen and Peskir (2012), our aim is to show that the solution to the initial problem can be equivalently obtained by constraining the variance stopping problem to the expected size of the stopped process and then by maximizing the solution to the latter problem over all the admissible constraints. An application to a diffusion process used for modeling the dynamics of interest rates illustrates the proposed technique.

As a class of optimal stopping problems with monotone thresholds, we define the candidate-choice problem (CCP) and derive two formulae for calculating its expected payoff. We apply the first formula to a particular CCP, i.e. the best-choice duration problem treated by Ferguson et al. (1992). The recall case is also examined as a comparison. We also derive the distribution of the stopping time of the CCP and find, as a by-product, that the best-choice problem has a remarkable feature in that the optimal probability of choosing the best is just the expected value of the (proportional) stopping time. The similarity between the best-choice duration problem and the best-choice problem with uniform freeze studied by Samuel-Cahn (1996) is recognized.