This paper establishes a new version of integration by parts formula of Markov chains for sensitivity computation, under much lower restrictions than the existing researches. Our approach is more fundamental and applicable without using Girsanov theorem or Malliavin calculus as did by past papers. Numerically, we apply this formula to compute sensitivity regarding the transition rate matrix and compare with a recent research by an IPA (infinitesimal perturbation analysis) method and other approaches.

In this paper, we mainly study a class of small deviation theorems for Markov chains indexed by an infinite tree with uniformly bounded degree in Markovian environment. Firstly, we give the definition of Markov chains indexed by a tree with uniformly bounded degree in random environment. Then, we introduce the some lemmas which are the basis of the results. Finally, a class of small deviation theorems for functionals of random fields on a tree with uniformly bounded degree in Markovian environment is established.

The purpose of this paper is to establish some limit theorems of delayed averages for countable nonhomogeneous Markov chains. The definition of the generalized C-strong ergodicity and the generalized uniformly C-strong ergodicity for countable nonhomogeneous Markov chains is introduced first. Then a theorem about the generalized C-strong ergodicity and the generalized uniformly C-strong ergodicity for the nonhomogeneous Markov chains is established, and its applications to the information theory are given. Finally, the strong law of large numbers of delayed averages of bivariate functions for countable nonhomogeneous Markov chains is proved.

In this paper, we give the definition of tree-indexed Markov chains in random environment with countable state space, and then study the realization of Markov chain indexed by a tree in random environment. Finally, we prove the strong law of large numbers and Shannon–McMillan theorem for Markov chains indexed by a Cayley tree in a Markovian environment with countable state space.

In this paper, we main consider spherically symmetric tree T. First, under the condition lim supn→∞ |T(n)|/|Ln|<∞, we investigate the strong law of large numbers (SLLNs) for T-indexed Markov chains on the nth level of T. Then, combining the Stolz theorem, we obtain the SLLNs on T. Finally, we get Shannon–McMillan theorem for T-indexed Markov chains. The obtained theorems are generalizations of some known results on Cayley tree TC, N and Bethe tree TB, N.