## 1 Introduction

Distributed cooperative control of multi-agent systems has been a rapidly emerging topic in networked control theory over the past decades. Reaching consensus is a primary goal, which concerns designing control protocols such that all agents reach a common state or output via information interaction with their nearest neighbors. This control issue can be roughly categorized into leaderless consensus [Reference Fornasier, Lisini, Orrieri and Savaré10, Reference Olfati-Saber and Murray20] and tracking consensus (or leader-follower consensus) [Reference Gong, Han and Lan12, Reference Song, Cao and Yu28, Reference You, Hua, Li and Jia33]. The former considers agreement seeking in a situation where no leader is present, while the objective of the latter is for other agents to follow the trajectory of a leader. Leaderless and tracking consensus problems have been extensively studied in fixed networks [Reference Burbano, DeLellis and diBernardo5, Reference Olfati-Saber and Murray20, Reference Song, Cao and Yu28], switching or time-varying networks [Reference Gong, Han and Lan12, Reference You, Hua, Li and Jia33], and random networks [Reference Razaee, Parisini and Polycarpou21, Reference Shang24]. An overview of some classical control theory results on consensus problems can be found in the review paper [Reference Olfati-Saber and Murray20], and more recent advances are reviewed in the works [Reference Amirkhani and Barshooi2, Reference Becchetti, Clementi and Natale4] from a broader perspective of system science.

In terms of consensus problems in large-scale networked systems, a realistic difficulty lies in the malicious behavior of some unknown adversaries due to cyber-physical attacks. In computer science, the strongest adversaries are dubbed as Byzantine agents [Reference Lamport, Shostak and Pease17], who do not follow a desired coordination strategy and potentially ruled by some sophisticated attackers. Byzantine agents are notoriously difficult to cope with as they may even collude with other agents to arbitrarily distort information exchange in the network [Reference LeBlanc, Zhang, Koutsoukos and Sundaram18, Reference Mitra, Richards, Bagchi and Sundaram19]. In this setting, (tracking) consensus problems are referred to as resilient (tracking) consensus because the control objective is to ensure all cooperative agents to converge in spite of the interference of potential Byzantine agents. To filter out the potential malicious behaviors, a class of Weighted-Mean-Subsequence-Reduced (W-MSR) algorithm is proposed in [Reference LeBlanc, Zhang, Koutsoukos and Sundaram18] to remove extremal values in the neighboring area of any cooperative agent. It is shown that if the communication graph is sufficiently robust with respect to the maximum number of Byzantine neighbors, resilient consensus can be achieved. The idea is that when the graph is sufficiently connected, scrapping some extremal neighbors at each time step of the iteration could ensure the states of cooperative agents (i) lie within the convex hull of their initial values and (ii) are updated towards a final convergence.

The idea of W-MSR algorithms has been later generalized to cope with resilient consensus for agents with higher-order [Reference Dibaji and Ishii7] and asynchronous [Reference Senejohnny, Sundaram, Persis and Tesi23] dynamics as well as hybrid multi-agent systems with both continuous-time and discrete-time agents [Reference Shang25, Reference Shang27]. Trusted agents have been introduced in [Reference Abbas, Laszka and Koutsoukos1] to relax the robustness requirement in W-MSR algorithms, where trusted agents are assumed to be insusceptible to attacks. In [Reference Shang26], the problem of constrained resilient consensus is considered by employing a projection-based method to push the states of cooperative agents back to their constraint sets. To achieve resilient consensus on relatively sparse networks, two-hop communications have been introduced to multi-agent control in discrete-time [Reference Yuan and Ishii34] and continuous-time [Reference Zhao, Lv, Yu, Wen and Chen35].

On top of these deterministic system dynamics, some randomness has also been incorporated recently in resilient consensus problems to reflect varied challenges in real-world complex systems. Randomized quantized states [Reference Dibaji, Ishii and Tempo8] and random faulty links [Reference Razaee, Parisini and Polycarpou21] have been factored in under the framework of almost sure convergence. In [Reference Shang24], mean square convergence for a class of hybrid systems containing some randomly interacting agents as well as misbehaving ones has been examined. Mean square consensus for a group of cooperative agents is shown in [Reference Fiore and Russo9] under the stochastic influence of malicious agents with differential privacy of their initial states corrupted by communication noise.

Most of the existing research on resilient consensus problems including those above only studies leaderless consensus. The W-MSR type algorithms do not differentiate leaders from malicious attackers. These protocols ensure convergence to the convex hull of cooperative agents regardless of how leaders/malicious agents behave. As a result, tracking consensus cannot be achieved via a traditional W-MSR algorithm if a leader moves out of the convex hull. To overcome this difficulty, a set of leaders taking the same constant reference value is introduced in [Reference Usevitch and Panagou29] to guarantee resilient tracking consensus in discrete-time multi-agent systems. The minimum number of leaders is determined based upon the maximum number of Byzantine agents in the communication graph. This method is further extended in [Reference Usevitch and Panagou30] to allow some piecewise constant trajectories for leaders by using a sliding-window analysis. Leader-following resilient consensus has been investigated in [Reference Razaee, Parisini and Polycarpou22] with a single leader in continuous-time multi-agent systems by employing some saturation functions to regulate the errors between the leader and followers. The leader agent may take a time-dependent bounded trajectory, where the bounds and the leader’s identity are assumed to be known to all cooperative agents.

Along this line, we here further study the resilient tracking consensus problem in multi-agent systems in the presence of Byzantine agents. To overcome the restrictions in the existing works such as the large number and identifiability of the leaders, we here design a new W-MSR-type resilient consensus protocol for systems with a single leader and adopt a linear system approach to show that almost sure resilient tracking consensus can be achieved for discrete-time agents over directed random graphs. The main novelties of this work are as follows. Firstly, in the previous research on resilient tracking consensus [Reference Razaee, Parisini and Polycarpou22, Reference Usevitch and Panagou29, Reference Usevitch and Panagou30], the communication topology is assumed to be a fixed graph and the system dynamic is deterministic. Here, we lift these restrictions by considering a general class of random graphs with time-dependent edge probabilities. The resilient tracking consensus will be guaranteed by a martingale convergence result benefiting from the construction of our linear system and communication topology. Secondly, the proposed distributed control protocol accommodates not only the conventional linear diffusive mechanism (i.e. the Laplacian-type coupling) but also a general nonlinear coupling function, which is more general than most existing work on resilient consensus problems; see for example [Reference Abbas, Laszka and Koutsoukos1, Reference Fiore and Russo9, Reference LeBlanc, Zhang, Koutsoukos and Sundaram18, Reference Razaee, Parisini and Polycarpou21, Reference Shang25]. Thirdly, compared to the work in [Reference Razaee, Parisini and Polycarpou22], we assume the leader is a trustworthy node whose value is always trusted by the followers. The identity of the leader is not revealed to other nodes (see Remark 4 below).

The rest of the paper is organized as follows. Section 2 prepares some preliminaries on graph theory and introduces the model of our multi-agent system. Section 3 presents the resilient tracking consensus analysis. Section 4 contains numerical simulation examples, and Section 5 concludes the paper.

## 2 Problem formulation

### 2.1 Graph theory

In a multi-agent system, the interaction topology of agents (i.e. vertices) is represented by a time-varying directed graph
$\mathcal{G}(t)=(\mathcal{V},\mathcal{E}(t), \mathcal{A}(t))$
for
$t\in\mathbb{N}$
, where
$\mathbb{N}$
is the set of non-negative integers. The vertex set
$\mathcal{V}=\{1, 2, \cdots, N\}$
is composed of all agents, and the edge set
$\mathcal{E}(t)\subseteq\mathcal{V}\times\mathcal{V}$
describes the information interaction. Let
$\mathbb{R}$
be the set of reals. The adjacency matrix of
$\mathcal{G}(t)$
,
$\mathcal{A}(t)=(a_{ij}(t))\in\mathbb{R}^{N\times N}$
, is a weighted random matrix, where
$a_{ij}(t)>0$
when
$(\,j,i)\in\mathcal{E}(t)$
, that is, agent *j* can send information to agent *i* at time *t*, and
$a_{ij}(t)=0$
otherwise. We assume
$a_{ij}(t)>0$
with probability
$p_{ij}(t)$
and
$a_{ij}(t)=0$
with probability
$1-p_{ij}(t)$
. We mention that we do not require independence for the entries
$a_{ij}(t)$
in the adjacency matrix with respect to *i*, *j* or *t*. This gives desirable flexibility in practical applications.

Denote by
$\mathcal{N}_i^{\mathrm{in}}(t)=\{j\in\mathcal{V}:(\,j,i)\in\mathcal{E}(t)\}$
the set of in-neighbors of agent *i*. Similarly, the set of out-neighbors of *i* can be defined as
$\mathcal{N}^{\mathrm{out}}_i(t)=\big\{j\in\mathcal{V}:i\in\mathcal{N}^{\mathrm{in}}_j(t)\big\}$
. The agents in the graph
$\mathcal{G}(t)$
are divided into three categories, the cooperative agents in
$\mathcal{C}$
, the Byzantine agents in
$\mathcal{B}$
and a leader agent
$\ell$
. In other words,
$\mathcal{V}=\{\ell\}\cup\mathcal{C}\cup\mathcal{B}$
. The rules for these agents will be given in the next section. Fix
$t\in\mathbb{N}$
. A vertex set
$\mathcal{S}\subseteq\mathcal{V}$
is *r*-reachable at time *t* [Reference LeBlanc, Zhang, Koutsoukos and Sundaram18] if there exists a vertex
$i\in\mathcal{S}$
such that
$\big|\mathcal{N}^{\mathrm{in}}_i(t)\backslash\mathcal{S}\big|\geqslant r$
. The graph
$\mathcal{G}(t)$
is said to be leader-follower *r*-robust at time *t* [Reference Razaee, Parisini and Polycarpou22] if (i)
$\big|\mathcal{N}^{\mathrm{out}}_{\ell}(t)\big|\geqslant r$
and (ii) for any
$\mathcal{S}\subseteq\mathcal{V}\backslash\big(\{\ell\}\cup\mathcal{N}^{\mathrm{out}}_{\ell}(t)\big)$
,
$\mathcal{S}$
is *r*-reachable. The concept of leader-follower *r*-robust is a natural generalization to the *r*-robustness notation which is first introduced in [Reference LeBlanc, Zhang, Koutsoukos and Sundaram18]. A directed graph
$\mathcal{G}=(\mathcal{V},\mathcal{E})$
is called *r*-robust if for any two nonempty mutually exclusive sets
$\mathcal{S}_1,\mathcal{S}_2\subseteq\mathcal{V}$
, at least one of them is *r*-reachable.

### 2.2 Model description

Recall that there are three types of agents in the graph
$\mathcal{G}(t)$
: cooperative agents in
$\mathcal{C}$
, Byzantine agents in
$\mathcal{B}$
and the leader
$\ell$
satisfying
$\mathcal{V}=\{\ell\}\cup\mathcal{C}\cup\mathcal{B}$
. Byzantine is a traditional name that is used in computer science and control theory [Reference Lamport, Shostak and Pease17]. These agents are able to arbitrarily update their data and hence can potentially be malicious. Formally, Byzantine agents can include both faulty agents, which are due to random errors, and malicious agents, which may be controlled by some malicious attackers. Hence, Byzantine agents in
$\mathcal{B}$
are potentially harmful and pose a serious threat to the collective control objective of the multi-agent systems [Reference LeBlanc, Zhang, Koutsoukos and Sundaram18, Reference Mitra, Richards, Bagchi and Sundaram19, Reference Shang25, Reference Shang27]. They can collude with other Byzantine agents, and their identity and number are not available to the cooperative agents. Cooperative agents in
$\mathcal{C}$
on the other hand are those we have control upon. They usually take a diffusive agent dynamics, and the aim of resilient (tracking) consensus problem is to design appropriate distributed protocols for these cooperative agents so that they can reach an agreement in some sense despite the obstruction from potential Byzantine agents. The leader
$\ell$
is also a cooperative agent by nature, but it does not receive information from other agents, namely,
$\mathcal{N}_{\ell}(t)=\emptyset$
for all *t*. We will adopt a linear system design to enable the cooperative agents to track the leader’s trajectory in the sense of almost sure convergence.

Formally, the dynamics of any agent $i\in\mathcal{V}$ takes the following form

where
$A\in\mathbb{R}^{n\times n}$
,
$B\in\mathbb{R}^n$
,
$x_i(t)\in\mathbb{R}^n$
means the state of agent *i* and
$u_i(t)\in\mathbb{R}$
is its control input. Here, the linear system design enables us to accommodate vector state spaces (see e.g. [Reference Shang26]), which is more general than most of the existing work on resilient consensus problems. The control input
$u_i(t)$
for cooperative agents
$i\in\mathcal{C}$
will be designed below in (2.5). For a Byzantine agent
$i\in\mathcal{B}$
,
$u_i(t)$
can take any value [Using the unifying framework (2.1) for Byzantine agents is for the ease of presentation; essentially, no restriction is imposed on the behavior of Byzantine dynamics]. The leader agent
$\ell$
assumes
$u_{\ell}(t)=f(t)$
for some function
$f:\mathbb{N}\rightarrow\mathbb{R}$
satisfying the following assumption.

Assumption 1. We assume $\lim_{t\rightarrow\infty}x_{\ell}(t)=x_f$ for some $x_f\in\mathbb{R}^n$ , and $\|x_{\ell}(t)-x_f\|=o\big(t^{-1}\big)$ as $t\rightarrow\infty$ . Here, $\|\cdot\|$ is the Euclidean norm in $\mathbb{R}^n$ .

Remark 1. The convergence here is more general compared to the constant reference states required in [Reference Usevitch and Panagou29, Reference Usevitch and Panagou30]. Under Assumption 1, the state $x_{\ell}(t)$ is obviously bounded. Although the convergence assumption is more restrictive compared with the boundedness requirement in [Reference Razaee, Parisini and Polycarpou22], the bounds information has to be shared with all cooperative agents therein, which is difficult to realize in a distributed manner in practice. It is also worth noting that the convergence here is non-probabilistic because $\ell$ is not influenced by other agents through random edges.

Remark 2. A simple example that satisfies Assumption 1 is
$f(t)\equiv0$
. In fact, the dynamics for the leader reduces to
$x_{\ell}(t+1)=Ax_{\ell}(t)$
by (2.1). It follows from the Jordan normal form, if any eigenvalue
$\lambda$
of *A* has norm less than 1, Assumption 1 is met. The convergence is exponentially fast.

Denote by
$\operatorname{det}(\lambda I_n-A)=\lambda^n+\alpha_1\lambda^{n-1}+\alpha_2\lambda^{n-2}+\cdots+\alpha_n$
the characteristic polynomial of *A*. If the pair (*A*, *B*) is controllable, we define the transformation matrix

Here, the controllability of (*A*, *B*) means the matrix
$\left(B, AB, A^2B,\cdots, A^{n-1}B\right)$
has rank *n* [Reference Chen6]. The system (2.1) can be recast as the following canonical form [Reference Wang, Wang and Kong32] by using the similarity transformation
$x_i(t)=\Gamma y_i(t)$
for
$i\in\mathcal{V}$
:

where $\,\hat{\!A}=\Gamma^{-1}A\Gamma=\left(\begin{array}{c@{\quad}c@{\quad}c@{\quad}c@{\quad}c}-\alpha_1&-\alpha_2&-\alpha_3&\cdots&-\alpha_n\\[1pt]1&0&0&\cdots&0\\[1pt]0&1&0&\cdots&0\\[1pt]\vdots&\vdots&\vdots&\ddots&\vdots\\[1pt]0&0&0&\cdots&0\end{array}\right)\in\mathbb{R}^{n\times n}$ and $\hat{B}=\Gamma^{-1}B=$ $(1,0,0,\cdots,0)^{\operatorname{T}}\in\mathbb{R}^n$ , where $\operatorname{T}$ is the matrix transpose.

Write $y_i(t)=\left(y_{i,n-1}(t),y_{i,n-2}(t),\cdots,y_{i,0}(t)\right)^{\operatorname{T}}$ . Instead of transmitting vector states between agents in $\mathcal{G}(t)$ , we define a scalar value $z_i(t)$ for each agent $i\in\mathcal{V}$ :

which encodes the components of $y_i(t)$ . We will see in the next section the information agents exchange over $\mathcal{G}(t)$ will be expressed by $z_i(t)$ . Here, $\beta_k\in\mathbb{R}$ for $k=1,2,\cdots, n-1$ are chosen such that the polynomial $\lambda^{n-1}+\beta_1\lambda^{n-2}+\beta_2\lambda^{n-3}+\cdots+\beta_{n-1}$ becomes Schur stable, meaning that its roots are in the open unit disk. A quick sufficient condition for Schur stability is, for example, $0<\beta_{n-1}<\beta_{n-2}<\cdots<\beta_1<1$ [Reference Bhattacharyya, Datta and Keel3].

### 2.3 Resilient tracking consensus protocol

In this subsection, we introduce the resilient tracking consensus strategy for cooperative agents, which is a purely distributed protocol.

Fix a parameter
$r\in\mathbb{N}$
. For each
$i\in\mathcal{C}$
, at time step
$t\in\mathbb{N}$
, the agent *i* uses the encoded state information
$z_j(t)$
$\big(\,j\in\mathcal{N}^{\mathrm{in}}_i(t)\big)$
from its in-neighbors and arranges the list
$z_{\big(|\mathcal{N}^{\mathrm{in}}_i(t)|\big)}(t)\leqslant z_{\big(|\mathcal{N}^{\mathrm{in}}_i(t)|-1\big)}(t)\leqslant\cdots\leqslant z_{(1)}(t)$
. If at least *r* values in this list are larger than
$z_i(t)$
, we remove the corresponding vertices *j* of the largest *r* values (except for
$j=\ell$
). If there are less than *r* such values, all of their corresponding vertices are removed (except for
$j=\ell$
). Analogously, if at least *r* values in this sorted list are smaller than
$z_i(t)$
, we remove the corresponding vertices *j* of the smallest *r* values (except for
$j=\ell$
). If there are less than *r* such values, all of their corresponding vertices are removed (except for
$j=\ell$
). All removed vertices are recorded in the set
$\mathcal{R}_i(t)$
. The strategy is shown in Tab. 1 below. The control input in (2.1) for agent *i* is taken as

where $0<\gamma_i(t)<\big(\sum_{j\in\mathcal{N}^{\mathrm{in}}_i(t)}a_{ij}(t)\big)^{-1}$ . This control function $u_i(t)$ can be further generalized to accommodate a nonlinear diffusion; see Section 3.

Remark 3. The removal of extreme values is a key feature of W-MSR-type algorithms. The intuition behind is that the Byzantine agents might manipulate the system states by taking some extremely large or small values to deviate the states of cooperative agents. If these extreme values are removed, the system is more likely to achieve consensus. This rationale is also in line with our daily wisdom in many scoring systems in sports and competitions, where the top and bottom scores are removed to ensure a fair judgment. The meaning of the parameter *r* will be explained below.

Remark 4. In the above algorithm, the leader takes the role of a trustworthy node [Reference Abbas, Laszka and Koutsoukos1], whose value will always be trusted by a cooperative agent *i*. The idea of trustworthy nodes has been employed in the study of resilient leaderless consensus to effectively mitigate the influence of Byzantine agents [Reference Abbas, Laszka and Koutsoukos1, Reference Fu, Qin, Shi, Zheng and Kang11, Reference Huang, Wu, Chang, Tao and He15]. This assumption is also necessary in resilient tracking consensus because if the leader is not trusted, it plays a role essentially the same as a malicious agent and leader-following may not be feasible. A stronger condition that requires a leader being identifiable has been proposed in [Reference Razaee, Parisini and Polycarpou22], where the value of a leader is fed into the protocol differently from a cooperative agent (A convergence rate is estimated as a result of this).

Remark 5. Although the entry
$a_{ij}(t)$
of the adjacency matrix is a random variable for any given time *t*,
$\gamma_i(t)$
can be prescribed as a deterministic number. For example, if we know
$a_{ij}(t)\leqslant M(t)$
holds for some
$M(t)>0$
and all
$i,j\in\mathcal{V}$
,
$\gamma_i(t)$
can be chosen in the range of
$(0,M(t)|\mathcal{N}^{\mathrm{in}}_i(t)|)$
.

The above strategy is pertinent to a parameter *r*, which we will use to bound the maximum number of Byzantine agents in the neighborhood of a cooperative agent. In particular, a vertex set
$\mathcal{S}\subseteq\mathcal{V}$
is called *r*-local if for any
$i\not\in\mathcal{S}$
,
$|\mathcal{N}^{\mathrm{in}}_i(t)\cap\mathcal{S}|\leqslant r$
for any
$t\in\mathbb{N}$
. If
$\mathcal{B}$
is *r*-local, clearly any cooperative agent will have at most *r* Byzantine in-neighbors.

With the above resilient tracking consensus strategy, we will show that the cooperative agents in $\mathcal{C}$ can follow the leader $\ell$ in the sense of almost sure convergence as $t\rightarrow\infty$ . Specifically, we say the resilient tracking consensus is achieved for the multi-agent system (2.1) if $\|x_i(t)-x_{\ell}(t)\|\rightarrow0$ as $t\rightarrow\infty$ almost surely for any $i\in\mathcal{C}$ and initial configuration $\{x_i(0)\}_{i\in\mathcal{V}}$ .

## 3 Convergence analysis

In this section, we investigate the resilient tracking consensus of the linear system (2.1) in the presence of Byzantine agents in a dynamic random graph $\mathcal{G}(t)$ . Our main result is the following.

Theorem 1. Consider the multi-agent system (2.1) over the dynamic random graph $\mathcal{G}(t)=(\mathcal{V},\mathcal{E}(t),\mathcal{A}(t))$ with $\mathcal{V}=\{\ell\}\cup\mathcal{C}\cup\mathcal{B}$ , where $\ell$ is the leader agent, cooperative agents in $\mathcal{C}$ update their states following the strategy (2.5), and the set $\mathcal{B}$ of Byzantine agents is r-local. Suppose $\mathcal{G}(t)$ is leader-follower $(2r+1)$ -robust with positive probability for each $t\in\mathbb{N}$ and Assumption 1 holds. If (A, B) is controllable, then the resilient tracking consensus for (2.1) is achieved almost surely.

Remark 6. Fix $t\in\mathbb{N}$ . Define $\hat{\mathcal{E}}(t)=\{(i,\,j)\in\mathcal{V}\times\mathcal{V}:p_{ij}(t)>0\}$ . Note that the graph $\mathcal{G}(t)$ is a finite graph. Therefore, if the adjacency matrix $\mathcal{A}(t)$ has independent elements, the condition that $\mathcal{G}(t)$ is leader-follower $(2r+1)$ -robust with positive probability is equivalent to saying that the graph $(\mathcal{V},\hat{\mathcal{E}}(t))$ is leader-follower $(2r+1)$ -robust.

Proof. For any agent $i\in\mathcal{C}$ , by (2.3) and (2.5), we have

Noting that $y_i(t)=(y_{i,n-1}(t),y_{i,n-2}(t),\cdots,y_{i,0}(t))^{\operatorname{T}}$ for $t\in\mathbb{N}$ , we obtain

and

It follows from the definition of $z_i(t)$ in (2.4) and (3.2) that for $i\in\mathcal{C}$ ,

Since
$0<\gamma_i(t)<\big(\!\sum_{j\in\mathcal{N}^{\mathrm{in}}_i(t)}a_{ij}(t)\big)^{-1}$
,
$z_i(t+1)$
in (3.4) can be viewed as a convex combination of
$z_i(t)$
and
$z_j(t)$
for
$j\in\mathcal{N}^{\mathrm{in}}_i(t)\backslash\mathcal{R}_i(t)$
. Define
$\overline{z}(t)=\max\{\{z_i(t)\}_{i\in\mathcal{C}},z_{\ell}(t)\}$
and
$\underline{z}(t)=\min\{\{z_i(t)\}_{i\in\mathcal{C}},z_{\ell}(t)\}$
. Fix
$t\in\mathbb{N}$
. For any agent
$i\in\mathcal{C}$
, since
$\mathcal{B}$
is *r*-local and the convex combination is in effect, there are at most *r* in-neighbors (excluding possibly
$\ell$
) taking values outside the range
$[\underline{z}(t),\overline{z}(t)]$
at time
$t+1$
. By our resilient tracking consensus strategy, any such neighbors will be removed in the update for agent *i* at time step
$t+1$
with two possible exceptions: (i) a neighbor *j* has state
$z_j(t+1)$
satisfying
$\overline{z}(t)<z_j(t+1)\leqslant z_{\ell}(t+1)$
or (ii) a neighbor *j* has state
$z_j(t+1)$
satisfying
$\underline{z}(t)>z_j(t+1)\geqslant z_{\ell}(t+1)$
. In both cases,
$\ell\in\mathcal{N}^{\mathrm{in}}_i(t)$
. Employing Assumption 1, we know there exists
$0\leqslant\varepsilon(t)=o\big(t^{-1}\big)$
such that

for $t\in\mathbb{N}$ . This relationship holds regardless of the randomness of edges in $\mathcal{G}(t)$ .

For any cooperative agent $i\in\mathcal{C}$ , write $w_i(t)=(y_{i,n-2}(t), y_{i, n-3}(t),\cdots,y_{i,0}(t))^{\operatorname{T}}\in\mathbb{R}^{n-1}$ . Therefore, $y_i(t)=(y_{i,n-1}(t),w_i(t)^{\operatorname{T}})^{\operatorname{T}}$ . In view of (3.3), we obtain a new linear system

where $\,\tilde{\!A}=\left(\begin{array}{c@{\quad}c@{\quad}c@{\quad}c@{\quad}c}-\beta_1&-\beta_2&-\beta_3&\cdots&-\beta_{n-1}\\[2pt]1&0&0&\cdots&0\\[2pt]0&1&0&\cdots&0\\[2pt]\vdots&\vdots&\vdots&\ddots&\vdots\\[2pt]0&0&0&\cdots&0\end{array}\right)\in\mathbb{R}^{(n-1)\times(n-1)}$ and $\tilde{B}=(1, 0, 0, \cdots,0)^{\operatorname{T}}$ $\in$ $\mathbb{R}^{n-1}$ , where $z_i(t)$ is given by (2.4).

Define
$b(t)=\overline{z}(t)-\underline{z}(t)-2t\varepsilon(t)$
for
$t\in\mathbb{N}$
and let
$\mathcal{F}(t)$
be the
$\sigma$
-algebra generated by
$\{\{z_i(0)\}_{i\in\mathcal{V}},\{z_i(1)\}_{i\in\mathcal{V}}, \cdots,\{z_i(t)\}_{i\in\mathcal{V}}\}$
. By (3.5), we obtain
$\mathbb{E}|b(t)|<\infty$
for
$t\in\mathbb{N}$
and
$\mathbb{E}(b(t)|\mathcal{F}(t-1))\leqslant b(t-1)$
for
$t\geqslant1$
. Thus, *b*(*t*) is a super-martingale relevant to the filtration
$\mathcal{F}(t)$
. There is some
$b\geqslant0$
such that

almost surely by employing the martingale convergence theorem [Reference Hall and Heyde14, p. 2] and our assumption of $\varepsilon(t)$ .

Next, we will show the limit $b=0$ . In fact, if this is not the case we define three random sets $\mathcal{I}_1(t)=\{i\in\mathcal{V}\backslash\mathcal{B}:z_i(t)=\overline{z}(t)\}\not=\emptyset$ , $\mathcal{I}_2(t)=\{i\in\mathcal{V}\backslash\mathcal{B}:z_i(t)=\underline{z}(t)\}\not=\emptyset$ , and $\mathcal{I}_3(t)=(\mathcal{V}\backslash\mathcal{B})\backslash(\mathcal{I}_1(t)\cup\mathcal{I}_2(t))$ . Clearly, these three sets partition $\mathcal{V}\backslash\mathcal{B}$ . We consider three cases depending on the location of leader $\ell$ : (i) $\ell\in\mathcal{I}_1(t)$ , (ii) $\ell\in\mathcal{I}_2(t)$ , and (iii) $\ell\in\mathcal{I}_3(t)$ .

(i) Suppose
$\mathcal{I}_2(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)\not=\emptyset$
. We have, say,
$i\in\mathcal{I}_2(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)$
. By our resilient tracking consensus strategy,
$z_{\ell}(t)$
will be used in the update for agent *i* at time step
$t+1$
via (3.4). Since
$\mathcal{B}$
is *r*-local, it is easy to see our protocol guarantees any value outside
$[\underline{z}(t),\overline{z}(t)]$
will not be used in the update for agent *i* at time step
$t+1$
in this situation. Because of the convex combination in (3.4), agent *i* must take a larger value at the next time step, that is,
$z_i(t+1)>z_i(t)$
.

If
$\mathcal{I}_2(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)=\emptyset$
, by our assumption that
$\mathcal{G}(t)$
is leader-follower
$(2r+1)$
-robust with positive probability, we know
$\mathcal{I}_2(t)$
is
$(2r+1)$
-robust with positive probability. Hence, with positive probability there exists
$i\in\mathcal{I}_2(t)$
such that *i* has at least
$2r+1$
in-neighbors outside the set
$\mathcal{I}_2(t)$
. As
$\mathcal{B}$
is *r*-local, *i* has at least
$r+1$
cooperative in-neighbors outside
$\mathcal{I}_2(t)$
. By our algorithm, at least one of such neighbors will be used in the update for agent *i* at time step
$t+1$
. Using the fact that
$\mathcal{B}$
is *r*-local again, we know that any value outside
$[\underline{z}(t),\overline{z}(t)]$
will not be used by *i* in the update at time
$t+1$
. Therefore, via the convex combination in (3.4), we obtain
$z_i(t+1)\geqslant z_i(t)$
and with positive probability
$z_i(t+1)>z_i(t)$
.

(ii) This case can be shown analogously as in (i) by considering two situations: $\mathcal{I}_1(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)\not=\emptyset$ and $\mathcal{I}_1(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)=\emptyset$ . Consequently, we know that with positive probability there exists some agent $i\in\mathcal{I}_1(t)$ such that $z_i(t+1)\leqslant z_i(t)$ and with positive probability $z_i(t+1)<z_i(t)$ .

(iii) We first consider two situations $\mathcal{I}_2(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)\not=\emptyset$ and $\mathcal{I}_2(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)=\emptyset$ and proceed similarly as in case (i) to conclude there is some agent $i\in\mathcal{I}_2(t)$ satisfying $z_i(t+1)\geqslant z_i(t)$ and with positive probability $z_i(t+1)>z_i(t)$ . Then, we consider two situations $\mathcal{I}_1(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)\not=\emptyset$ and $\mathcal{I}_1(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)=\emptyset$ and proceed similarly as in case (ii) to derive there exists agent $j\in\mathcal{I}_1(t)$ satisfying $z_j(t+1)\leqslant z_j(t)$ and with positive probability $z_j(t+1)<z_j(t)$ .

Combining (i), (ii), (iii) and noting that $\mathcal{G}(t)$ is a finite graph, we conclude that $\overline{z}(t)-\underline{z}(t)$ tends to zero in probability as $t\rightarrow\infty$ . But this is at odds with our assumption that $b>0$ in (3.7). Therefore, we have shown that almost surely

Using Assumption 1, (3.5) and (3.8), we obtain that there is some
$c\in\mathbb{R}$
satisfying
$\lim_{t\rightarrow\infty}z_i(t)=c$
almost surely for all
$i\in\mathcal{C}\cup\{\ell\}$
. Moreover, by (2.4) we know that *c* is determined via the following

where $\Gamma^{-1}x_f=\left(\left(\Gamma^{-1}x_f\right)_{n-1},\left(\Gamma^{-1}x_f\right)_{n-2},\cdots,\left(\Gamma^{-1}x_f\right)_0\right)$ .

With the convergence of $z_i(t)$ at hand, we now move onto the convergence of $w_i(t)$ for $i\in\mathcal{C}$ . In doing so, define an error vector $\delta_i(t)=w_i(t)-c/\left(1+\sum_{k=1}^{n-1}\beta_k\right)1_{n-1}$ , where $1_{n-1}\in\mathbb{R}^{n-1}$ is an all one vector. Using (3.6), we arrive at

Denote by $\rho_i(t)=z_i(t)-c$ for $i\in\mathcal{C}$ and hence $\lim_{t\rightarrow\infty}\rho_i(t)=0$ almost surely. From (3.10), we have

It is easy to see that the matrix $\,\tilde{\!A}$ is Schur stable because the characteristic polynomial $\operatorname{det}(\lambda I_{n-1}-\,\tilde{\!A})$ is a Schur stable polynomial as defined at the end of Section 2.2. Thanks to the Lyapunov stability theory [Reference Chen6, Theorem 5.D5], we obtain for any positive definite matrix $Q\in\mathbb{R}^{(n-1)\times(n-1)}$ there is a positive definite matrix $P\in\mathbb{R}^{(n-1)\times(n-1)}$ such that $P-Q=\tilde{A}^{\operatorname{T}}P\,\tilde{\!A}$ . For any cooperative agent $i\in\mathcal{C}$ , define the Lyapunov candidate function $\Theta_i(t)=\delta_i(t)^{\operatorname{T}}P\delta_i(t)$ and along the solution of (3.11) it follows

Recall that
$\,\tilde{\!A}$
is stable. Applying the input-to-state stability result [Reference Jiang and Wang16, Example 3.4, Lemma 3.5] to the linear system (3.6), we derive that
$w_i(t)$
is bounded for all
$t\in\mathbb{N}$
. In light of the definition of
$\delta_i(t)$
, we know
$\delta_i(t)$
is also bounded. Since
$\lim_{t\rightarrow\infty}\rho_i(t)=0$
almost surely, we obtain
$\lim_{t\rightarrow\infty}|2\delta_i(t)^{\operatorname{T}}\tilde{A}^{\operatorname{T}}P\tilde{B}\rho_i(t)+\rho_i(t)^2\tilde{B}^{\operatorname{T}}P\tilde{B}|=0$
almost surely. By (3.12), it is not difficult to see
$\lim_{t\rightarrow\infty}\Theta_i(t)=0$
almost surely for any cooperative agent
$i\in\mathcal{C}$
. Indeed, if this is not true,
$\delta_i(t)\not\rightarrow0$
as
$n\rightarrow\infty$
by the definition of
$\Theta_i(t)$
and the positive definiteness of *P*. Using (3.12) and the positive definiteness of *Q*, for any
$t\in\mathbb{N}$
there are
$\varepsilon>0$
and
$\hat{t}\geqslant t$
such that
$\Theta_i(\hat{t}+1)-\Theta_i(\hat{t})\leqslant-\varepsilon$
. This is in conflict with the fact that
$\Theta_i(t)$
for all
$t\in\mathbb{N}$
is lower bounded by zero.

Using the definition of $\Theta_i(t)$ again, we have $\lim_{t\rightarrow\infty}\delta_i(t)=0$ almost surely for $i\in\mathcal{C}$ . Thereby, $\lim_{t\rightarrow\infty}w_i(t)=c/\left(1+\sum_{k=1}^{n-1}\beta_k\right)1_{n-1}$ almost surely. Applying (3.3), we know $\lim_{t\rightarrow\infty}y_i(t)=c/\left(1+\sum_{k=1}^{n-1}\beta_k\right)1_n$ almost surely for all cooperative agents $i\in\mathcal{C}$ . As $x_i(t)=\Gamma y_i(t)$ , it follows that

almost surely by using (3.9). This means all cooperative agents will track the evolution of the leader agent $\ell$ by Assumption 1. The proof is complete. $\Box$

Remark 7. As with W-MSR like protocols, our resilient tracking consensus strategy does not guarantee ditching all Byzantine agents nor does it guarantee keeping all cooperative agents at any time step. As a result, the transient trajectories are influenced by the Byzantine agents in general. However, distinct from leaderless consensus counterparts (e.g. [Reference Dibaji and Ishii7, Reference LeBlanc, Zhang, Koutsoukos and Sundaram18, Reference Razaee, Parisini and Polycarpou21, Reference Shang25, Reference Shang26]), the final consensus state goes with that of the leader (and hence not influenced by any Byzantine agents).

Remark 8. The above linear system analysis method can be applied to a more general control function $u_i(t)$ given as follows

where $\gamma_i(t)$ is chosen as in (2.5) and the nonlinear function $g_{ij}:\mathbb{R}\times\mathbb{R}\rightarrow\mathbb{R}$ satisfies (i) $g_{ij}(z_1,z_2)=0$ if and only if $z_1=z_2$ and (ii) there exist two constants $0<q_i\leqslant q^{\prime}_i\leqslant1$ such that

for any
$z_1\not=z_2$
. The control function
$u_i(t)$
in (2.5) is a special case of (3.14) by taking
$g_{ij}(z_1,z_2)=z_1-z_2$
. It is worth mentioning that the two bounds in (3.15) do not depend on any node *j* as we can always take the maximum and minimum over all nodes of
$\mathcal{G}(t)$
(which is a finite graph) without loss of generality.

With this coupling function, Theorem 1 can be proved along the same line. In fact, when (2.5) is replaced by (3.14), the update rule (3.4) for an agent $i\in\mathcal{C}$ becomes

Although $z_i(t+1)$ is no longer a convex combination of $z_i(t)$ and $\{z_j(t)\}_{j\in\mathcal{N}^{\mathrm{in}}_i(t)\backslash\mathcal{R}_i(t)}$ , it is bounded by two convex combinations of them from the above and the below.

## 4 Numerical simulations

In this section, we present two examples to illustrate our theory.

Example 1. Consider a directed graph
$\mathcal{G}$
over
$N=8$
vertices with
$\ell=1$
,
$\mathcal{B}=\{2\}$
and
$\mathcal{C}=\{3,4,\cdots,8\}$
; see Figure 1. The time-dependent random graph
$\mathcal{G}(t)$
is built upon
$\mathcal{G}$
by assuming binary edge weights, namely,
$a_{ij}(t)$
takes 1 or 0. The edge probabilities
$p_{ij}(t)\equiv0.5$
and the corresponding weights
$a_{ij}(t)$
are independent for all *i*, *j*, and *t*. It is straightforward to check that
$\mathcal{G}$
is leader-follower 3-robust. Hence,
$\mathcal{G}(t)$
is leader-follower 3-robust with positive probability for any
$t\in\mathbb{N}$
; c.f. Remark 6.

The agents’ dynamics for $i\in\mathcal{V}$ and $t\in\mathbb{N}$ and are taken as

where
$u_1(t)=f(t)\equiv0$
,
$u_2(t)=0.2\cos(t/5)$
, and
$u_i(t)$
for
$i\in\mathcal{C}$
is given by (2.5) with
$\alpha_1=-1$
,
$\alpha_2=0.25$
,
$\beta_1=0.5$
,
$\gamma_i(t)\equiv0.2$
. It is easy to check that the pair (*A*, *B*) is stable,
$x_f=(0,0)^{\operatorname{T}}$
, and all conditions in Theorem 1 are satisfied. Let
$x_i(t)=(x_{i,1}(t),x_{i,2}(t))^{\operatorname{T}}$
for
$i\in\mathcal{C}$
.

By choosing initial states randomly within $[-4,4]$ , we display the dynamical evolution of the multi-agent system in Figure 2. We observe that both components of the states of cooperative agents track the leader agent 1 and converge to the equilibrium $x_f$ as expected.

In this example, the Byzantine agent $2\in\mathcal{N}^{\mathrm{out}}_1(t)$ with probability 0.5 and it influences half of the cooperative agents in $\mathcal{C}$ . Moreover, the Byzantine agent possesses a dynamics repeatedly oscillating around the equilibrium. All this represents a highly adverse scenario to the collective consensus task. Remarkably, our simple distributed protocol enables the cooperative agents to converge towards the leader’s equilibrium relatively quickly.

Example 2. As is known, determining robustness is an NP-hard problem [Reference Usevitch and Panagou31], which prevents us from considering large-scale empirical networks. In this example, we consider an animal interaction network of primates in Yakushima, Japan [Reference Griffin and Nunn13], which contains
$N=12$
macaca fuscata with a leader; see Figure 3. Specifically, the directed graph
$\mathcal{G}$
has a leader
$\ell=1$
, a Byzantine agent in
$\mathcal{B}=\{2\}$
and cooperative agents in
$\mathcal{C}=\{3,4,\cdots,12\}$
. We construct a time-dependent random graph
$\mathcal{G}(t)$
by assuming binary edge weights, namely
$a_{ij}(t)$
takes 1 or 0 and edge probabilities
$p_{ij}(t)\equiv p$
. The weights
$a_{ij}(t)$
are independent for all *i*, *j*, and *t*. It is straightforward to verify that
$\mathcal{G}$
is leader-follower 3-robust. Hence,
$\mathcal{G}(t)$
is leader-follower 3-robust with positive probability for all
$t\in\mathbb{N}$
. Note that the Byzantine agent 2 is highly connected in the network.

Let $n=1$ . The agents’ dynamics for $i\in\mathcal{V}$ and $t\in\mathbb{N}$ and are chosen as

where $u_1(t)=f(t)\equiv0$ , $u_2(t)=0.1\cos(t/20)$ , and $u_i(t)$ for $i\in\mathcal{C}$ is given by (2.5) with $\alpha_1=-0.5$ , $\gamma_i(t)\equiv 1/12$ . It is clear that $x_f=0,$ and all conditions in Theorem 1 are satisfied.

The initial configuration is chosen as $x_1(0)=-0.8$ , $x_2(0)=0.3$ , $x_3(0)=-0.1$ , $x_4(0)=-0.5$ , $x_5(0)=1$ , $x_6(0)=-0.7$ , $x_7(0)=0.2$ , $x_8(0)=0.6$ , $x_9(0)=0.7$ , $x_{10}(0)=-1$ , $x_{11}(0)=0.1$ , $x_{12}(0)=-0.3$ . In Figure 4(a) and (b), we show the state evolution for different edge probability $p=0.1$ and $p=0.8$ , respectively. In both cases, the followers are able to track the leader and a denser network tends to give rise to a faster convergence.

## 5 Conclusion

In this paper, we adopt a linear system approach to solving the resilient tracking consensus problems over random graphs. The agents are divided into three categories, namely one leader, some Byzantine agents and cooperative agents. The consensus is reached when the cooperative agents collapse on the target position of the leader, despite the possibly malevolent action of the Byzantine agents. We provide sufficient conditions that guarantee almost sure tracking consensus of cooperative agents towards the leader over time-varying directed graphs with random edges. The designed control protocol is purely distributed and enables consensus against a bounded number of Byzantine agents. Our future work will include generalizing the linear system method to deal with other coordinated complex system problems.

## Acknowledgement

The author is grateful to the anonymous reviewers for their careful reading and constructive suggestions.

## Conflicts of interest

The author declares that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.