Introduction
Let X and Y be two finite-dimensional vector spaces. The space
$\operatorname {\mathrm {Hom}}(X,Y)$
is stratified by the submanifolds

of codimension

This generalizes to infinite dimensions as follows. Let X and Y be two Banach spaces. The space of Fredholm operators from X to Y, denoted by
$\mathscr {F}(X,Y)$
, is stratified by the submanifolds

of codimension

In many geometric problems, especially in the study of moduli spaces in algebraic geometry, gauge theory and symplectic topology, one is led to consider families of Fredholm operators
$D\mskip 0.5mu\colon \thinspace \mathscr {P} \to \mathscr {F}(X,Y)$
parametrized by a Banach manifold
$\mathscr {P}$
, and to analyze the subsets
$D^{-1}(\mathscr {F}_{d,e})$
.
The archetypal example is Brill–Noether theory in algebraic geometry. Let
$\Sigma $
be a closed, connected Riemann surface of genus g. Denote by
$\operatorname {\mathrm {Pic}}(\Sigma )$
the Picard group of isomorphism classes of holomorphic line bundles
$\mathscr {L} \to \Sigma $
. Brill–Noether theory is concerned with the study of the subsets
$G_d^r \subset \operatorname {\mathrm {Pic}}(\Sigma )$
, called the Brill–Noether loci, defined by

The fundamental results of this theory deal with the questions of whether
$G_d^r$
is nonempty, smooth and of the expected complex codimension.
This connects to the previous discussion as follows. Let L be a Hermitian line bundle of degree d over
$\Sigma $
. Denote by
$\mathscr {A}(L)$
the space of unitary connections on L. The complex gauge group
$\mathscr {G}^{\mathbf {C}}(L)$
acts on
$\mathscr {A}(L)$
and the quotient
$\mathscr {A}(L)/\mathscr {G}^{\mathbf {C}}(L)$
is biholomorphic to
$\operatorname {\mathrm {Pic}}^d(\Sigma )$
, the component of
$\operatorname {\mathrm {Pic}}(\Sigma )$
parametrizing holomorphic line bundles of degree d. Define the family of Fredholm operators

by assigning to every connection A the Dolbeault operator
$\bar {\partial }_A = \nabla _A^{0,1}$
. Set

It follows from the Riemann–Roch theorem and Hodge theory that the Brill–Noether loci can be described as the quotients

If
$G_d^r$
is nonempty, then

This is an immediate consequence of the definition of
$\tilde G_d^r$
and
$\operatorname {\mathrm {codim}}_{\mathbf {C}} \mathscr {F}_{d,e} = de$
. Ideally, every
$G_d^r$
is smooth of complex codimension
$(r+1)(g-d+r)$
. This is not always true, but Gieseker [Reference GiesekerGie82] proved that it holds for generic
$\Sigma $
; see also [Reference Eisenbud and HarrisEH83; Reference LazarsfeldLaz86]. Furthermore, Kempf [Reference KempfKem71] and Kleiman and Laksov [Reference Kleiman and LaksovKL72; Reference Kleiman and LaksovKL74] proved that if
$(r+1)(g-d+r) \leqslant g$
, then
$G_d^r$
is nonempty. For an extensive discussion of Brill–Noether theory in algebraic geometry we refer the reader to [Reference Arbarello, Cornalba, Griffiths and HarrisACGH85].
By analogy, for a family of Fredholm operators
$D\mskip 0.5mu\colon \thinspace \mathscr {P} \to \mathscr {F}(X,Y)$
one might ask:
-
(1) When are the subsets
$D^{-1}(\mathscr {F}_{d,e})$ nonempty?
-
(2) When are they smooth submanifolds of
$\mathscr {P}$ ?
-
(3) What are their codimensions?
Index theory and the theory of spectral flow sometimes give partial results regarding (1). A simple answer to (2) and (3) is that
$D^{-1}(\mathscr {F}_{d,e})$
is smooth and of codimension
$de$
if the map D is transverse to
$\mathscr {F}_{d,e}$
. However, for many naturally occurring families of elliptic operators this condition does not hold. For example, if D is a family of elliptic operators over a manifold M and
${\underline V}$
is a local system, then the family
$D^{\underline V}$
of the elliptic operators D twisted by
${\underline V}$
often is not transverse to
$\mathscr {F}_{d,e}$
even if D is. Related issues arise for families of elliptic operators pulled back by a covering map
$\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$
. The purpose of this article is to give useful tools for answering (2) and (3) which apply to these equivariant situations. This theory is developed in Part 2.
The issues discussed above are well known to arise from multiple covers in the theory of J-holomorphic maps in symplectic topology. In fact, our motivation for writing this article came from trying to understand Wendl’s proof of Bryan and Pandharipande’s superrigidity conjecture for J-holomorphic maps [Reference WendlWen19b]. The theory developed in Part 2 is essentially an abstraction of Wendl’s ideas, some of which can themselves be traced back to Taubes [Reference TaubesTau96] and Eftekhary [Reference EftekharyEft16]. In Part 2, we use this theory to give a concise exposition of the proof of the superrigidity conjecture. The main results of Part 2 are contained in [Reference WendlWen19b], and most of the proofs closely follow Wendl’s approach. There are, however, two key differences:
-
(1) Our discussion consistently uses the language of local systems. This appears to us to be more natural for the problem at hand. It also avoids the use of representation theory and covering theory. In particular, there is no need to take special care of nonnormal covering maps.
-
(2) Our approach to dealing with branched covering maps is geometric: Branched covering maps between Riemann surfaces are reinterpreted as unbranched covering maps between orbifold Riemann surfaces. This is to be compared with Wendl’s analytic approach which uses suitable weighted Sobolev spaces on punctured Riemann surfaces. One feature of our approach is that it leads to a simple proof of the crucial index theorem; cf. Section 2.B and [Reference WendlWen19b, Theorem 4.1].
We expect the theory developed in Part 1 to have many applications outside of the theory of J-holomorphic maps. In future work, we plan to study transversality for multiple covers of calibrated submanifolds in manifolds with special holonomy, such as associative submanifolds in
$G_2$
-manifolds.
1 Equivariant Brill–Noether theory
Throughout this part, let
$(M,g)$
be a closed, connected, oriented Riemannian orbifold of dimension
$\dim M = n$
, and let E and F be Euclidean vector bundles of rank
$\operatorname {\mathrm {rk}} E = \operatorname {\mathrm {rk}} F = r$
over M equipped with orthogonal covariant derivatives.Footnote
1
Here and throughout this article,
$\dim $
and
$\operatorname {\mathrm {rk}}$
denote the dimension and rank over the real numbers. If dimension and rank are to be taken over a different field, then this is indicated by a subscript.
$\Gamma (E)$
denotes the space of smooth sections of E. For an open subset
$U \subset M$
,
$\Gamma (U,E)$
and
$\Gamma _c(U,E) \subset \Gamma (E)$
denote the spaces of smooth sections of E defined over U and with support in U respectively. For
$k \in {\mathbf {N}}_0 = {\lbrace }{0,1,2,\ldots }{\rbrace }$
, denote by
$W^{k,2}\Gamma (E)$
the Sobolev completion of
$\Gamma (E)$
with respect to the
$W^{k,2}$
norm induced by the Euclidean metric and covariant derivative. For
$k \in {\mathbf {N}}$
set
. Set
. Denote by
$L^1\Gamma (E)$
the completion of
$\Gamma (E)$
with respect to the
$L^1$
norm. (Analogous notation is used, instead of E, for F, etc.)
1.1 Brill–Noether loci
Let us begin by discussing the nonequivariant theory.
Definition 1.1.1. Let
$k \in {\mathbf {N}}_0$
. A family of linear elliptic differential operators of order k consists of a Banach manifold
$\mathscr {P}$
and a smooth map

such that for every
$p \in \mathscr {P}$
the operator
is a linear elliptic differential operator.Footnote
2
,Footnote
3
Definition 1.1.2. Let
$(D_p)_{p \in \mathscr {P}}$
be a family of linear elliptic differential operators. For
$d,e \in {\mathbf {N}}_0$
define the Brill–Noether locus
$\mathscr {P}_{d,e}$
by

Remark 1.1.3. Let
$(D_p)_{p \in \mathscr {P}}$
be a family of linear elliptic operators of index
$i \in \mathbf {Z}$
. If
$\mathscr {P}_{d,e} \neq \varnothing $
, then
$d-e = i$
; in particular:
$d \geqslant i$
and
$e \geqslant -i$
.
The following elementary fact from the theory of Fredholm operators reduces the discussion to the finite-dimensional case. As in the introduction, if X and Y are Banach spaces, then
$\mathscr {L}(X,Y)$
denotes the Banach space of bounded linear maps from X to Y equipped with the operator norm, and
$\mathscr {F}(X,Y) \subset \mathscr {L}(X,Y)$
denotes the open subset of Fredholm operators from X to Y.
Lemma 1.1.4 (cf. Koschorke [Reference KoschorkeKos68, Chapter I §1.b]).
Let X and Y be Banach spaces. For every
$L \in \mathscr {F}(X,Y)$
, there is an open neighborhood
$\mathscr {U} \subset \mathscr {F}(X,Y)$
and a smooth map
$\mathscr {S} \mskip 0.5mu\colon \thinspace \mathscr {U} \to \operatorname {\mathrm {Hom}}(\ker L,\operatorname {\mathrm {coker}} L)$
such that for every
$T \in \mathscr {U}$
there are isomorphisms

furthermore, the derivative of S at L,

satisfies

for every
$\hat L \in T_L\mathscr {F}(X,Y) = \mathscr {L}(X,Y)$
.
Proof. Pick a complement
$\operatorname {\mathrm {coim}} L$
of
$\ker L$
in X, and a lift of
$\operatorname {\mathrm {coker}} L$
to Y. With respect to the splittings
$X = \operatorname {\mathrm {coim}} L \oplus \ker L$
and
$Y = \operatorname {\mathrm {im}} L \oplus \operatorname {\mathrm {coker}} L$
every
$T \in \mathscr {F}(X,Y)$
can be written as

By construction,
$L_{11}$
is invertible, and the remaining components of L vanish.
Choose an open neighborhood
$\mathscr {U}$
of L in
$\mathscr {F}(X,Y)$
such that for every
$T \in \mathscr {U}$
the operator
$T_{11}$
is invertible. Define
$\mathscr {S}\mskip 0.5mu\colon \thinspace \mathscr {U} \to \operatorname {\mathrm {Hom}}(\ker L,\operatorname {\mathrm {coker}} L)$
by

A brief computation shows that for every
$T \in \mathscr {U}$

hence,
$\ker T \cong \ker \mathscr {S}(T)$
and
$\operatorname {\mathrm {coker}} T \cong \operatorname {\mathrm {coker}} \mathscr {S}(T)$
.
The formula for
$\mathrm {d}_L S$
is evident from the fact that
$L_{21}$
and
$L_{12}$
vanish.
Lemma 1.1.4 together with the regular value theorem immediately imply the following.
Theorem 1.1.5. Let
$(D_p)_{p \in \mathscr {P}}$
be a family of linear elliptic differential operators. For
$p \in \mathscr {P}$
define
$\Lambda _p \mskip 0.5mu\colon \thinspace T_p\mathscr {P} \to \operatorname {\mathrm {Hom}}(\ker D_p,\operatorname {\mathrm {coker}} D_p)$
by

Let
$d,e \in {\mathbf {N}}_0$
and
$p \in \mathscr {P}_{d,e}$
. If
$\Lambda _p$
is surjective, then following hold:
-
(1) There is an open neighborhood
$\mathscr {U}$ of
$p \in \mathscr {P}$ such that
$\mathscr {P}_{d,e} \cap \mathscr {U}$ is a submanifold of codimension
$$ \begin{align*} \operatorname{\mathrm{codim}} (\mathscr{P}_{d,e} \cap \mathscr{U}) = de. \end{align*} $$
-
(2)
$\mathscr {P}_{\tilde d,\tilde e} \neq \varnothing $ for every
$\tilde d,\tilde e \in {\mathbf {N}}_0$ with
$\tilde d \leqslant d, \tilde e \leqslant e$ and
$\tilde d-\tilde e = d-e$ .
Remark 1.1.6. If E and F are Hermitian vector bundles and
$(D_p)_{p \in \mathscr {P}}$
is a family of complex linear elliptic differential operators, then the map
$\Lambda _p$
factors through
$\operatorname {\mathrm {Hom}}_{\mathbf {C}}(\ker D_p,\operatorname {\mathrm {coker}} D_p)$
. Therefore, the hypothesis of Theorem 1.1.5 cannot be satisfied (unless it holds trivially). Of course, this issue is rectified by replacing
$\mathbf {R}$
with
${\mathbf {C}}$
throughout the above discussion.
Example 1.1.7 (Brill–Noether theory for holomorphic line bundles over a Riemann surface).
Let
$\Sigma $
be a closed, connected Riemann surface of genus g. Let L be a Hermitian line bundle of degree d over
$\Sigma $
. Denote by
$\mathscr {A}(L)$
the space of unitary connections on L.Footnote
4
Define the family of complex linear elliptic differential operators

by assigning to every connection A the Dolbeault operator
.
Let
$A \in \mathscr {A}(L)$
. The map
$\Lambda _A\mskip 0.5mu\colon \thinspace T_A\mathscr {A}(L) \to \operatorname {\mathrm {Hom}}_{\mathbf {C}}(\ker \bar {\partial }_A, \operatorname {\mathrm {coker}} \bar {\partial }_A)$
can be described concretely as follows. Since the derivative of the map
$\bar {\partial }$
is
$\mathrm {d}_A\bar {\partial }(a) = a^{0,1}$
, the map
$\Lambda _A$
factors through the isomorphism
$T_A\mathscr {A}(L) = \Omega ^1(\Sigma ,i\mathbf {R}) \cong \Omega ^{0,1}(\Sigma )$
defined by
$a \mapsto a^{0,1}$
. Denote by
$\mathscr {L}$
the holomorphic line bundle associated with
$\bar {\partial }_A$
. By Serre duality,

hence,

The Petri map

is induced by the isomorphism
$\mathscr {L} \otimes _{\mathbf {C}} \mathscr {L}^* \cong \mathscr {O}_{\Sigma }$
. The adjoint of
$\Lambda _A$
is the composition of the Petri map
$\varpi _{\mathscr {L}}$
with the inclusion
$H^0(\Sigma ,K_{\Sigma }) \hookrightarrow \Omega ^{1,0}(\Sigma )$
. Here, the duality between
$\Omega ^{1,0}(\Sigma )$
and
$\Omega ^{0,1}(\Sigma )$
is given by integration.
As a consequence,
$\Lambda _A$
is surjective if and only if
$\varpi _{\mathscr {L}}$
is injective. If
$\varpi _{\mathscr {L}}$
is injective for every
$[\mathscr {L}] \in \operatorname {\mathrm {Pic}}^d(\Sigma )$
, then

is a complex submanifold of codimension
$(r+1)(g-d+r)$
; therefore, so is the Brill–Noether locus

cf. [Reference Arbarello, Cornalba, Griffiths and HarrisACGH85, Lemma 1.6, Chapter IV].
This example motivates the following definitions, which are particularly appropriate for first order operators appearing in geometric applications.
Definition 1.1.9. Let
$U \subset M$
be an open subset. A family of linear elliptic differential operators
$(D_p)_{p \in \mathscr {P}}$
is flexible in U at
$p_{\star } \in \mathscr {P}$
if for every
$A \in \Gamma _c(U,\operatorname {\mathrm {Hom}}(E,F))$
there is a
$\hat p \in T_{p_{\star }}\mathscr {P}$
such that

for every
$s \in \ker D_{p_{\star }}$
.
Definition 1.1.10. Let
$D\mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$
be a linear differential operator. Set

The formal adjoint of D is the linear differential operator
$D^{\dagger } \mskip 0.5mu\colon \thinspace L^2\Gamma (F^{\dagger }) \to W^{-k,2}\Gamma (E^{\dagger })$
characterized by

for
$s \in \Gamma (E)$
and
$t \in \Gamma (F^{\dagger })$
. Here,
${\langle }{\cdot ,\cdot }{\rangle }$
denotes the canonical pairings
$E\otimes E^{\dagger } \to \Lambda ^nT^* M$
and
$F\otimes F^{\dagger } \to \Lambda ^nT^* M$
.
Definition 1.1.11. The Petri map
$\varpi \mskip 0.5mu\colon \thinspace \Gamma (E)\otimes \Gamma (F^{\dagger }) \to \Gamma (E\otimes F^{\dagger })$
is defined by

Let
$U \subset M$
be an open subset. A linear elliptic differential operator
$D\mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$
satisfies Petri’s condition in U if the map

induced by the Petri map is injective.
Proposition 1.1.12. Let
$(D_p)_{p \in \mathscr {P}}$
be a family of linear elliptic differential operators. Let
$U \subset M$
be an open subset. If
$(D_p)_{p \in \mathscr {P}}$
is flexible in U at
$p_{\star } \in \mathscr {P}$
and
$D_{p_{\star }}$
satisfies Petri’s condition in U, then the map
$\Lambda _{p_{\star }}$
defined in Theorem 1.1.5 is surjective.
Proof. Define the map
$\mathrm {ev}_p \mskip 0.5mu\colon \thinspace \Gamma _c(U,\operatorname {\mathrm {Hom}}(E,F)) \to \operatorname {\mathrm {Hom}}(\ker D_p,\operatorname {\mathrm {coker}} D_p)$
by

$(D_p)_{p \in \mathscr {P}}$
is flexible in U at
$p_{\star } \in \mathscr {P}$
if and only
$\operatorname {\mathrm {im}} \mathrm {ev}_{p_{\star }} \subset \operatorname {\mathrm {im}} \Lambda _{p_{\star }}$
.
$D_{p_{\star }}$
satisfies Petri’s condition in U if and only if
$\mathrm {ev}_{p_{\star }}$
is surjective. To see this, observe the following. Since
$\ker D_{p_{\star }}^{\dagger } \cong (\operatorname {\mathrm {coker}} D_{p_{\star }})^*$
, the pairing
$\operatorname {\mathrm {Hom}}(\ker D_{p_{\star }},\operatorname {\mathrm {coker}} D_{p_{\star }}) \otimes (\ker D_{p_{\star }} \otimes \ker D_{p_{\star }}^{\dagger }) \to \mathbf {R}$
induced by

is perfect, that is, it induces an isomorphism
$\operatorname {\mathrm {Hom}}(\ker D_{p_{\star }},\operatorname {\mathrm {coker}} D_{p_{\star }})^* \cong \ker D_{p_{\star }} \otimes \ker D_{p_{\star }}^{\dagger }$
. There is a canonical perfect pairing
${\langle }{\cdot ,\cdot }{\rangle }\mskip 0.5mu\colon \thinspace \operatorname {\mathrm {Hom}}(E,F) \otimes (E\otimes F^{\dagger }) \to \Lambda ^n T^*M$
. Evidently,

Therefore, an element
$B \in \ker D_{p_{\star }} \otimes \ker D_{p_{\star }}^{\dagger }$
annihilates
$\operatorname {\mathrm {im}} \mathrm {ev}_{p_{\star }}$
if and only if

for every
$A \in \Gamma _c(U,\operatorname {\mathrm {Hom}}(E,F))$
; that is,
$\varpi (B) = 0$
in U. Therefore,
$(\operatorname {\mathrm {im}} \mathrm {ev}_{p_{\star }})^{\perp } = \ker \varpi _{D_{p_{\star }},U}$
.
Remark 1.1.13. In Example 1.1.7,
$\operatorname {\mathrm {im}} \Lambda _p = \operatorname {\mathrm {im}} \mathrm {ev}_p$
(with
$U=\Sigma $
, and
$\mathbf {R}$
replaced with
${\mathbf {C}}$
). Therefore,
$\Lambda _p$
being surjective is equivalent to Petri’s condition. Furthermore, tracing through the isomorphisms identifies the restriction of Petri map
$\varpi $
to
$\ker \bar {\partial }_A\otimes _{\mathbf {C}} \ker \bar {\partial }_A^*$
with the Petri map
$\varpi _{\mathscr {L}}$
.
Flexibility is not a rare condition. Petri’s condition appears to be more subtle. (By the uniqueness theorem for ordinary differential equations, it holds for first-order linear elliptic differential operators on
$1$
-manifolds. This is somewhat useful; see, for example, [Reference EftekharyEft19].) The upcoming Remark 1.1.14(3) hints at the connection between Petri’s condition and the unique continuation property. In Section 1.6, we revisit Petri’s condition and discuss an algebraic criterion due to Wendl for Petri’s condition to hold away from a subset of infinite codimension.
Remark 1.1.14. Assume the situation of Theorem 1.1.5. The following observations are useful in situations where the primary objective is to estimate the codimension of
$\mathscr {P}_{d,e}$
.
-
(1) Every
$p \in \mathscr {P}_{d,e}$ has an open neighborhood
$\mathscr {U}$ in
$\mathscr {P}$ such that
$\mathscr {P}_{d,e}\cap \mathscr {U}$ is contained in a submanifold of codimension
$\operatorname {\mathrm {rk}} \Lambda _p$ .
-
(2) Let
$\rho \in {\mathbf {N}}$ , and let
$U \subset M$ be an open subset. A linear elliptic differential operator
$D\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ satisfies Petri’s condition up to rank
$\rho $ in U if for every nonzero
$B \in \ker D\otimes \ker D^{\dagger }$ of rank at most
$\rho $ the section
$\varpi (B)$ does not vanish on U. (A simple tensor is nonzero tensor of the form
$v\otimes w$ . Every tensor B is a sum of simple tensors. The rank of B is the minimal number of simple tensors that sum to B.) If
$D_{p_{\star }}$ satisfies this condition and
$(D_p)_{p\in \mathscr {P}}$ is flexible in U at
$p_{\star } \in \mathscr {P}$ , then
$$ \begin{align*} \operatorname{\mathrm{rk}} \Lambda_{p_{\star}} \geqslant \min{\lbrace}{\rho,d,e}{\rbrace}\max{\lbrace}{d,e}{\rbrace}.\\[-30pt] \end{align*} $$
Proof. Set
. If
$d \leqslant e$ , then choose an injection
$\mathbf {R}^{\sigma } \hookrightarrow \ker D_{p_{\star }}$ and set
; otherwise, choose a surjection
$\operatorname {\mathrm {coker}} D_{p_{\star }} \twoheadrightarrow \mathbf {R}^{\sigma }$ and set
. In either case, composition defines a surjection
$$ \begin{align*} \pi_{p_{\star}}\mskip0.5mu\colon\thinspace \operatorname{\mathrm{Hom}}(\ker D_{p_{\star}},\operatorname{\mathrm{coker}} D_{p_{\star}}) \to H. \end{align*} $$
The subspace
$\operatorname {\mathrm {im}} \pi _{p_{\star }}^* \subset \operatorname {\mathrm {Hom}}(\ker D_{p_{\star }},\operatorname {\mathrm {coker}} D_{p_{\star }})^* \cong \ker D_{p_{\star }} \otimes \ker D_{p_{\star }}^{\dagger }$ consists of elements of rank at most
$\sigma \leqslant \rho $ . The argument of the proof of Proposition 1.1.12 thus shows that
$\pi _{p_{\star }} \circ \Lambda _{p_{\star }}$ is surjective. Therefore,
$\operatorname {\mathrm {rk}} \Lambda _{p_{\star }} \geqslant \dim H = \min {\lbrace }{\rho ,d,e}{\rbrace }\max {\lbrace }{d,e}{\rbrace }$ .
-
(3) (This is due to Eftekhary [Reference EftekharyEft16, Proof of Lemma 4.4].) Let
$U \subset M$ be a nonempty open subset. Let
$D \mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ be a first linear elliptic differential operator of first order. If
$\ker D$ and
$\ker D^{\dagger }$ consists of continuous sections, and D and
$D^{\dagger }$ have the weak unique continuation property, then D satisfies Petri’s condition up to rank three in U.Footnote 5 ,Footnote 6
Proof. Every
$B \in \ker D\otimes \ker D^{\dagger }$ can be written as
$B = s_1\otimes t_1 + \cdots + s_{\rho }\otimes t_{\rho }$ with
, and
$s_1,\ldots ,s_{\rho } \in \ker D$ and
$t_1,\ldots ,t_{\rho } \in \ker D^{\dagger }$ linearly independent. If
$\rho = 1$ and
$\varpi (B) = 0$ , then
$s_1$ or
$t_1$ vanishes on an open subset; hence, by unique continuation,
$s_1 = 0$ or
$t_1 = 0$ : a contradiction.
Henceforth, assume that
$\rho \geqslant 2$ . Define
$\delta ,\varepsilon \mskip 0.5mu\colon \thinspace U \to {\mathbf {N}}_0$ by
and
. By unique continuation,
$\delta $ and
$\varepsilon $ are positive on a dense open subset. In fact,
$\delta ,\varepsilon \geqslant 2$ on a dense open subset. To see this, observe that if
$\delta = 1$ on a nonempty open subset, then there is a nonempty open subset
$V \subset U$ and a function
$f \in C^{\infty }(V)$ such that
$s_1(x) = f(x)s_2(x)$ for every
$x \in V$ . Therefore,
$\sigma (\mathrm {d} f)s_2 = 0$ with
$\sigma $ denoting the symbol of D. Since D is elliptic, f must be constant: a contradiction to
$s_1$ and
$s_2$ being linearly independent. The same argument applies to
$\varepsilon $ .
If
$\rho = 2$ , then there exists an
$x \in U$ such that
$\delta (x) = \varepsilon (x) = 2$ ; therefore:
$\varpi (B)$ does not vanish at x. If
$\rho = 3$ , then there is an
$x \in U$ such that
$\min {\lbrace }{\delta (x),\varepsilon (x)}{\rbrace } \geqslant 2$ . If
$\delta (x) = \varepsilon (x) = 3$ , then
$\varpi (B)$ evidently does not vanishing at x; otherwise, without loss of generality,
$s_1(x)$ and
$s_2(x)$ are linearly independent, and
$s_3(x) = \lambda _1s_1(x) + \lambda _2s_2(x)$ . In the latter case,
$$ \begin{align*} \varpi(B)(x) = s_1(x) \otimes ({t_1(x)+\lambda_1t_3(x)}) + s_2(x)\otimes ({t_2(x) + \lambda_2t_3(x)}) \end{align*} $$
which cannot vanish because
$\varepsilon (x) \geqslant 2$ .
There are examples of first-order linear elliptic operators which fail to satisfy Petri’s condition up to rank four; see [Reference WendlWen19b, Example 5.5] or Proposition 2.5.4. Finally, a brief warning: The preceding observation is false when
$\mathbf {R}$ is replaced with
${\mathbf {C}}$ or
$\mathbf {H}$ . The analogue of Petri’s condition only holds up to rank one in this case. (The issue is that
$\sigma ({\mathrm {d} f})s_2 =0$ does not imply
$\mathrm {d} f = 0$ if f takes values in
${\mathbf {C}}$ or
$\mathbf {H}$ .)
1.2 Pulling back and twisting
This section introduces two constructions which produce new linear elliptic operators from old ones: pulling back by a covering map and twisting by a Euclidean local system.
Definition 1.2.1. Let
$k \in {\mathbf {N}}_0$
. Let
$\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$
be a covering map with
$\tilde M$
connected.Footnote
7
Let
$D\mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$
be a linear differential operator of order k. The pullback of D by
$\pi $
is the linear differential operator of order k

characterized by

Remark 1.2.2. If
$\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$
is a branched covering map of manifolds whose ramification locus is a closed submanifold of codimension two, then
$\tilde M$
and M can be equipped with orbifold structures and
$\pi $
can be lifted to an unbranched covering map of orbifolds. Section 2.7 discusses this construction in the case of Riemann surfaces; the higher-dimensional case follows immediately from the two-dimensional case and the above local model.
Definition 1.2.3. A Euclidean local system on M is a Euclidean vector bundle
${\underline V}$
over M together with a flat orthogonal connection.
Remark 1.2.4. Let
$x_0 \in M$
. Denote by
$\pi _1(M,x_0)$
the fundamental group with base-point
$x_0$
. If
$*$
denotes the usual concatenation of paths, then the multiplication in
$\pi _1(M,x_0)$
is defined by
.Footnote
8
Parallel transport induces a monodromy representation

with V denoting the fiber of
${\underline V}$
over
$x_0$
.
${\underline V}$
can be recovered from
$\mu $
as follows. Denote by
$\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$
the universal covering map and by
$\operatorname {\mathrm {Aut}}(\pi )$
the group of deck transformations. A choice of
$\tilde x_0 \in \pi ^{-1}(x_0)$
induces an anti-isomorphism from
$\operatorname {\mathrm {Aut}}(\pi )$
to
$\pi _1(X,x_0)$
. Therefore,
$\tilde M$
is a principal
$\pi _1(M,x_0)$
-bundle, and
${\underline V}$
is the associated bundle:

This sets up a bijection between gauge equivalence classes
$[{\underline V}]$
of Euclidean local systems of rank r and equivalence classes
$[\mu ]$
of representations
$\pi _1(M,x_0) \to \mathrm {O}(r)$
up to conjugation by
$\mathrm {O}(r)$
. For a more detailed discussion—in particular, of how the to interpret the above in the category of orbifolds—we refer the reader to [Reference Shen and YuSY19, Sections 2.4 and 2.5].
Definition 1.2.5. Let
$k \in {\mathbf {N}}_0$
Let
$D \mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$
be a linear differential operator of order k. Let
${\underline V}$
be a Euclidean local system on M. The twist of D by
${\underline V}$
is the linear differential operator of order k

characterized as follows: If U is a open subset M,
$s \in \Gamma (U,E)$
, and
$f \in \Gamma (U,{\underline V})$
is covariantly constant with respect to the flat connection on
${\underline V}$
, then

Proposition 1.2.9 shows that the pullback
$\pi ^*D$
is equivalent to the twist
$D^{\underline V}$
for a suitable choice of
${\underline V}$
. Its statement requires the following preparation.
Definition 1.2.6. Let
$\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$
be a finite covering map. Let E be a Euclidean vector bundle over
$\tilde M$
. The pushforward of E by
$\pi $
is the unique Euclidean vector bundle
$\pi _*E$
over X such that for every
$x \in X$

as Euclidean vector spaces and such that a section s of
$\pi _*E$
is smooth if and only if the induced section
$\tilde s$
of E is smooth.
Remark 1.2.7. The following facts about the construction from Definition 1.2.6 are important.
-
(1) If E is a Euclidean vector bundle over
$\tilde M$ , then the sheaf
$\Gamma (\cdot ,\pi _*E)$ is the sheaf-theoretic pushforward of the sheaf
$\Gamma (\cdot ,E)$ ; that is: There are canonical isomorphisms
$$ \begin{align*} \Gamma(U,\pi_*E) \cong \Gamma(\pi^{-1}(U),E) \end{align*} $$
$U \subset M$ and these are compatible with the restriction maps.
-
(2) If E is a Euclidean local system on
$\tilde M$ , then
$\pi _*E$ is a Euclidean local system on M:
$s \in \Gamma (U,\pi _*E)$ is covariantly constant if and only if
$\tilde s \in \Gamma (\pi ^{-1}(U),E)$ is.
-
(3) Let E and F be a Euclidean vector bundles over M and
$\tilde M$ respectively. For every
$x \in M$ , there is a conical isomorphism
$$ \begin{align*} \pi_*(\pi^* E\otimes F)_x \cong \bigoplus_{\tilde x \in \pi^{-1}(x)} E_x \otimes F_{\tilde x} \cong (E\otimes \pi_*F)_x. \end{align*} $$
These assemble into the push-pull formula
$$ \begin{align*} \pi_*(\pi^* E\otimes F) \cong E \otimes \pi_*F. \end{align*} $$
In particular,
$$ \begin{align*} \pi_*(\pi^*E) \cong E \otimes \pi_*{\underline{\mathbf{R}}}. \end{align*} $$
Here,
${\underline {\mathbf {R}}}$ denotes the trivial rank one Euclidean local system on
$\tilde M$ .
Definition 1.2.8. Let G be a group, and let
$H < G$
be a subgroup. The normal core of H is the normal subgroup

Proposition 1.2.9. Let
$k \in {\mathbf {N}}_0$
. Let
$\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$
be a finite covering map with
$\tilde M$
connected. Let
$x_0 \in M$
and
$\tilde x_0 \in \pi ^{-1}(x_0)$
. Denote by

the characteristic subgroup of the covering map and by N the normal core of C. Set

Denote by
${\underline {\mathbf {R}}}$
the trivial rank one Euclidean local system on
$\tilde M$
. Set

Let
$D \mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$
be a linear differential operator of order k. The following hold:
-
(1) The monodromy representation of
${\underline V}$ factors through
; indeed, it is induced by the representation of G on
$\operatorname {\mathrm {Map}}(S,\mathbf {R})$ defined by
-
(2) The push-pull formula induces isometries
$$ \begin{align*} \pi_*\mskip0.5mu\colon\thinspace W^{k,2}\Gamma(\pi^*E) \cong W^{k,2}\Gamma(E\otimes{\underline V}) \quad\text{and}\quad \pi_*\mskip0.5mu\colon\thinspace L^2\Gamma(\pi^*F) \cong L^2\Gamma(F\otimes{\underline V}) \end{align*} $$
$$ \begin{align*} D^{{\underline V}} = \pi_* \circ \pi^*D \circ \pi_*^{-1}. \end{align*} $$
Remark 1.2.10. If
$\pi $
is a normal covering, then
$C=N$
and
$G = \pi _1(M,x_0)/N$
is isomorphic to its deck transformation group. If
$\pi $
has k sheets, then C has index k. Its normal core has index at most
$k!$
by an elementary result known as Poincaré’s theorem. This theorem follows from the observation that the kernel of the canonical homomorphism
$\pi _1(M,x_0) \to \operatorname {\mathrm {Bij}}(G/C)$
is precisely N and
$\operatorname {\mathrm {Bij}}(G/C) \cong S_k$
. Here,
$\operatorname {\mathrm {Bij}}(G/C)$
denotes the set of bijections of
$G/C$
,
Proof of Proposition 1.2.9.
The monodromy representation
$\mu \mskip 0.5mu\colon \thinspace \pi _1(M,x_0) \to \mathrm {O}(V)$
of
${\underline V}$
is trivial on C; hence, it factors through G. Denote by
$\rho \mskip 0.5mu\colon \thinspace (\hat M,\hat x_0) \to (M,x_0)$
the pointed covering map with characteristic subgroup N.
$\hat M$
is a principal G-bundle and
$\tilde M \cong \hat M \times _G S$
. This implies the assertion about the monodromy representation.

Denote the resulting isomorphism
$\Gamma (\pi ^*E) \cong \Gamma (\pi _*\pi ^*E) \cong \Gamma (E\otimes {\underline V})$
by
$\pi _*$
. For
$s \in \Gamma (E)$
and
$f \in C^{\infty }(\tilde M)$

Let U be an open subset of M,
$s \in \Gamma (U,E)$
, and
$f \in \Gamma (U,{\underline V})$
. Suppose that f is covariantly constant. This is equivalent to the corresponding function
on
being locally constant. By the characterizing properties of
$D^{\underline V}$
and
$\pi ^*D$
and since
$\pi ^*D$
is a differential operator,

and

This proves that
$D^{{\underline V}} = \pi _* \circ \pi ^*D \circ \pi _*^{-1}$
.
1.3 Equivariant Brill–Noether loci, I: twists
Pulling back and twisting lead to families of linear elliptic differential operators which fail to satisfy the hypotheses of Theorem 1.1.5 (except for a few corner cases). In this section, we formulate a variant of this result which applies to families of twisted linear elliptic differential operators. Throughout this section, assume the following.
Situation 1.3.1. Let
$x_0 \in M$
. Let
${\mathfrak V} = ({\underline V}_{\alpha })_{\alpha =1}^m$
be a finite collection of irreducible Euclidean local systems which are pairwise nonisomorphic. (A Euclidean local system is irreducible if it is not a direct sum of two nonzero Euclidean local systems.) For every
$\alpha =1,\ldots ,m$
, denote by
${\mathbf {K}}_{\alpha }$
the algebra of parallel endomorphisms of
${\underline V}_{\alpha }$
and set
.
Remark 1.3.2. Since
${\underline V}_{\alpha }$
is irreducible,
${\mathbf {K}}_{\alpha }$
is a division algebra; hence, by Frobenius’ Theorem it is isomorphic to either
$\mathbf {R}$
,
${\mathbf {C}}$
, or
$\mathbf {H}$
and
$k_{\alpha } \in {\lbrace }{1,2,4}{\rbrace }$
.
If D is a linear elliptic differential operator, then the twists
$D^{{\underline V}_{\alpha }}$
commute with the action of
${\mathbf {K}}_{\alpha }$
. Therefore,
$\ker D^{{\underline V}_{\alpha }}$
and
$\operatorname {\mathrm {coker}} D^{{\underline V}_{\alpha }}$
are left
${\mathbf {K}}_{\alpha }$
-modules and, hence, right
${\mathbf {K}}_{\alpha }^{\mathrm {op}}$
-modules. Here,
${\mathbf {K}}_{\alpha }^{\mathrm {op}}$
denotes the opposite algebra of
${\mathbf {K}}_{\alpha }$
.
Definition 1.3.3. Let
$(D_p)_{p \in \mathscr {P}}$
be a family of linear elliptic differential operators. For
$d,e \in {\mathbf {N}}_0^m$
, define the
${\mathfrak V}$
-equivariant Brill–Noether locus
$\mathscr {P}_{d,e}^{\mathfrak V}$
by

Remark 1.3.4. Let
$i \in \mathbf {Z}^m$
. Let
$(D_p)_{p \in \mathscr {P}}$
be a family of linear elliptic operators such that
$\operatorname {\mathrm {index}}_{{\mathbf {K}}_{\alpha }} D_p^{{\underline V}_{\alpha }} = i_{\alpha }$
for every
$p \in \mathscr {P}$
and
$\alpha = 1,\ldots ,m$
. If
$\mathscr {P}_{d,e}^{\mathfrak V} \neq \varnothing $
, then
$d-e = i$
; in particular:
$d_{\alpha } \geqslant i_{\alpha }$
and
$e_{\alpha } \geqslant -i_{\alpha }$
.
If M is a manifold, then

by the Atiyah–Singer index theorem; therefore, the
$i_{\alpha }$
all have the same sign. If M is an orbifold, there are corrections terms in the index formula which spoil this relation between the indices; see, for example, Proposition 2.8.6.
Lemma 1.1.4 immediately implies the following.
Theorem 1.3.5. Let
$(D_p)_{p \in \mathscr {P}}$
be a family of linear elliptic differential operators. For
$p \in \mathscr {P}$
define
$\Lambda _p^{\mathfrak V} \mskip 0.5mu\colon \thinspace T_p\mathscr {P} \to \bigoplus _{\alpha =1}^m \operatorname {\mathrm {Hom}}_{{\mathbf {K}}_{\alpha }}(\ker D_p^{{\underline V}_{\alpha }},\operatorname {\mathrm {coker}} D_p^{{\underline V}_{\alpha }})$
by

Let
$d,e \in {\mathbf {N}}_0^m$
and
$p \in \mathscr {P}_{d,e}^{\mathfrak V}$
. If
$\Lambda _p^{\mathfrak V}$
is is surjective, then the following hold:
-
(1) There is an open neighborhood
$\mathscr {U}$ of
$p \in \mathscr {P}$ such that
$\mathscr {P}_{d,e}^{\mathfrak V} \cap \mathscr {U}$ is a submanifold of codimension
$$ \begin{align*} \operatorname{\mathrm{codim}} (\mathscr{P}_{d,e}^{\mathfrak V} \cap \mathscr{U}) = \sum_{\alpha=1}^m k_{\alpha} d_{\alpha} e_{\alpha}. \end{align*} $$
-
(2)
$\mathscr {P}_{\tilde d,\tilde e}^{\mathfrak V} \neq \varnothing $ for every
$\tilde d,\tilde e \in {\mathbf {N}}_0^m$ with
$\tilde d \leqslant d, \tilde e \leqslant e$ , and
$\tilde d-\tilde e = d-e$ .
Remark 1.3.6. If E and F are Hermitian vector bundles and
$(D_p)_{p \in \mathscr {P}}$
is a family of complex linear elliptic differential operators, then Theorem 1.3.5 does not apply; cf. Remark 1.1.6. Again, this issue is rectified by replacing
$\mathbf {R}$
with
${\mathbf {C}}$
throughout. In fact, this somewhat simplifies the discussion since
${\mathbf {C}}$
is the unique complex division algebra; hence, there is no need to introduce
${\mathbf {K}}_{\alpha }$
.
Remark 1.3.7 (Twists by arbitrary Euclidean local systems).
Every Euclidean local system
${\underline V}$
decomposes into irreducible local systems

with
$\ell _1,\ldots ,\ell _m \in {\mathbf {N}}_0$
for a suitable choice of
${\mathfrak V}$
. For every
$\bar d,\bar e \in {\mathbf {N}}_0$
the Brill–Noether locus

is the finite disjoint union of the subsets
$\mathscr {P}_{d,e}^{\mathfrak V}$
with
$(d,e) \in {\mathbf {N}}_0^m\times {\mathbf {N}}_0^m$
satisfying

Through this observation Theorem 1.3.5 can be brought to bear on families of linear elliptic differential operators twisted by
${\underline V}$
.
Definition 1.1.9, Definition 1.1.11, and Proposition 1.1.12 have the following analogues in the present situation.
Definition 1.3.8. A family of linear elliptic differential operators
$(D_p)_{p \in \mathscr {P}}$
is
${\mathfrak V}$
-equivariantly flexible in U at
$p_{\star } \in \mathscr {P}$
if for every
$A \in \Gamma _c(U,\operatorname {\mathrm {Hom}}(E,F))$
there is a
$\hat p \in T_{p_{\star }}\mathscr {P}$
such that

for every
$\alpha = 1,\ldots ,m$
and
$s \in \ker D_{p_{\star }}^{{\underline V}_{\alpha }}$
.
Definition 1.3.9. The
${\mathfrak V}$
-equivariant Petri map

is defined by
with
$\varpi _{\alpha }$
denoting the composition of the Petri map

and the map induced by

Here,
is the dual of
${\underline V}_{\alpha }$
. (Its algebra of parallel endomorphisms is
${\mathbf {K}}_{\alpha }^{\mathrm {op}}$
.)
Let
$U \subset M$
be an open subset. A linear elliptic differential operator
$D\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$
satisfies the
${\mathfrak V}$
-equivariant Petri condition in U if the map

induced by the
${\mathfrak V}$
-equivariant Petri map is injective. Here,
.
Remark 1.3.10. The
${\mathfrak V}$
-equivariant Petri condition appears even more difficult to verify than the Petri condition. It turns out, however, that there is a general method for verifying both of these conditions simultaneously. This will be discussed in Section 1.6.
Proposition 1.3.11. Let
$(D_p)_{p \in \mathscr {P}}$
be a family of linear elliptic differential operators. Let
$U \subset M$
be an open subset. If
$(D_p)_{p \in \mathscr {P}}$
is
${\mathfrak V}$
-equivariantly flexible in U at
$p_{\star } \in \mathscr {P}$
and
$D_{p_{\star }}$
satisfies the
${\mathfrak V}$
-equivariant Petri condition in U, then the map
$\Lambda _{p_{\star }}^{\mathfrak V}$
defined in Theorem 1.3.5 is surjective.
Remark 1.3.12. There are analogues of the observations from Remark 1.1.14 in the equivariant setting.
-
(1) Every
$p \in \mathscr {P}_{d,e}^{\mathfrak V}$ has an open neighborhood
$\mathscr {U}$ in
$\mathscr {P}$ such that
$\mathscr {P}_{d,e}^{\mathfrak V}\cap \mathscr {U}$ is contained in a submanifold of codimension
$\operatorname {\mathrm {rk}} \Lambda _p^{\mathfrak V}$ .
-
(2) Let
$\rho \in {\mathbf {N}}_0^m$ , and let
$U \subset M$ be an open subset. A linear elliptic differential operator
$D\mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$ satisfies the
${\mathfrak V}$ -equivariant Petri condition up to rank
$\rho $ in U if for every nonzero
$B = (B_1,\ldots ,B_m) \in \bigoplus _{\alpha =1}^m \ker D_p^{{\underline V}_{\alpha }} \otimes _{\,{\mathbf {K}}_{\alpha }^{\mathrm {op}}} \ker D_p^{\dagger ,{\underline V}_{\alpha }^*}$ with
$\operatorname {\mathrm {rk}} B_{\alpha } \leqslant \rho _{\alpha }$ for
$\alpha = 1,\ldots ,m$ the section
$\varpi ^{\mathfrak V}(B)$ does not vanish on U. If
$D_p$ satisfies this condition and
$(D_p)_{p\in \mathscr {P}}$ is
${\mathfrak V}$ -equivariantly flexible in U at
$p_{\star } \in \mathscr {P}$ , then
$$ \begin{align*} \operatorname{\mathrm{rk}} \Lambda_{p_{\star}} \geqslant \sum_{\alpha=1}^m \min{\lbrace}{\rho_{\alpha},d_{\alpha},e_{\alpha}}{\rbrace}\max{\lbrace}{d_{\alpha},e_{\alpha}}{\rbrace}. \end{align*} $$
-
(3) Let
$\rho \in {\mathbf {N}}_0^m$ , and let
$U \subset M$ be an open subset. Every first-order linear elliptic differential operator
$D \mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ satisfies the
${\mathfrak V}$ -equivariant Petri condition up to rank
$\rho $ on U provided
(1.3.13)$$ \begin{align} \sum_{\alpha=1}^m \operatorname{\mathrm{rk}}_{\mathbf{R}} V_{\alpha} \cdot \rho_{\alpha} \leqslant 3. \end{align} $$
Proof. Set
. Denote by
$\pi \mskip 0.5mu\colon \thinspace (\tilde M,\tilde x_0) \to (M,x_0)$ the universal covering map. Every
$s \in \ker D^{{\underline V}_{\alpha }}$ can be regarded as an element
$\tilde s \in \Gamma (\pi ^*E\otimes V_{\alpha })^G$ in the space of G-invariant sections, with
$G \to \mathrm {O}(V_{\alpha })$ denoting the monodromy representation of
${\underline V}_{\alpha }$ . This section can be regarded as
sections
$s_1,\ldots ,s_{r_{\alpha }}$ of
$\pi ^*E$ . For every
$\alpha =1,\ldots ,m$ let
$s_1^{\alpha },\ldots ,s_{q_{\alpha }}^{\alpha } \in \ker D^{{\underline V}_{\alpha }}$ be linearly independent over
${\mathbf {K}}_{\alpha }$ . The resulting collection of sections
$s_{j,k}^{\alpha } \in \ker \pi ^*D$ (
$\alpha = 1,\ldots ,m, j=1,\ldots ,q_{\alpha }, k=1,\ldots ,r_{\alpha }$ ) are linearly independent. The latter is a consequence of Proposition 1.3.14 applied to
$W = \ker \pi ^*D$ . (Analogous statements hold for
$D^{\dagger }$ instead of D.) At this point, one can apply the argument in Remark 1.1.14(3).
Unfortunately, this is not as useful as Remark 1.1.14(3) because equation (1.3.13) is very restrictive; however, it is what lies at the heart of Eftekhary’s proof of the
$4$ -rigidity conjecture [Reference EftekharyEft16].
We end this section with a useful fact of representation theory, which is essentially a consequence of Schur’s lemma. This fact is needed to justify Remark 1.3.12(3) above. More importantly, it plays a crucial role in verifying the
${\mathfrak V}$
-equivariant Petri condition in Section 1.6, specifically in Proposition 1.6.6.
Let G be a group. For a vector space V with an action of G, denote by
$V^G$
the subspace of G-invariant vectors, and set
and
with W a further vector space with an action of G. We do not assume here that G is finite or that V or W are finite dimensional. For example, as in Remark 1.3.12(3), G could be the monodromy representation of a local system and V and W kernels of linear elliptic operators twisted by that local system.
Proposition 1.3.14. Let
$({V_{\alpha }})_{\alpha =1}^m$
be a finite collection of irreducible finite-dimensional orthogonal representations which are pairwise nonisomorphic. Set
. For every representation W of G, the map

induced by the trace maps
$V_{\alpha }^* \otimes _{\,{\mathbf {K}}_{\alpha }} V_{\alpha } \to \mathbf {R}$
is injective.
Proof of Proposition 1.3.14.
If
$\operatorname {\mathrm {tr}}$
is not injective, then there are finite-dimensional
${\mathbf {K}}_{\alpha }$
-linear subspaces
$X_{\alpha } \subset (V_{\alpha }\otimes W)^G$
such that

is not injective. (W need not be finite dimensional.)
G acts on
$V_{\alpha }^*$
via the contragredient representation and trivially on
$X_{\alpha }$
. Choose a
${\mathbf {K}}_{\alpha }$
-sesquilinear inner product on
$X_{\alpha }$
(e.g., by choosing a basis
$X_{\alpha } \cong {\mathbf {K}}_{\alpha }^{d_{\alpha }}$
). This exhibits
$V_{\alpha }^* \otimes _{\,{\mathbf {K}}_{\alpha }} X_{\alpha }$
as an orthogonal representation. Since
$\operatorname {\mathrm {tr}}$
is G-equivariant,

is an orthogonal subrepresentation.
Since
$\ker \operatorname {\mathrm {tr}}$
is an orthogonal representation, it decomposes into irreducible orthogonal representations

A priori,
$(V_{\beta })_{\beta =1}^n$
might not be a subset of
$(V_{\alpha })_{\alpha =1}^m$
. However, for every copy of
$V_{\beta }^*$
appearing in the above decomposition, the induced map

is injective. Therefore, by Schur’s lemma,
$\beta $
is among the
$\alpha $
. Moreover, the image of the of the above map is
$V_{\beta }^* \otimes _{{\mathbf {K}}_{\beta }} L$
for some
$L \subset X_{\beta }$
with
$\dim _{{\mathbf {K}}_{\beta }} L = 1$
. Denote by
$S_{\beta } \subset X_{\beta }$
the
${\mathbf {K}}_{\beta }$
-linear subspace spanned by the corresponding lines L. The upshot of this discussion is that

For every nonzero
$T \in S_{\alpha }$
, there is a
$v^* \in V_{\alpha }^*$
with
$\operatorname {\mathrm {tr}}(v^* \otimes T) \neq 0$
. Therefore,
$S_{\alpha } = 0$
; hence,
$\ker \operatorname {\mathrm {tr}} = 0$
.
Proposition 1.3.15. In the situation of Proposition 1.3.14, suppose that we have two representations V and W of G. In that case, the trace map

is injective.
Proof. By Proposition 1.3.14, the map

is injective. Tensoring with the identity map
$V \to V$
, we get that

is injective. This map is G-equivariant, where the action on
$(V_{\alpha }\otimes W)^G$
is trivial. Thus, it induces an injective map on the G-invariant parts.
1.4 Equivariant Brill–Noether loci, II: pullbacks
In this section, we formulate a variant of Theorem 1.1.5 which applies to families of linear elliptic differential operators pulled back by a finite normal covering map. While this is not needed in Part 2, this approach to equivariant Brill–Noether theory for elliptic operators is similar in spirit to the framework used by Wendl in his proof of the superrigidity conjecture. In Section 1.5, we relate this approach to the one from Section 1.3.
Throughout this section, assume the following.
Situation 1.4.1. Let
$x_0 \in M$
. Let G be the quotient of
$\pi _1(M,x_0)$
by a finite index normal subgroup N. Denote by
$\pi \mskip 0.5mu\colon \thinspace (\tilde M,\tilde x_0) \to (M,x_0)$
a pointed covering map with characteristic subgroup N. Let

be the irreducible representations of G (up to isomorphism). Set

If
$D\mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$
is a linear elliptic differential operator of order k, then
$\ker \pi ^*D$
and
$\operatorname {\mathrm {coker}} \pi ^*D$
are representations of G. Every representation V of G can be decomposed into irreducible representations. The evaluation map defines a G-equivariant isomorphism

Hence,

In particular,
$d = (d_1,\ldots ,d_m) \in {\mathbf {N}}^m$
determines V up to isomorphism.
Definition 1.4.3. Let
$(D_p)_{p \in \mathscr {P}}$
be a family of linear elliptic differential operators. For
$d,e \in {\mathbf {N}}_0^m$
define the G
-equivariant Brill–Noether locus
$\mathscr {P}_{d,e}^G$
by

Remark 1.4.4. Let
$D\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$
is a linear elliptic differential operator. The G-equivariant index of
$\pi ^*D$
is
. Here,
$R(G)$
denotes the representation ring of G; its elements are formal differences of isomorphism classes of representations of G. It is a consequence of the above discussion that
$R(G) \cong \mathbf {Z}^m$
as abelian groups.
For families of linear elliptic operators with G-equivariant index corresponding to
$i \in \mathbf {Z}^m$
, what was said in Remark 1.3.4 applies in the present situation as well.
Lemma 1.1.4 has the following refinement for G-equivariant Fredholm operators.
Lemma 1.4.5. Let X and Y be two Banach spaces equipped with G-actions. Denote by
$\mathscr {F}_G(X,Y)$
the space of G-equivariant Fredholm operators. For every
$L \in \mathscr {F}_G(X,Y)$
, there is an open neighborhood
$\mathscr {U} \subset \mathscr {F}_G(X,Y)$
and a smooth map
$\mathscr {S} \mskip 0.5mu\colon \thinspace \mathscr {U} \to \operatorname {\mathrm {Hom}}_G(\ker L,\operatorname {\mathrm {coker}} L)$
such that for every
$T \in \mathscr {U}$
there are G-equivariant isomorphisms

furthermore,
$\mathrm {d}_L\mathscr {S} \mskip 0.5mu\colon \thinspace T_L\mathscr {F}_G(X,Y) \to \operatorname {\mathrm {Hom}}_G(\ker L,\operatorname {\mathrm {coker}} L)$
satisfies

Proof. The proof of Lemma 1.1.4 carries over provided
$\operatorname {\mathrm {coim}} L$
and the lift of
$\operatorname {\mathrm {coker}} L$
are chosen G-invariant.
Lemma 1.4.5 immediately implies the following.
Theorem 1.4.6. Let
$(D_p)_{p \in \mathscr {P}}$
be a family of linear elliptic differential operators. For
$p \in \mathscr {P}$
define
$\Lambda _p^G \mskip 0.5mu\colon \thinspace T_p\mathscr {P} \to \operatorname {\mathrm {Hom}}_G(\ker \pi ^*D_p,\operatorname {\mathrm {coker}} \pi ^*D_p)$
by

Let
$d,e \in {\mathbf {N}}_0^m$
and
$p \in \mathscr {P}_{d,e}^G$
. If
$\Lambda _p^G$
is surjective, then the following hold:
-
(1) There is an open neighborhood
$\mathscr {U}$ of
$p \in \mathscr {P}$ such that
$\mathscr {P}_{d,e}^G \cap \mathscr {U}$ is a submanifold of codimension
$$ \begin{align*} \operatorname{\mathrm{codim}} (\mathscr{P}_{d,e}^G \cap \mathscr{U}) = \sum_{\alpha=1}^m k_{\alpha} d_{\alpha} e_{\alpha}. \end{align*} $$
-
(2)
$\mathscr {P}_{\tilde d,\tilde e}^G \neq \varnothing $ for every
$\tilde d,\tilde e \in {\mathbf {N}}_0^m$ with
$\tilde d \leqslant d, \tilde e \leqslant e$ and
$\tilde d-\tilde e = d-e$ .
Remark 1.4.7 (Pullbacks by arbitrary covering maps).
Suppose that
$\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$
is a finite covering map with characteristic subgroup
$C < \pi _1(M,x_0)$
. Denote by N the normal core of C, denote by
$\rho \mskip 0.5mu\colon \thinspace (\hat M,\hat x_0) \to (M,x_0)$
the pointed covering map with characteristic subgroup N and set
. For
the decomposition (1.4.2) of
$\operatorname {\mathrm {Map}}(S,\mathbf {R})$
is

indeed: The map
$\mathrm {ev}_{[\mathbf {1}]}\mskip 0.5mu\colon \thinspace \operatorname {\mathrm {Hom}}_G(V_{\alpha },\operatorname {\mathrm {Map}}(S,\mathbf {R})) \to V_{\alpha }^*$
defined by
is injective and its image is
$(V_{\alpha }^*)^C$
. Therefore, by Proposition 1.2.9(2)

With the above in mind, Theorem 1.4.6 can be brought to bear on nonnormal covering maps; cf. Remark 1.3.7.
Definition 1.1.9, Definition 1.1.11, and Proposition 1.1.12 have the following analogues in the present situation.
Definition 1.4.8. A family of linear elliptic differential operators
$(D_p)_{p \in \mathscr {P}}$
is G
-equivariantly flexible in U at
$p_{\star } \in \mathscr {P}$
if for every
$A \in \Gamma (\operatorname {\mathrm {Hom}}(E,F))$
supported in U there is a
$\hat p \in T_{p_{\star }}\mathscr {P}$
such that

for every
$s \in \ker \pi ^*D_{p_{\star }}$
.
Definition 1.4.9. Let
$U \subset M$
be an open subset. A linear elliptic differential operator
$D\mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$
satisfies the G
-equivariant Petri condition in U if the map

induced by the Petri map is injective.
Remark 1.4.10. Remark 1.3.10 applies to the the G-equivariant Petri condition as well.
Proposition 1.4.11. Let
$(D_p)_{p \in \mathscr {P}}$
be a family of linear elliptic differential operators. Let
$U \subset M$
be an open subset. If
$(D_p)_{p \in \mathscr {P}}$
is G-equivariantly flexible in U at
$p_{\star } \in \mathscr {P}$
and
$D_{p_{\star }}$
satisfies the G-equivariant Petri condition in U, then the map
$\Lambda _{p_{\star }}^G$
defined in Theorem 1.4.6 is surjective.
1.5 Equivariant Brill–Noether loci, III: comparison
This section discusses the relation between the two approaches to Brill–Noether theory of equivariant elliptic operators: using local systems, discussed in Section 1.3 and using group actions, discussed in Section 1.4. Results from these sections are not used in Part 2. Throughout this section, assume Situation 1.4.1. This yields an instance of Situation 1.3.1 by setting

Denote by
$\sigma \in S_m$
the permutation such that
$V_{\alpha }^* \cong V_{\sigma (\alpha )}$
. The following summarizes the what lies at the heart of the relation.
Proposition 1.5.1. Let
$D\mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$
be a linear differential operator of order k. The following hold:
-
(1) The action of G by deck transformations of
$\pi $ induces a G-action on the local system
The isomorphisms
$\pi _* \mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (\pi ^*E) \cong W^{k,2}\Gamma (E\otimes {\underline V})$ and
$\pi _* \mskip 0.5mu\colon \thinspace L^2\Gamma (\pi ^*F) \cong L^2\Gamma (F\otimes {\underline V})$ from Proposition 1.2.9(2) are G-equivariant.
-
(2) There is a G-equivariant isomorphism
$$ \begin{align*} \phi \mskip0.5mu\colon\thinspace {\underline V} \cong \bigoplus_{\alpha=1}^m V_{\alpha}^* \otimes_{\,{\mathbf{K}}_{\alpha}} {\underline V}_{\alpha}. \end{align*} $$
Here, G acts on
$V_{\alpha }^*$ .
-
(3) Denote by
$$ \begin{align*} \psi_E \mskip0.5mu\colon\thinspace W^{k,2}\Gamma(\pi^*E) \cong W^{k,2}\Gamma(E \otimes {\underline V}) \cong \bigoplus_{\alpha=1}^m V_{\alpha}^* \otimes_{\,{\mathbf{K}}_{\alpha}} W^{k,2}\Gamma(E \otimes {\underline V}_{\alpha}) \end{align*} $$
$\pi _*$ and
$\phi $ (and analogously for F and
$F^{\dagger }$ ). The composition
$$ \begin{align*} \psi_F \circ \pi^*D \circ \psi_E^{-1} \end{align*} $$
$$ \begin{align*} \bigoplus_{\alpha=1}^m \mathrm{id}_{V_{\alpha}^*} \otimes_{\,{\mathbf{K}}_{\alpha}} D^{{\underline V}_{\alpha}} \mskip0.5mu\colon\thinspace \bigoplus_{\alpha=1}^m V_{\alpha}^*\otimes_{\,{\mathbf{K}}_{\alpha}} W^{k,2}\Gamma(E\otimes {\underline V}_{\alpha}) \to \bigoplus_{\alpha=1}^m V_{\alpha}^*\otimes_{\,{\mathbf{K}}_{\alpha}} L^2\Gamma(F\otimes {\underline V}_{\alpha}). \end{align*} $$
-
(4) Define
$\gamma $ to be the composition of the isomorphisms
Here,
$(\star )$ is induced by the identification
$$ \begin{align*} ({V_{\alpha}^* \otimes V_{\sigma(\beta)}})^G = \begin{cases} {\mathbf{K}}_{\alpha}^{\mathrm{op}} & \text{if } \alpha = \sigma(\beta) \\ 0 & \text{otherwise}. \end{cases} \end{align*} $$
The following diagram commutes:
(1.5.2)
Proof. Let
$g \in G$
. Denote by
$\delta _g$
the corresponding deck transformation:
. There is a canonical isomorphism
${\underline {\mathbf {R}}} \cong ({\delta _g})_*{\underline {\mathbf {R}}}$
identifying
${\underline {\mathbf {R}}}_{\tilde x} = \mathbf {R} = ({(\delta _g)_*{\underline {\mathbf {R}}}})_{\tilde x} = {\underline {\mathbf {R}}}_{\tilde xg}$
. This defines an isomorphism

A moment’s thought shows that this isomorphism maps
$v \in {\underline V}_x = C^{\infty }(\pi ^{-1}(x),\mathbf {R})$
to
$gv \in {\underline V}_x$
defined by
. These isomorphisms (1.5.3) assemble into a G-action on
${\underline V}$
. This description makes (1) evident.
The left and right regular representations of G on

are defined by

respectively. By Proposition 1.2.9(1), the monodromy representation of
${\underline V}$
is
$\lambda $
; that is,
${\underline V} \cong \tilde M \times _{\lambda } \mathbf {R}[G]$
. Since
$\lambda $
and
$\rho $
commute,
$\rho $
defines an action of G on
${\underline V}$
. This is precisely the action described above.
Since
$\lambda _g$
and
$\rho _h$
commute,
$(g,h) \mapsto \lambda _g\circ \rho _h$
defines a representation of
$G\times G$
on
$\mathbf {R}[G]$
.
$G\times G$
also acts on
$V_{\alpha }^*\otimes _{\,{\mathbf {K}}_{\alpha }} V_{\alpha }$
via
$(g,h) \mapsto \mu _{\alpha }(h^{-1})^* \otimes \mu _{\alpha }(g)$
. The isomorphism (1.4.2) corresponding to
$\lambda $
reads

$\operatorname {\mathrm {Hom}}_G(V_{\alpha },\mathbf {R}[G])$
inherits a G-action from
$\rho $
. The map
$\mathrm {ev}_{\mathbf {1}}\mskip 0.5mu\colon \thinspace \operatorname {\mathrm {Hom}}_G(V_{\alpha },\mathbf {R}[G]) \to V_{\alpha }^*$
defined by
is a G-equivariant isomorphism. This yields the
$G\times G$
-equivariant Peter–Weyl isomorphism

It induces a G-equivariant isomorphism

This proves (2).
(2) and Proposition 1.2.9(2) imply (3).
It suffices to prove (4) for
$M = {\lbrace }{\mathbf {1}}{\rbrace }$
and
$\tilde M = G$
. In this case, E and
$F^{\dagger }$
are vector spaces,
$\Gamma (\pi ^*E) = \operatorname {\mathrm {Map}}(G,E) = \mathbf {R}[G]\otimes E$
with the G-action induced by
$\rho $
,
$\Gamma (E \otimes {\underline V}_{\alpha }) = E \otimes V_{\alpha }$
(and analogously for
$F^{\dagger }$
and
$E\otimes F^{\dagger }$
). The diagram (1.5.2) becomes

Since every map in this diagram has a factor
$\mathrm {id}_{E\otimes F^{\dagger }}$
, it suffices to prove that it commutes for
$E = F^{\dagger } = \mathbf {R}$
.
The map
$\varpi \mskip 0.5mu\colon \thinspace (\mathbf {R}[G]\otimes \mathbf {R}[G])^G \to (\mathbf {R}[G])^G$
is the pointwise multiplication and the map
$(\pi ^*)^{-1}\mskip 0.5mu\colon \thinspace \mathbf {R}[G]^G \to \mathbf {R}$
is evaluation at
$\mathbf {1}$
. Therefore,

The computation of the composition
$\varpi ^{\mathfrak V} \circ \gamma \circ (\psi \otimes \psi )$
relies on the following.
Proposition 1.5.7. After identifying
$V_{\alpha }^* \otimes _{\,{\mathbf {K}}_{\alpha }} V_{\alpha } = \operatorname {\mathrm {End}}_{{\mathbf {K}}_{\alpha }}(V_{\alpha })$
, the Peter–Weyl isomorphism (1.5.4) is given by

Proof. The inverse of the evaluation map
$\mathrm {ev}\mskip 0.5mu\colon \thinspace \bigoplus _{\alpha =1}^m \operatorname {\mathrm {Hom}}_G(V_{\alpha },V) \otimes _{\,{\mathbf {K}}_{\alpha }} V_{\alpha } \cong V$
is the map
$\Pi = (\Pi _1,\ldots ,\Pi _m)$
with

Here,

,
$e_{\alpha ,i}$
(
$i=1,\ldots ,r_{\alpha }$
) is a basis of
$V_{\alpha }$
and
$e_{\alpha ,i}^*$
(
$i=1,\ldots ,r_{\alpha }$
) is the dual basis of
$V_{\alpha }^*$
. Indeed,

Here, the the first identity follows by direct inspection, the second uses the existence of the Peter–Weyl isomorphism (1.5.4), and the last identity follows by direct computation of
${\operatorname {\mathrm {tr}}} \circ \lambda $
. (The composition of
$\Pi _{\alpha }$
with
$\mathrm {ev}_{\alpha } \mskip 0.5mu\colon \thinspace \operatorname {\mathrm {Hom}}_G(V_{\alpha },V)\otimes _{\,{\mathbf {K}}_{\alpha }} V_{\alpha } \to V$
is the projection to the
$V_{\alpha }$
-isotypic component.)
The Peter–Weyl isomorphism (1.5.4) is the composition

By direct computation

This yields the asserted expression for
$\psi $
.
In the definition of
$\gamma $
, the map
$(V_{\alpha }^* \otimes V_{\alpha })^G \to {\mathbf {K}}_{\alpha }$
in
$(\star )$
is induced by the composition

Therefore,
$\gamma $
is induced by
$1/\dim _{{\mathbf {K}}_{\alpha }} V_{\alpha }$
times the map
$\operatorname {\mathrm {End}}_{{\mathbf {K}}_{\alpha }}(V_{\alpha }) \otimes \operatorname {\mathrm {End}}_{{\mathbf {K}}_{\alpha }^{\mathrm {op}}}(V_{\alpha }^*) \to \operatorname {\mathrm {End}}_{{\mathbf {K}}_{\alpha }}(V_{\alpha })$
,
$A\otimes B \mapsto A \circ B^*$
. The Petri map
$\varpi ^{\mathfrak V}$
is the sum of the traces
$\operatorname {\mathrm {tr}}\mskip 0.5mu\colon \thinspace \operatorname {\mathrm {End}}_{{\mathbf {K}}_{\alpha }}(V_{\alpha }) \to \mathbf {R}$
. Therefore,

Here, the second identity follows as in the proof of Proposition 1.5.7 above, and the third identity uses the G-invariance:
$a_{g,g} = a_{\mathbf {1},\mathbf {1}}$
.
With Proposition 1.5.1 in hand, the discussions in Section 1.3 and Section 1.4 can be related as follows:
-
(1) By Proposition 1.5.1(3), For every
$\alpha =1,\ldots ,m$ the isomorphisms
$\psi _E$ and
$\psi _F$ induce isomorphisms
$$ \begin{align*} \operatorname{\mathrm{Hom}}_G(V_{\alpha},\ker \pi^*D) \cong \ker D^{{\underline V}_{\alpha}^*} \quad\text{and}\quad \operatorname{\mathrm{Hom}}_G(V_{\alpha},\operatorname{\mathrm{coker}} \pi^*D) \cong \operatorname{\mathrm{coker}} D^{{\underline V}_{\alpha}^*} \end{align*} $$
(and analogously for
$\pi ^* D^{\dagger }$ and
$D^{{\underline V}_{\alpha },\dagger }$ ). If V and W are representations of G, then equation (1.4.2) induces isomorphisms
$$ \begin{align*} \operatorname{\mathrm{Hom}}_G(V,W) &\cong \bigoplus_{\alpha=1}^m \operatorname{\mathrm{Hom}}_{{\mathbf{K}}_{\alpha}}(\operatorname{\mathrm{Hom}}_G(V_{\alpha}^*,V),\operatorname{\mathrm{Hom}}_G(V_{\alpha}^*,W)) \quad\text{and} \\ (V\otimes W)^G &\cong \bigoplus_{\alpha=1}^m \operatorname{\mathrm{Hom}}_G(V_{\alpha}^*,V) \otimes_{\,{\mathbf{K}}_{\alpha}^{\mathrm{op}}} \operatorname{\mathrm{Hom}}_G(V_{\alpha},W). \end{align*} $$
Hence, there are isomorphisms
$$ \begin{align*} \eta\mskip0.5mu\colon\thinspace \operatorname{\mathrm{Hom}}_G(\ker \pi^*D,\operatorname{\mathrm{coker}} \pi^*D) &\to \bigoplus_{\alpha=1}^m \operatorname{\mathrm{Hom}}_{{\mathbf{K}}_{\alpha}}(\ker D^{{\underline V}_{\alpha}},\operatorname{\mathrm{coker}} D^{{\underline V}_{\alpha}}) \quad\text{and} \\ \tau\mskip0.5mu\colon\thinspace ({\ker \pi^*D \otimes \ker \pi^*D^{\dagger}})^G &\to \bigoplus_{\alpha=1}^m \ker D^{{\underline V}_{\alpha}}\otimes_{\,{\mathbf{K}}_{\alpha}^{\mathrm{op}}} \ker D^{{\underline V}_{\alpha},\dagger}. \end{align*} $$
-
(2) In the situation of Definition 1.3.3 and Definition 1.4.3,
$$ \begin{align*} \mathscr{P}_{d,e}^G = \mathscr{P}_{\sigma^*d,\sigma^*e}^{\mathfrak V} \end{align*} $$
$(\sigma ^*d)_{\alpha } = d_{\sigma (\alpha )}$ and
$(\sigma ^*e)_{\alpha } = e_{\sigma (\alpha )}$ .
-
(3) In the situation of Theorem 1.3.5 and Theorem 1.4.6,
$$ \begin{align*} \Lambda_p^{\mathfrak V} = \eta \circ \Lambda_p^G. \end{align*} $$
-
(4) In the situation of Definition 1.4.8, the maps
$$ \begin{align*} \mathrm{ev}_p^{\mathfrak V}&\mskip0.5mu\colon\thinspace \Gamma_c(U,\operatorname{\mathrm{Hom}}(E,F)) \to \bigoplus_{\alpha=1}^m \operatorname{\mathrm{Hom}}_{{\mathbf{K}}_{\alpha}}(\ker D_p^{{\underline V}_{\alpha}},\operatorname{\mathrm{coker}} D_p^{{\underline V}_{\alpha}}) \quad\text{and} \\ \mathrm{ev}_p^G &\mskip0.5mu\colon\thinspace \Gamma_c(U,\operatorname{\mathrm{Hom}}(E,F)) \to \operatorname{\mathrm{Hom}}_G(\ker \pi^*D_p,\operatorname{\mathrm{coker}} \pi^*D_p) \end{align*} $$
$$ \begin{align*} \mathrm{ev}_p^{\mathfrak V} = \eta \circ \mathrm{ev}_p^G. \end{align*} $$
Therefore,
$(D_p)_{p \in \mathscr {P}}$ is G-equivariantly flexible in U at
$p \in \mathscr {P}$ if and only if it is
${\mathfrak V}$ -equivariantly flexible in U at p.
-
(5) By Proposition 1.5.1(4), in the situation of Definition 1.4.9, the map
$\varpi _{D,U}^G$ satisfies
(1.5.8)$$ \begin{align} \varpi_{D,U}^G = \pi^* \circ \varpi_{D,U}^{\mathfrak V} \circ \tau. \end{align} $$
Therefore, D satisfies the G-equivariant Petri condition in U if and only if it satisfies the
${\mathfrak V}$ -equivariant Petri condition in U.
1.6 Petri’s condition revisited
While Petri’s condition typically is hard to verify for any particular elliptic operator, one can sometimes prove that it is satisfied for a generic element of a family of operators. Theorem 1.6.17 provides a useful tool for proving such statements. This result has been developed by Wendl [Reference WendlWen19b, Section 5.2] and was the essential innovation which allowed Wendl to prove the superrigidity conjecture.
Throughout this section, let
$x \in M$
and, furthermore, amend Definition 1.1.1 as follows.
Definition 1.6.1. Let
$k \in {\mathbf {N}}_0$
. A family of linear elliptic differential operators of order k with smooth coefficients is a family of linear elliptic differential operators
$(D_p)_{p \in \mathscr {P}}$
of the form

with
$a_{\ell }$
a smooth section of
$\mathrm {pr}_2^*\operatorname {\mathrm {Hom}}(T^*M^{\otimes \ell } \otimes E,F)$
over
$\mathscr {P} \times M$
(
$\ell = 0,\ldots ,k$
).
Let us begin by introducing the following algebraic variant of Petri’s condition.
Definition 1.6.2. Denote by
$\mathscr {E}$
the sheaf of sections of E and by
$\mathscr {E}_x$
its stalk at x; that is,

If
$s \in \mathscr {E}_x$
vanishes at x, then its derivative at x does not depend on the choice of a local trivialization and defines an element
$\mathrm { d}_xs \in \operatorname {\mathrm {Hom}}(T_xM,E_x)$
. If
$\mathrm {d}_xs = 0$
, then s has a second derivative
$\mathrm {d}_x^2s \in \operatorname {\mathrm {Hom}}(S^2T_xM,E_x)$
at x. Here,
$S^j T_x M$
is the j-th symmetric tensor power. In general, if
$s(x), \mathrm {d}_xs, \ldots , \mathrm {d}_x^{j-1}s$
vanish, then s is said to vanish to
$(j-1)^{\text {st}}$
order and its
$j^{\text {th}}$
derivative

is defined. The vanishing order filtration
$\mathscr {V}_{\bullet }\mathscr {E}_x$
on
$\mathscr {E}_x$
is defined by

for
$j \in {\mathbf {N}}_0$
and

for
$j \in {\mathbf {N}}$
. For
$\ell \in {\mathbf {N}}_0$
, the
$\ell $
-jet space of E at x is

The
$\infty $
-jet space of E at x is

For
$\ell \in {\mathbf {N}}_0\cup {\lbrace }{\infty }{\rbrace }$
, the
$\ell $
-jet of a linear differential operator
$D\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$
of order k with smooth coefficients is the linear map

induced by D.
Definition 1.6.3. The
$\infty $
-jet of a linear elliptic differential operator
$J_x^{\infty } D\mskip 0.5mu\colon \thinspace J_x^{\infty } E \to J_x^{\infty } F$
satisfies the
$\infty $
-jet Petri condition if the map

induced by the Petri map is injective.
The
$\infty $
-jet Petri condition and the equivariant Petri conditions are related by the following proposition.
Definition 1.6.4. Let
$x \in M$
. Let U be an open neighborhood of
$x \in M$
. A differential operator
$D \mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$
has the strong unique continuation property at x in U if the map

is injective.
Remark 1.6.5. If D has smooth coefficients, satisfies
$D^*D = \nabla ^*\nabla + \text {lower order terms}$
and U is connected, then it has the strong unique continuation property at
$x \in U$
[Reference CordesCor56; Reference AronszajnAro57]; see also [Reference Garofalo and LinGL87; Reference KazdanKaz88] for streamlined proofs using Almgren’s frequency function.
Proposition 1.6.6. Assume Situation 1.3.1 (or Situation 1.4.1). Let
$x \in M$
. Let
$U \subset M$
be an open neighborhood of x. Let
$D\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$
be a linear elliptic differential operator with smooth coefficients. Suppose that D and
$D^{\dagger }$
possess the strong unique continuation property at x in U. If
$J_x^{\infty } D$
satisfies the
$\infty $
-jet Petri condition, then D satisfies the
${\mathfrak V}$
-equivariant (or G-equivariant) Petri condition in U.
Proof. By Section 1.5, it suffices to consider Situation 1.3.1. Set

. Denote by
$\pi \mskip 0.5mu\colon \thinspace (\tilde M,\tilde x_0) \to (M,x_0)$
the universal covering map. Set

. The upcoming arguments prove that

is injective. Let
$\tilde x \in \pi ^{-1}(x)$
. Since
$\pi $
is a covering map,

Therefore, there is a commutative diagram

Since D and
$D^{\dagger }$
have the strong unique continuation property at
$x \in U$
,
$\pi ^*D$
and
$\pi ^*D^{\dagger }$
have the strong unique continuation property at
$\tilde x \in \tilde U$
. Consequently, the left vertical map is injective. Therefore, since
$\varpi _{J_x^{\infty } D}$
is injective, so is
$\varpi _{\pi ^*D,\tilde U}$
.
Every
$s \in \Gamma (E \otimes {\underline V}_{\alpha })$
can be regarded as an element
$\tilde s \in ({\Gamma (\pi ^*E) \otimes V_{\alpha }})^G$
. This establishes an isomorphism
$\ker D_p^{{\underline V}_{\alpha }} \cong (\ker \pi ^*D_p \otimes V_{\alpha })^G$
(and similarly for
$D^{\dagger }$
). Consider the commutative diagram

Here,
$\operatorname {\mathrm {tr}}$
is the sum of the maps induced by the trace maps