Equivariant Brill–Noether theory for elliptic operators and superrigidity of J-holomorphic maps

Remark 1.1.3. Let $(D_p)_{p \in \mathscr {P}}$ be a family of linear elliptic operators of index $i \in \mathbf {Z}$ . If $\mathscr {P}_{d,e} \neq \varnothing $ , then $d-e = i$ ; in particular: $d \geqslant i$ and $e \geqslant -i$ .

Remark 1.1.6. If E and F are Hermitian vector bundles and $(D_p)_{p \in \mathscr {P}}$ is a family of complex linear elliptic differential operators, then the map $\Lambda _p$ factors through $\operatorname {\mathrm {Hom}}_{\mathbf {C}}(\ker D_p,\operatorname {\mathrm {coker}} D_p)$ . Therefore, the hypothesis of Theorem 1.1.5 cannot be satisfied (unless it holds trivially). Of course, this issue is rectified by replacing $\mathbf {R}$ with ${\mathbf {C}}$ throughout the above discussion.

Remark 1.1.13. In Example 1.1.7, $\operatorname {\mathrm {im}} \Lambda _p = \operatorname {\mathrm {im}} \mathrm {ev}_p$ (with $U=\Sigma $ , and $\mathbf {R}$ replaced with ${\mathbf {C}}$ ). Therefore, $\Lambda _p$ being surjective is equivalent to Petri’s condition. Furthermore, tracing through the isomorphisms identifies the restriction of Petri map $\varpi $ to $\ker \bar {\partial }_A\otimes _{\mathbf {C}} \ker \bar {\partial }_A^*$ with the Petri map $\varpi _{\mathscr {L}}$ .

Remark 1.1.14. Assume the situation of Theorem 1.1.5. The following observations are useful in situations where the primary objective is to estimate the codimension of $\mathscr {P}_{d,e}$ .

1. (1) Every $p \in \mathscr {P}_{d,e}$ has an open neighborhood $\mathscr {U}$ in $\mathscr {P}$ such that $\mathscr {P}_{d,e}\cap \mathscr {U}$ is contained in a submanifold of codimension $\operatorname {\mathrm {rk}} \Lambda _p$ .

2. (2) Let $\rho \in {\mathbf {N}}$ , and let $U \subset M$ be an open subset. A linear elliptic differential operator $D\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ satisfies Petri’s condition up to rank $\rho $ in U if for every nonzero $B \in \ker D\otimes \ker D^{\dagger }$ of rank at most $\rho $ the section $\varpi (B)$ does not vanish on U. (A simple tensor is nonzero tensor of the form $v\otimes w$ . Every tensor B is a sum of simple tensors. The rank of B is the minimal number of simple tensors that sum to B.) If $D_{p_{\star }}$ satisfies this condition and $(D_p)_{p\in \mathscr {P}}$ is flexible in U at $p_{\star } \in \mathscr {P}$ , then

   $$ \begin{align*} \operatorname{\mathrm{rk}} \Lambda_{p_{\star}} \geqslant \min{\lbrace}{\rho,d,e}{\rbrace}\max{\lbrace}{d,e}{\rbrace}.\\[-30pt] \end{align*} $$

   Proof. Set . If $d \leqslant e$ , then choose an injection $\mathbf {R}^{\sigma } \hookrightarrow \ker D_{p_{\star }}$ and set ; otherwise, choose a surjection $\operatorname {\mathrm {coker}} D_{p_{\star }} \twoheadrightarrow \mathbf {R}^{\sigma }$ and set . In either case, composition defines a surjection

   $$ \begin{align*} \pi_{p_{\star}}\mskip0.5mu\colon\thinspace \operatorname{\mathrm{Hom}}(\ker D_{p_{\star}},\operatorname{\mathrm{coker}} D_{p_{\star}}) \to H. \end{align*} $$

   The subspace $\operatorname {\mathrm {im}} \pi _{p_{\star }}^* \subset \operatorname {\mathrm {Hom}}(\ker D_{p_{\star }},\operatorname {\mathrm {coker}} D_{p_{\star }})^* \cong \ker D_{p_{\star }} \otimes \ker D_{p_{\star }}^{\dagger }$ consists of elements of rank at most $\sigma \leqslant \rho $ . The argument of the proof of Proposition 1.1.12 thus shows that $\pi _{p_{\star }} \circ \Lambda _{p_{\star }}$ is surjective. Therefore, $\operatorname {\mathrm {rk}} \Lambda _{p_{\star }} \geqslant \dim H = \min {\lbrace }{\rho ,d,e}{\rbrace }\max {\lbrace }{d,e}{\rbrace }$ .

3. (3) (This is due to Eftekhary [Reference EftekharyEft16, Proof of Lemma 4.4].) Let $U \subset M$ be a nonempty open subset. Let $D \mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ be a first linear elliptic differential operator of first order. If $\ker D$ and $\ker D^{\dagger }$ consists of continuous sections, and D and $D^{\dagger }$ have the weak unique continuation property, then D satisfies Petri’s condition up to rank three in U.Footnote ⁵ ,Footnote ⁶

   Proof. Every $B \in \ker D\otimes \ker D^{\dagger }$ can be written as $B = s_1\otimes t_1 + \cdots + s_{\rho }\otimes t_{\rho }$ with , and $s_1,\ldots ,s_{\rho } \in \ker D$ and $t_1,\ldots ,t_{\rho } \in \ker D^{\dagger }$ linearly independent. If $\rho = 1$ and $\varpi (B) = 0$ , then $s_1$ or $t_1$ vanishes on an open subset; hence, by unique continuation, $s_1 = 0$ or $t_1 = 0$ : a contradiction.

   Henceforth, assume that $\rho \geqslant 2$ . Define $\delta ,\varepsilon \mskip 0.5mu\colon \thinspace U \to {\mathbf {N}}_0$ by and . By unique continuation, $\delta $ and $\varepsilon $ are positive on a dense open subset. In fact, $\delta ,\varepsilon \geqslant 2$ on a dense open subset. To see this, observe that if $\delta = 1$ on a nonempty open subset, then there is a nonempty open subset $V \subset U$ and a function $f \in C^{\infty }(V)$ such that $s_1(x) = f(x)s_2(x)$ for every $x \in V$ . Therefore, $\sigma (\mathrm {d} f)s_2 = 0$ with $\sigma $ denoting the symbol of D. Since D is elliptic, f must be constant: a contradiction to $s_1$ and $s_2$ being linearly independent. The same argument applies to $\varepsilon $ .

   If $\rho = 2$ , then there exists an $x \in U$ such that $\delta (x) = \varepsilon (x) = 2$ ; therefore: $\varpi (B)$ does not vanish at x. If $\rho = 3$ , then there is an $x \in U$ such that $\min {\lbrace }{\delta (x),\varepsilon (x)}{\rbrace } \geqslant 2$ . If $\delta (x) = \varepsilon (x) = 3$ , then $\varpi (B)$ evidently does not vanishing at x; otherwise, without loss of generality, $s_1(x)$ and $s_2(x)$ are linearly independent, and $s_3(x) = \lambda _1s_1(x) + \lambda _2s_2(x)$ . In the latter case,

   $$ \begin{align*} \varpi(B)(x) = s_1(x) \otimes ({t_1(x)+\lambda_1t_3(x)}) + s_2(x)\otimes ({t_2(x) + \lambda_2t_3(x)}) \end{align*} $$

   which cannot vanish because $\varepsilon (x) \geqslant 2$ .

   There are examples of first-order linear elliptic operators which fail to satisfy Petri’s condition up to rank four; see [Reference WendlWen19b, Example 5.5] or Proposition 2.5.4. Finally, a brief warning: The preceding observation is false when $\mathbf {R}$ is replaced with ${\mathbf {C}}$ or $\mathbf {H}$ . The analogue of Petri’s condition only holds up to rank one in this case. (The issue is that $\sigma ({\mathrm {d} f})s_2 =0$ does not imply $\mathrm {d} f = 0$ if f takes values in ${\mathbf {C}}$ or $\mathbf {H}$ .)

Remark 1.2.2. If $\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$ is a branched covering map of manifolds whose ramification locus is a closed submanifold of codimension two, then $\tilde M$ and M can be equipped with orbifold structures and $\pi $ can be lifted to an unbranched covering map of orbifolds. Section 2.7 discusses this construction in the case of Riemann surfaces; the higher-dimensional case follows immediately from the two-dimensional case and the above local model.

Remark 1.2.4. Let $x_0 \in M$ . Denote by $\pi _1(M,x_0)$ the fundamental group with base-point $x_0$ . If $*$ denotes the usual concatenation of paths, then the multiplication in $\pi _1(M,x_0)$ is defined by .Footnote ⁸ Parallel transport induces a monodromy representation

$$ \begin{align*} \mu\mskip0.5mu\colon\thinspace \pi_1(M,x_0) \to \mathrm{O}(V) \end{align*} $$

with V denoting the fiber of ${\underline V}$ over $x_0$ . ${\underline V}$ can be recovered from $\mu $ as follows. Denote by $\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$ the universal covering map and by $\operatorname {\mathrm {Aut}}(\pi )$ the group of deck transformations. A choice of $\tilde x_0 \in \pi ^{-1}(x_0)$ induces an anti-isomorphism from $\operatorname {\mathrm {Aut}}(\pi )$ to $\pi _1(X,x_0)$ . Therefore, $\tilde M$ is a principal $\pi _1(M,x_0)$ -bundle, and ${\underline V}$ is the associated bundle:

$$ \begin{align*} {\underline V} \cong \tilde M \times_{\mu} V. \end{align*} $$

This sets up a bijection between gauge equivalence classes $[{\underline V}]$ of Euclidean local systems of rank r and equivalence classes $[\mu ]$ of representations $\pi _1(M,x_0) \to \mathrm {O}(r)$ up to conjugation by $\mathrm {O}(r)$ . For a more detailed discussion—in particular, of how the to interpret the above in the category of orbifolds—we refer the reader to [Reference Shen and YuSY19, Sections 2.4 and 2.5].

Remark 1.2.7. The following facts about the construction from Definition 1.2.6 are important.

1. (1) If E is a Euclidean vector bundle over $\tilde M$ , then the sheaf $\Gamma (\cdot ,\pi _*E)$ is the sheaf-theoretic pushforward of the sheaf $\Gamma (\cdot ,E)$ ; that is: There are canonical isomorphisms

   $$ \begin{align*} \Gamma(U,\pi_*E) \cong \Gamma(\pi^{-1}(U),E) \end{align*} $$
   for every open set $U \subset M$ and these are compatible with the restriction maps.

2. (2) If E is a Euclidean local system on $\tilde M$ , then $\pi _*E$ is a Euclidean local system on M: $s \in \Gamma (U,\pi _*E)$ is covariantly constant if and only if $\tilde s \in \Gamma (\pi ^{-1}(U),E)$ is.

3. (3) Let E and F be a Euclidean vector bundles over M and $\tilde M$ respectively. For every $x \in M$ , there is a conical isomorphism

   $$ \begin{align*} \pi_*(\pi^* E\otimes F)_x \cong \bigoplus_{\tilde x \in \pi^{-1}(x)} E_x \otimes F_{\tilde x} \cong (E\otimes \pi_*F)_x. \end{align*} $$

   These assemble into the push-pull formula

   $$ \begin{align*} \pi_*(\pi^* E\otimes F) \cong E \otimes \pi_*F. \end{align*} $$

   In particular,

   $$ \begin{align*} \pi_*(\pi^*E) \cong E \otimes \pi_*{\underline{\mathbf{R}}}. \end{align*} $$

   Here, ${\underline {\mathbf {R}}}$ denotes the trivial rank one Euclidean local system on $\tilde M$ .

Remark 1.2.10. If $\pi $ is a normal covering, then $C=N$ and $G = \pi _1(M,x_0)/N$ is isomorphic to its deck transformation group. If $\pi $ has k sheets, then C has index k. Its normal core has index at most $k!$ by an elementary result known as Poincaré’s theorem. This theorem follows from the observation that the kernel of the canonical homomorphism $\pi _1(M,x_0) \to \operatorname {\mathrm {Bij}}(G/C)$ is precisely N and $\operatorname {\mathrm {Bij}}(G/C) \cong S_k$ . Here, $\operatorname {\mathrm {Bij}}(G/C)$ denotes the set of bijections of $G/C$ ,

Remark 1.3.2. Since ${\underline V}_{\alpha }$ is irreducible, ${\mathbf {K}}_{\alpha }$ is a division algebra; hence, by Frobenius’ Theorem it is isomorphic to either $\mathbf {R}$ , ${\mathbf {C}}$ , or $\mathbf {H}$ and $k_{\alpha } \in {\lbrace }{1,2,4}{\rbrace }$ .

Remark 1.3.4. Let $i \in \mathbf {Z}^m$ . Let $(D_p)_{p \in \mathscr {P}}$ be a family of linear elliptic operators such that $\operatorname {\mathrm {index}}_{{\mathbf {K}}_{\alpha }} D_p^{{\underline V}_{\alpha }} = i_{\alpha }$ for every $p \in \mathscr {P}$ and $\alpha = 1,\ldots ,m$ . If $\mathscr {P}_{d,e}^{\mathfrak V} \neq \varnothing $ , then $d-e = i$ ; in particular: $d_{\alpha } \geqslant i_{\alpha }$ and $e_{\alpha } \geqslant -i_{\alpha }$ .

If M is a manifold, then

$$ \begin{align*} \operatorname{\mathrm{index}}_{{\mathbf{K}}_{\alpha}} D_p^{{\underline V}_{\alpha}} = \operatorname{\mathrm{rk}}_{{\mathbf{K}}_{\alpha}} {\underline V}_{\alpha} \cdot \operatorname{\mathrm{index}} D_p \end{align*} $$

by the Atiyah–Singer index theorem; therefore, the $i_{\alpha }$ all have the same sign. If M is an orbifold, there are corrections terms in the index formula which spoil this relation between the indices; see, for example, Proposition 2.8.6.

Remark 1.3.6. If E and F are Hermitian vector bundles and $(D_p)_{p \in \mathscr {P}}$ is a family of complex linear elliptic differential operators, then Theorem 1.3.5 does not apply; cf. Remark 1.1.6. Again, this issue is rectified by replacing $\mathbf {R}$ with ${\mathbf {C}}$ throughout. In fact, this somewhat simplifies the discussion since ${\mathbf {C}}$ is the unique complex division algebra; hence, there is no need to introduce ${\mathbf {K}}_{\alpha }$ .

Remark 1.3.7 (Twists by arbitrary Euclidean local systems).

Every Euclidean local system ${\underline V}$ decomposes into irreducible local systems

$$ \begin{align*} {\underline V} \cong \bigoplus_{\alpha=1}^m {\underline V}_{\alpha}^{\oplus \ell_{\alpha}} \end{align*} $$

with $\ell _1,\ldots ,\ell _m \in {\mathbf {N}}_0$ for a suitable choice of ${\mathfrak V}$ . For every $\bar d,\bar e \in {\mathbf {N}}_0$ the Brill–Noether locus

is the finite disjoint union of the subsets $\mathscr {P}_{d,e}^{\mathfrak V}$ with $(d,e) \in {\mathbf {N}}_0^m\times {\mathbf {N}}_0^m$ satisfying

$$ \begin{align*} \sum_{\alpha=1}^m \ell_{\alpha} k_{\alpha} d_{\alpha} = \bar d \quad\text{and}\quad \sum_{\alpha=1}^m \ell_{\alpha} k_{\alpha} e_{\alpha} = \bar e. \end{align*} $$

Through this observation Theorem 1.3.5 can be brought to bear on families of linear elliptic differential operators twisted by ${\underline V}$ .

Remark 1.3.10. The ${\mathfrak V}$ -equivariant Petri condition appears even more difficult to verify than the Petri condition. It turns out, however, that there is a general method for verifying both of these conditions simultaneously. This will be discussed in Section 1.6.

Remark 1.3.12. There are analogues of the observations from Remark 1.1.14 in the equivariant setting.

1. (1) Every $p \in \mathscr {P}_{d,e}^{\mathfrak V}$ has an open neighborhood $\mathscr {U}$ in $\mathscr {P}$ such that $\mathscr {P}_{d,e}^{\mathfrak V}\cap \mathscr {U}$ is contained in a submanifold of codimension $\operatorname {\mathrm {rk}} \Lambda _p^{\mathfrak V}$ .

2. (2) Let $\rho \in {\mathbf {N}}_0^m$ , and let $U \subset M$ be an open subset. A linear elliptic differential operator $D\mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$ satisfies the ${\mathfrak V}$ -equivariant Petri condition up to rank $\rho $ in U if for every nonzero $B = (B_1,\ldots ,B_m) \in \bigoplus _{\alpha =1}^m \ker D_p^{{\underline V}_{\alpha }} \otimes _{\,{\mathbf {K}}_{\alpha }^{\mathrm {op}}} \ker D_p^{\dagger ,{\underline V}_{\alpha }^*}$ with $\operatorname {\mathrm {rk}} B_{\alpha } \leqslant \rho _{\alpha }$ for $\alpha = 1,\ldots ,m$ the section $\varpi ^{\mathfrak V}(B)$ does not vanish on U. If $D_p$ satisfies this condition and $(D_p)_{p\in \mathscr {P}}$ is ${\mathfrak V}$ -equivariantly flexible in U at $p_{\star } \in \mathscr {P}$ , then

   $$ \begin{align*} \operatorname{\mathrm{rk}} \Lambda_{p_{\star}} \geqslant \sum_{\alpha=1}^m \min{\lbrace}{\rho_{\alpha},d_{\alpha},e_{\alpha}}{\rbrace}\max{\lbrace}{d_{\alpha},e_{\alpha}}{\rbrace}. \end{align*} $$

3. (3) Let $\rho \in {\mathbf {N}}_0^m$ , and let $U \subset M$ be an open subset. Every first-order linear elliptic differential operator $D \mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ satisfies the ${\mathfrak V}$ -equivariant Petri condition up to rank $\rho $ on U provided

   (1.3.13) $$ \begin{align} \sum_{\alpha=1}^m \operatorname{\mathrm{rk}}_{\mathbf{R}} V_{\alpha} \cdot \rho_{\alpha} \leqslant 3. \end{align} $$

   Proof. Set . Denote by $\pi \mskip 0.5mu\colon \thinspace (\tilde M,\tilde x_0) \to (M,x_0)$ the universal covering map. Every $s \in \ker D^{{\underline V}_{\alpha }}$ can be regarded as an element $\tilde s \in \Gamma (\pi ^*E\otimes V_{\alpha })^G$ in the space of G-invariant sections, with $G \to \mathrm {O}(V_{\alpha })$ denoting the monodromy representation of ${\underline V}_{\alpha }$ . This section can be regarded as sections $s_1,\ldots ,s_{r_{\alpha }}$ of $\pi ^*E$ . For every $\alpha =1,\ldots ,m$ let $s_1^{\alpha },\ldots ,s_{q_{\alpha }}^{\alpha } \in \ker D^{{\underline V}_{\alpha }}$ be linearly independent over ${\mathbf {K}}_{\alpha }$ . The resulting collection of sections $s_{j,k}^{\alpha } \in \ker \pi ^*D$ ( $\alpha = 1,\ldots ,m, j=1,\ldots ,q_{\alpha }, k=1,\ldots ,r_{\alpha }$ ) are linearly independent. The latter is a consequence of Proposition 1.3.14 applied to $W = \ker \pi ^*D$ . (Analogous statements hold for $D^{\dagger }$ instead of D.) At this point, one can apply the argument in Remark 1.1.14(3).

   Unfortunately, this is not as useful as Remark 1.1.14(3) because equation (1.3.13) is very restrictive; however, it is what lies at the heart of Eftekhary’s proof of the $4$ -rigidity conjecture [Reference EftekharyEft16].

Remark 1.4.4. Let $D\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ is a linear elliptic differential operator. The G-equivariant index of $\pi ^*D$ is . Here, $R(G)$ denotes the representation ring of G; its elements are formal differences of isomorphism classes of representations of G. It is a consequence of the above discussion that $R(G) \cong \mathbf {Z}^m$ as abelian groups.

For families of linear elliptic operators with G-equivariant index corresponding to $i \in \mathbf {Z}^m$ , what was said in Remark 1.3.4 applies in the present situation as well.

Remark 1.4.7 (Pullbacks by arbitrary covering maps).

Suppose that $\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$ is a finite covering map with characteristic subgroup $C < \pi _1(M,x_0)$ . Denote by N the normal core of C, denote by $\rho \mskip 0.5mu\colon \thinspace (\hat M,\hat x_0) \to (M,x_0)$ the pointed covering map with characteristic subgroup N and set . For the decomposition (1.4.2) of $\operatorname {\mathrm {Map}}(S,\mathbf {R})$ is

$$ \begin{align*} \operatorname{\mathrm{Map}}(S,\mathbf{R}) \cong \bigoplus_{\alpha=1}^m (V_{\alpha}^*)^C\otimes_{\,{\mathbf{K}}_{\alpha}} V_{\alpha}; \end{align*} $$

indeed: The map $\mathrm {ev}_{[\mathbf {1}]}\mskip 0.5mu\colon \thinspace \operatorname {\mathrm {Hom}}_G(V_{\alpha },\operatorname {\mathrm {Map}}(S,\mathbf {R})) \to V_{\alpha }^*$ defined by is injective and its image is $(V_{\alpha }^*)^C$ . Therefore, by Proposition 1.2.9(2)

$$ \begin{align*} \ker \pi^*D &\cong \bigoplus_{\alpha=1}^m V_{\alpha}^C\otimes_{\,{\mathbf{K}}_{\alpha}^{\mathrm{op}}} \operatorname{\mathrm{Hom}}_G(V_{\alpha},\ker \rho^*D) \quad\text{and} \\ \operatorname{\mathrm{coker}} \pi^*D &\cong \bigoplus_{\alpha=1}^m V_{\alpha}^C\otimes_{\,{\mathbf{K}}_{\alpha}^{\mathrm{op}}} \operatorname{\mathrm{Hom}}_G(V_{\alpha},\operatorname{\mathrm{coker}} \rho^*D). \end{align*} $$

With the above in mind, Theorem 1.4.6 can be brought to bear on nonnormal covering maps; cf. Remark 1.3.7.

Remark 1.4.10. Remark 1.3.10 applies to the the G-equivariant Petri condition as well.

Remark 1.6.5. If D has smooth coefficients, satisfies $D^*D = \nabla ^*\nabla + \text {lower order terms}$ and U is connected, then it has the strong unique continuation property at $x \in U$ [Reference CordesCor56; Reference AronszajnAro57]; see also [Reference Garofalo and LinGL87; Reference KazdanKaz88] for streamlined proofs using Almgren’s frequency function.