Hostname: page-component-78c5997874-ndw9j Total loading time: 0 Render date: 2024-11-18T01:23:01.997Z Has data issue: false hasContentIssue false

On a critical time-harmonic Maxwell equation in nonlocal media

Published online by Cambridge University Press:  29 February 2024

Minbo Yang
Affiliation:
Department of Mathematics, Zhejiang Normal University, Jinhua, Zhejiang 321004, People's Republic of China (mbyang@zjnu.edu.cn)
Weiwei Ye
Affiliation:
Department of Mathematics, Zhejiang Normal University, Jinhua, Zhejiang 321004, People's Republic of China Department of Mathematics, Fuyang Normal University, Fuyang, Anhui 236037, People's Republic of China (yeweiweime@163.com)
Shuijin Zhang
Affiliation:
Department of Mathematics, Zhejiang Normal University, Jinhua, Zhejiang 321004, People's Republic of China (shuijinzhang@zjnu.edu.cn)
Rights & Permissions [Opens in a new window]

Abstract

In this paper, we study the existence of solutions for a critical time–harmonic Maxwell equation in nonlocal media

\[ \begin{cases} \nabla\times(\nabla\times u)+\lambda u=\left(I_{\alpha}\ast|u|^{2^{{\ast}}_{\alpha}}\right)|u|^{2^{{\ast}}_{\alpha}-2}u & \mathrm{in}\ \Omega,\\ \nu\times u=0 & \mathrm{on}\ \partial\Omega, \end{cases} \]
where $\Omega \subset \mathbb {R}^{3}$ is a bounded domain, either convex or with $\mathcal {C}^{1,1}$ boundary, $\nu$ is the exterior normal, $\lambda <0$ is a real parameter, $2^{\ast }_{\alpha }=3+\alpha$ with $0<\alpha <3$ is the upper critical exponent due to the Hardy–Littlewood–Sobolev inequality. By introducing some suitable Coulomb spaces involving curl operator $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )$, we are able to obtain the ground state solutions of the curl–curl equation via the method of constraining Nehari–Pankov manifold. Correspondingly, some sharp constants of the Sobolev-like inequalities with curl operator are obtained by a nonlocal version of the concentration–compactness principle.

Type
Research Article
Copyright
Copyright © The Author(s), 2024. Published by Cambridge University Press on behalf of The Royal Society of Edinburgh

1. Introduction and main results

1.1. Introduction

Let $\Omega \subset \mathbb {R}^{3}$ be a bounded domain, we are concerned with the curl–curl equation

(1.1)\begin{equation} \begin{cases} \nabla\times(\nabla\times u)+\lambda u=f(x,u) & \mathrm{in}\ \Omega,\\ \nu\times u=0 & \mathrm{on}\ \partial\Omega, \end{cases} \end{equation}

where $\lambda <0$ is a real parameter, $\nu :\partial \Omega \longrightarrow \mathbb {R}^{3}$ is the exterior normal. Equation (1.1) can be derived from the first order Maxwell equation [Reference Monk35]

(1.2)\begin{equation} \begin{cases} \nabla\times \mathcal{H}=\mathcal{J}+\partial_{t}\mathcal{D}, & (\mathrm{Ampere's\ circuital\ law})\\ div(\mathcal{D})=\varrho, & (\mathrm{Gauss's\ law})\\ \partial_{t}\mathcal{B}+\nabla\times \mathcal{E}=0, & (\mathrm{Faraday's\ law\ of\ induction})\\ div(\mathcal{B})=0, & (\mathrm{Gauss's\ law\ for\ magnetism}) \end{cases} \end{equation}

where $\mathcal {E},\mathcal {H},\mathcal {D},\mathcal {B}$ are corresponded to the electric field, magnetic induction, electric displacement and magnetic filed, respectively. $\mathcal {J}$ is the electric current intensity, and $\varrho$ is the electric charge density. Generally, these physical quantities satisfy the following constitutive equations (see [Reference Dörfler, Lechleiter, Plum, Schneider and Wieners14, § 1.1.3]):

(1.3)\begin{equation} \mathcal{\mathcal{J}}=\sigma \mathcal{E},~\mathcal{D}=\varepsilon \mathcal{E}+\mathcal{P}_{NL}, \mathcal{H}=\frac{1}{\mu}\mathcal{B}-\mathcal{M}, \end{equation}

where $\mathcal {P}_{NL}, \mathcal {M}$ denote the polarization field and magnetization filed respectively, $\varepsilon,\mu,\sigma$ are the electric permittivity, magnetic permeability and the electric conductivity . Taking the special case with the absence of charges, currents and magnetization, namely, $\mathcal {J}=\mathcal {M}=0$, $\rho =0$, equation (1.2) becomes the second curl-curl equation

(1.4)\begin{equation} \nabla\times(\frac{1}{\mu}\nabla\times\mathcal{E})+\varepsilon\partial_{t}^{2}\mathcal{E}={-}\partial_{t}^{2}\mathcal{P}_{NL}. \end{equation}

As the electric field and polarization field are time harmonic with the ansatz $\mathcal {E}(x,t)=E(x)\,{\rm e}^{i\omega t}$, $\mathcal {P}_{NL}(x,t)=P(x)\,{\rm e}^{i\omega t}$, equation (1.4) turns into the time-harmonic Maxwell equation

\[ \nabla\times(\frac{1}{\mu}\nabla\times E)-\varepsilon \omega^{2}E=\omega^{2} P. \]

In some Kerr-like medias, the polarization field function $\mathcal {P}_{NL}$ is usually chosen to be $\mathcal {P}_{NL}=\alpha (x)|\mathcal {E}|^{p-2}\mathcal {E}$ with $2\leq p\leq 6$ for the purpose of simplifying the model. Then by setting

\[ f(x,E)=\partial_{E}F(x,E)=\mu\omega^{2}\alpha(x)|E|^{p-2}E, \]

one can deduce the main equation (1.1)

\[ \nabla\times(\nabla\times E)+\lambda E=f(x,E), \]

where $\lambda =-\mu \omega ^{2}\varepsilon$. The boundary condition holds when $\Omega$ is surrounded by a perfect conductor, see [Reference Dörfler, Lechleiter, Plum, Schneider and Wieners14].

Apparently, equation (1.1) has a variational structure and the solutions are the critical points of the functional

(1.5)\begin{equation} J_{\lambda}(u)=\int_{\Omega}|\nabla\times u|^{2}\,{\rm d}x+\frac{\lambda}{2}\int_{\Omega}|u|^{2}\,{\rm d}x-\int_{\Omega}F(x,u)\,{\rm d}x, \end{equation}

which is well defined on the natural space

\[ X=W^{p}_{0}(\mathrm{curl};\Omega)=\overline{\mathcal{C}_{0}^{\infty}(\Omega,\mathbb{R}^{3})}^{||\cdot||_{W^{p}(\mathrm{curl};\Omega)}}, \]

where

\[ W^{p}(\mathrm{curl};\Omega)=\{u\in L^{p}(\Omega,\mathbb{R}^{3}):\nabla\times u\in L^{2}(\Omega,\mathbb{R}^{3})\} \]

is a Banach space, see [Reference Bartsch and Mederski5]. By introducing the Helmholtz decomposition

\[ W^{p}_{0}(\mathrm{curl};\Omega)=X_{\Omega}\oplus X^c_{\Omega}, \]

where

\begin{align*} X_{\Omega}:& =\{v\in W_{0}^{p}(\mathrm{curl};\Omega):\int_{\Omega}\langle v,\varphi\rangle\,{\rm d}x=0\ \mathrm{for\ every}\ \varphi\\ & \quad \in C^{\infty}_{0}(\Omega,\mathbb{R}^{3})\ \mathrm{with}\ \nabla\times\varphi=0\}\\ & =\{v\in W_{0}^{p}(\mathrm{curl};\Omega):\mathrm{div}(v)=0\ \mathrm{in\ the\ sense\ of\ distributions}\}, \end{align*}

and

\[ X^c_{\Omega}:=\{w\in W^{p}_{0}(\mathrm{curl};\Omega):\int_{\Omega}\langle w,\nabla\times\varphi\rangle\,{\rm d}x=0\ \mathrm{for all}\ \varphi\in C^{\infty}_{0}(\Omega,\mathbb{R}^{3})\}, \]

functional (1.5) can be rewritten as

\begin{align*} J_{\lambda}(u)=J_{\lambda}(v+w)& =\frac{1}{2}\int_{\Omega}|\nabla\times v|^{2}\,{\rm d}x+\frac{\lambda}{2}\int_{ \Omega}|v+w|^{2}\,{\rm d}x-\int_{\Omega}F(x,v+w)\,{\rm d}x\\ & =\frac{1}{2}\int_{\Omega}|\nabla v|^{2}\,{\rm d}x+\frac{\lambda}{2}\int_{ \Omega}|v+w|^{2}\,{\rm d}x-\int_{\Omega}F(x,v+w)\,{\rm d}x, \end{align*}

where $\nabla \times (\nabla \times v)=\nabla (\nabla \cdot v)-\nabla \cdot (\nabla v)=-\Delta v$ for $\mathrm {div}(v)=0$. Since the operator $\nabla \times (\nabla \times \cdot )$ has an infinite dimension kernel, i.e. $\nabla \times (\nabla \varphi )=0$ for $\varphi \in \mathcal {C}^{\infty }_{0}(\Omega )$, one can easily check that $J_{\lambda }$ has the strongly indefinite nature. Particularly, set

\[ \widetilde{\lambda}=\mu(\Omega)^{-\frac{p-2}{p}}p^{-\frac{2}{p}}\mathop{\mathrm{inf}}\limits_{v\in X_{\Omega}:|v|^{p}=1}\int_{\Omega}|\nabla\times v|^{2}\,{\rm d}x>0, \]

then $J_{\lambda }$ has a linking geometry as $\lambda \leq \widetilde {\lambda }$, see [Reference Mederski31].

To overcome the difficulty of the strong indefiniteness, by assuming that the additional condition $\nabla \cdot u=0$, then the curl–curl operator become the classical Laplace operator. Moreover, if consider the Dirichlet boundary condition, it becomes the classical elliptic equation

(1.6)\begin{equation} \begin{cases} -\Delta u+\lambda u=f(x,u) & \mathrm{in}\ \Omega,\\ u=0 & \mathrm{on}\ \partial\Omega. \end{cases} \end{equation}

This elliptic equation has been widely studied in different dimensional spaces and topology regions, see the pioneering work of Brezis and Nirenberg [Reference Brezis and Nirenberg10] and the more references [Reference Capozzi, Fortunato and Palmieri11, Reference Cerami, Solimini and Struwe12, Reference Gao and Yang19]. Essentially, the non-divergence condition is a Coulomb gauge condition, which holds in the gauge invariant field. This requires that the polarization field $\mathcal {P}_{NL}$ does not happen or linearly depends on $\mathcal {E}$, otherwise, it destroys the gauge invariance of the curl–curl equation. If the polarization field $\mathcal {P}_{NL}=0$, then the curl–curl equation becomes a linear time harmonic Maxwell equation, which has been extensively considered in [Reference Dörfler, Lechleiter, Plum, Schneider and Wieners14, Reference Monk35, Reference Picard, Weck and Witsch39]. Physically there indeed exist some special cylindrically symmetric transverse electric and transverse magnetic which satisfy the non-divergence condition, and they have been studied by Stuart and Zhou in [Reference Stuart and Zhou43, Reference Stuart and Zhou44].

For the general case with $\nabla \cdot u\neq 0$, the study of the curl–curl equation becomes much more challenging. The first attempt goes back to the pioneering work of Benci [Reference Benci and Fortunato7]. Under some nonlinear assumptions on $W(t)$, the authors investigated the Born–Infeld static magnetic model

(1.7)\begin{equation} \nabla\times(\nabla\times A)=W'(|A|^{2})A,\quad\mathrm{in}\ \mathbb{R}^{3}, \end{equation}

where $A=\nabla \times B$ is a magnetic potential. In a suitable subspace, Azzollini et al. [Reference Azzollini, Benci, D'Aprile and Fortunato2] obtained the cylindrically symmetric solutions of (1.7) by the Palais principle of symmetric criticality. By using the Hodge decomposition, the cylindrically symmetric solutions with a second form have also been constructed by D'Aprile and Siciliano in [Reference D'Aprile and Siciliano13]. In fact, in some bounded domains with cylindrically symmetric, the similar solutions were obtained in [Reference Bartsch and Mederski5, Reference Bartsch and Mederski6, Reference Mederski30]. Bartsch, Dohnal and et al. [Reference Bartsch, Dohnal, Plum and Reichel4] also analysed the spectrum of the curl–curl operator with cylindrically symmetric periodic potential $V(x)=V(r,x_{3})$, and considered the following time-harmonic Maxwell equation

(1.8)\begin{equation} \nabla\times(\nabla\times E)+V(x)E=\Gamma(x)|E|^{p-2}E,\quad \mathrm{in}\ \mathbb{R}^{3}, \end{equation}

where $\Gamma (x)$ is a period function with respect $x_{3}$. By the method of constraining symmetric sub-manifold, the cylindrically symmetric ground state solutions of (1.8) were obtained, one may see [Reference Zeng46] for other extended results.

If the problem was set in some non-symmetric bounded domains or some cases with non-symmetric potential, the methods mentioned above do not work well. Moreover, due to the lack of weak–weak$^{\ast }$ continuity of $J^{'}_{\lambda }(u)$ , the abstract linking theorems established in [Reference Benci and Rabinowitz8, Reference Guo and Mederski20, Reference Mederski29] do not work any longer, and so we fail to look for the suitable $(PS)$ sequences. Even if we can obtain the bounded $(PS)$ sequence, we still do not know whether the weak limit is a critical point of the functional. Inspired by the work of Szulkin and Weth in [Reference Szulkin and Weth45], Bartsch and Mederski[Reference Bartsch and Mederski5] constructed a Nehari–Pankov manifold, which is homeomorphism with the upper unit ball of the subspace $X_{\Omega }$, and in where, the $(PS)$ sequence is obtained by the Ekeland variational principle. On the other hand, by the compact embedding

(1.9)\begin{equation} X_{\Omega}\hookrightarrow L^{p}(\Omega,\mathbb{R}^{3}),\quad 2\leq p<6, \end{equation}

they succeeded in verifying the $(PS)_{c}^{\tau }$ condition, see definition 2.20, which implies the weak-weak$^{\ast }$ continuity of $J^{'}_{\lambda }(u)$. We would also like to mention that the convexity assumption of the nonlinearity $f(x,u)$ plays a key role in finding the bounded $(PS)$ sequence, see also [Reference Bartsch and Mederski6] for the weakened version. For other related results, we may turn to [Reference Qin and Tang40] for the asymptotically linear case and [Reference Mederski, Schino and Szulkin32] for the case with supercritical growth at $0$ and subcritical growth at infinity.

For the critical case $p=6$, the embedding (1.9) above is not compact any more, then it is rather difficult to verify the $(\mathrm {PS})^{\tau }_{c}$ condition. Mederski [Reference Mederski30] proposed a compactly perturbed method and proved that the $(\mathrm {PS})$ sequence contains a weakly convergent subsequence with a nontrivial limit point. Later, Mederski and Szulkin [Reference Mederski and Szulkin33] established a general concentration-compactness lemma in $\mathbb {R}^{N}$ and obtained the sharp constant in the curl inequality. As an application, the authors dealt with the Brezis-Nirenberg type problem by an extend skill. In the entire space $\mathbb {R}^{3}$, the embedding above is also not compact, then a new critical point theory related to a new topological manifold has been established by Mederski et al. in [Reference Mederski, Schino and Szulkin32], there the compactness was recovered and the existence of multiply solutions was obtained. In a direct way, Mederski [Reference Mederski29] established a global compactness lemma that accounts for the lack of weak-weak$^{\ast }$ continuity. Moreover, a Pohozaev identity has been established, which gives a criterion for the nonexistence of classical solution. In an earlier work, Bartsch [Reference Bartsch, Dohnal, Plum and Reichel4] showed that no interesting solution can be leaded under the fully radial symmetry assumption on the potential $V(x)$.

However, for some Kerr-type nonlinear mediums, the material law (1.3) between the electric field $\mathcal {E}$ and the displacement field $\mathcal {D}$ becomes more delicate, see [Reference Bartsch, Dohnal, Plum and Reichel4, (1.8)],

\[ \mathcal{D}=\epsilon_{0}(n(x)^{2}\mathcal{E}+\mathcal{P}_{NL}(x,\mathcal{E}))\ \mathrm{with}\ \mathcal{P}_{NL}(x,\mathcal{E})=\chi^{(3)}(\mathcal{E}\cdot\mathcal{E})\mathcal{E}, \]

where $n^{2}(x)=1+\chi ^{(1)}(x)$ is the square of the refractive index and $\chi ^{(1)}$, $\chi ^{(3)}$ denote the linear and cubic susceptibilities of the medium respectively. Particularly, in some nonlocal optic materials, the refractive index $n(x)$ is quite dependent on the electric field $\mathcal {E}$ in a small neighbourhood, and the refractive index change $\triangle n$ can be represented in general form as

\[ \triangle n(E)=s\int_{-\infty}^{+\infty}K(x-y)E(x)\,{\rm d}x, \]

see [Reference Krolikowski, Bang, Rasmussen and Wyller22, (1)]. This phenomenological model is of great significance in the research of laser beams and solitary waves in nonlocal nematic liquid crystals, see [Reference Krolikowski, Bang, Rasmussen and Wyller22, Reference Reimbert, Minzoni and Smyth41] and the reference therein. However, these articles are based on the nonlinear Schrödinger equation, which is an asymptotic approximation of Maxwell's equations. To investigate more information about the electromagnetic waves in the nonlocal optic mediums, one need to deal with the full three-dimensional Maxwell problem. Recently, Mandel [Reference Mandel27] investigated the curl–curl equation with nonlocal nonlinearity

\[ \nabla\times(\nabla\times E)+E=(K(x)\ast|E|^{p})|E|^{p-2}E\quad \mathrm{in}\ \mathbb{R}^{3}, \]

and the author proved that nonlocal media may admit ground states even though the corresponding local models do not admit. In there, the parameter ${\lambda =-\mu \omega ^{2}\varepsilon =1}$ with $\varepsilon <0$ is corresponded to the new artificially produced metamaterials with negative reflexive, see [Reference Padilla, Basov and Smith38], and the kernel $K(x)={\rm e}^{-|x|^{2}}$ is a exponent type responding function which expresses the nonlocal polarization of the nonlocal optical media, see [Reference Krolikowski, Bang, Rasmussen and Wyller22, Reference Nikolov, Neshev, Bang and Królikowski37] for more cases with oscillatory kernel functions. What's more, nonlocality appears naturally in optical systems with a thermal [Reference Litvak26] and it is known to influence the propagation of electromagnetic waves in plasmas [Reference Bergé and Couairon9]. Nonlocality also has attracted considerable interest as a means of eliminating collapse and stabilizing multidimensional solitary waves [Reference Bang, Krolikowski, Wyller and Rasmussen3] and it plays an important role in the theory of Bose–Einstein condensation [Reference Dalfovo15] where it accounts for the finite-range many-body interactions.

1.2. Main results

In the present paper, we are interested in the curl–curl equation with critical convolution part, namely, we consider the curl–curl equation with Riesz potential interaction part

\[ \nabla\times(\nabla\times E)+\lambda E=(I_{\alpha}(x)\ast|E|^{p})|E|^{p-2}E\quad\mathrm{in}\ \mathbb{R}^{3}, \]

where $I_{\alpha }:\mathbb {R}^{3}\longrightarrow \mathbb {R}$ is the Riesz potential of order $\alpha \in (0,3)$ defined for $x\in \mathbb {R}^{3}\setminus \{0\}$ as

\[ I_{\alpha}(x)=\frac{A_{\alpha}}{|x|^{3-\alpha}},\quad A^{\alpha}=\frac{\Gamma\left(\frac{3-\alpha}{2}\right)}{\Gamma\left(\frac{\alpha}{2}\right)\pi^{\frac{N}{2}}2^{\alpha}}. \]

The choice of normalization constant $A^{\alpha }$ ensures that the kernel $I_{\alpha }$ enjoys the semigroup property

\[ I_{\alpha+\beta}=I_{\alpha}\ast I_{\beta}\ \mathrm{for\ each}\ \alpha,\beta\in(0,3)\ \mathrm{such\ that}\ \alpha+\beta<3, \]

see for example [Reference du Plessis16, pp. 73-74]. Indeed, the classical elliptic equation with Riesz potential has been widely studied, and it also has a rich physical background and mathematical research value, see [Reference Du and Yang17Reference Gao and Yang19, Reference Mercuri, Moroz and Van Schaftingen34, Reference Moroz and Schaftingen36] and the reference therein.

We are going to consider the following Brezis–Nirenberg type problem for the curl–curl equation

(1.10)\begin{equation} \begin{cases} \nabla\times(\nabla\times u)+\lambda u=\left(I_{\alpha}\ast|u|^{2^{{\ast}}_{\alpha}}\right)|u|^{2^{{\ast}}_{\alpha}-2}u & \mathrm{in}\ \Omega,\\ \nu\times u=0 & \mathrm{on}\ \partial\Omega, \end{cases} \end{equation}

where $\Omega \subset \mathbb {R}^{3}$ is a bounded domain, either convex or with $\mathcal {C}^{1,1}$ boundary, $\nu$ is the exterior normal, $\lambda <0$ is a real parameter, $2^{\ast }_{\alpha }=3+\alpha$ with $0<\alpha <3$ is the upper critical exponent in the sense of the following Hardy–Littlewood–Sobolev (HLS for short) inequality, see [Reference Lieb and Loss24].

Proposition 1.1 Let $t,r\in (1,\infty )$, $\alpha \in (0,N)$ with $\frac {1}{t}+\frac {N-\alpha }{N}+\frac {1}{r}=2$. For $h\in L^{r}(\mathbb {R}^{N},\mathbb {R}^{N}), g\in L^{t}(\mathbb {R}^{N},\mathbb {R}^{N})$, there exists a sharp constant $C(r,t,N,\alpha )$ independent $g$ and $h$ such that

(1.11)\begin{equation} \int_{\mathbb{R}^{N}}(I_{\alpha}\ast |h|)|g|dx\leq C(r,t,N,\alpha)||h||_{L^{r}(\mathbb{R}^{N},\mathbb{R}^{N})}||g||_{L^{t}(\mathbb{R}^{N},\mathbb{R}^{N})}. \end{equation}

If $t=r=\frac {2N}{N+\alpha }$, then there is a equality in (1.11) if and only if $h(x)=Cg(x)$ and

(1.12)\begin{equation} h(x)=A(\gamma^{2}+|x-a|^{2})^{-\frac{N+\alpha}{2}} \end{equation}

for some $A\in \mathbb {C}$, $0\neq \gamma \in \mathbb {R}$ and $a\in \mathbb {R}^{N}$.

In view of the HLS inequality, the functional corresponds to the nonlocal curl–curl equation

(1.13)\begin{equation} J_{\lambda}(u)=\frac{1}{2}\int_{\Omega}|\nabla\times u|^{2}\,{\rm d}x+\frac{\lambda}{2}\int_{ \Omega}|u|^{2}\,{\rm d}x-\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\Omega}|I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x \end{equation}

is well defined on the natural space $W^{2^{\ast }}_{0}(\mathrm {curl};\Omega )$. However, due to the appearance of the convolution part, this space is not good enough for us to prove the coercive property of the functional. Therefore, it is necessary to introduce the Coulomb space

\[ Q^{\alpha,2^{{\ast}}_{\alpha}}(\Omega,\mathbb{R}^{3})=\{u:\Omega\longrightarrow \mathbb{R}^{3}\ \Big|\ \int_{\Omega}|I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x<\infty\}, \]

see definition 2.1 below. Then, we may define the Coulomb space involve curl operator as

\[ W^{\alpha,2^{{\ast}}_{\alpha}}(\mathrm{curl};\Omega)=\{u\in Q^{\alpha,2^{{\ast}}_{\alpha}}(\Omega,\mathbb{R}^{3}):\nabla\times u\in L^{2}(\Omega,\mathbb{R}^{3})\}, \]

which is a Banach space (see lemma 2.5) if provided with the norm

\[ ||u||_{W^{\alpha,2^{{\ast}}_{\alpha}}(\mathrm{curl};\Omega)}:=(||u||_{Q^{\alpha,2^{{\ast}}_{\alpha}}}^{2}+|\nabla\times u|^{2}_{2})^{1/2}. \]

We also need the following space

\[ W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\Omega)=\overline{\mathcal{C}_{0}^{\infty}(\Omega,\mathbb{R}^{3})}^{||\cdot||_{W^{\alpha,2^{{\ast}}_{\alpha}}(\mathrm{curl};\Omega)}}. \]

In this way, we can easily check that the functional (1.13) is well defined on $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )$, see lemma 2.7. In order to obtain the Brezis–Lieb lemma in the dual space, we extend the linear functionals of the Coulomb space to a mix-norm space, see proposition 2.2. Correspondingly, to establish the Helmholtz decomposition on the work space $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )$ and $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\mathbb {R}^{3})$, see (2.34) and lemma 2.15, we introduce the following subspace

(1.14)\begin{equation} \begin{aligned} \mathcal{V}_{\Omega}: & =\{v\in W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\Omega):\int_{\Omega}\langle v,\varphi\rangle\,{\rm d}x\\ & \quad =0\ \mathrm{for\ every}\ \varphi\in C^{\infty}_{0}(\Omega,\mathbb{R}^{3})\ \mathrm{with}\ \nabla\times\varphi=0\},\\ \mathcal{W}_{\Omega}: & =\{w\in W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\Omega):\int_{\Omega}\langle w,\nabla\times\varphi\rangle {\rm d}x=0\ \mathrm{for\ all}\ \varphi\in C^{\infty}_{0}(\Omega,\mathbb{R}^{3})\}\\ & =\{w\in W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\Omega):\nabla\times w =0\ \mathrm{in\ the\ sense\ of\ distributions}\}. \end{aligned} \end{equation}

Here and below $\langle \cdot,\cdot \rangle$ denote the inner product. Without misunderstanding, we shall write $\mathcal {V}_{\mathbb {R}^{3}}, \mathcal {W}_{\mathbb {R}^{3}}$ if $\Omega =\mathbb {R}^{3}$. Basing on this new decomposition, we can adapt the classical concentration–compactness lemma to suit the new situation. Owing to the concentration-compactness lemma, we obtain the weak–weak$^{\ast }$ continuity of $J'_{\lambda }(u)$ on the Nehari–Pankov manifold [see (4.3)]

\[ \mathcal{N}_{\lambda}:=\{u\in W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\Omega)\setminus(\widetilde{\mathcal{V}}_{\Omega}\oplus\mathcal{W}_{\Omega}):J'_{\lambda}(u)|_{\mathbb{R}u\oplus\widetilde{\mathcal{V}}_{\Omega}\oplus\mathcal{W}_{\Omega}}=0\}, \]

where $\widetilde {\mathcal {V}}_{\Omega }$ is a subspace of $\mathcal {V}_{\Omega }$ on which the quadratic part of $J_{\lambda }$ (see 4.1) is negative semi-definite. Meanwhile, the concentrated compactness lemma implies the $L^{2}(\Omega,\mathbb {R}^{3})$ convergence for the bounded sequence. This allows us to choose the compactly perturbed functional $J_{cp}=J_{0}=J_{\lambda =0}$ that satisfies the condition $(C1)$ in lemma 2.21. By setting another Nehari–Pankov manifold [see (4.4)]

\[ \mathcal{N}_{cp} = \{E\in (\mathcal{V}_{\Omega}\oplus\mathcal{W}_{\Omega})\setminus\mathcal{W}_{\Omega}: J'_{cp}(u)|_{\mathbb{R}u\oplus\mathcal{W}_{\Omega}}=0\}, \]

and controlling the ground state energy of $J_{\lambda }$ lower than the ground state energy of the perturbed functional $J_{cp}$, i.e.

\[ c_{\lambda}=\mathop{\mathrm{inf}}\limits_{\mathcal{N}_{\lambda}}J_{\lambda}< \mathop{\mathrm{inf}}\limits_{\mathcal{N}_{cp}}J_{cp}=c_{0}, \]

we can obtain the ground state solutions of the curl–curl equation (1.10).

It remains to prove the achievement of $c_{0}$. Actually, for the classical elliptic equation (1.6), the sharp constants corresponded to the infimums of the energy level are only attained provided $\Omega =\mathbb {R}^{3}$, and they are independent on the shape of domain. Moreover, one can use the extremal functions to prove that the Mountain-Pass level is below the level where the compactness holds. Inspired by the local case in [Reference Mederski and Szulkin33], we are motivated to investigate the sharp constant of the Sobolev type inequality involving the curl operator on the entire space $\mathbb {R}^{3}$. Let $S_{\mathrm {curl,HL}}=S_{\mathrm {curl,HL}}(\mathbb {R}^{3})$ be the largest constant such that the inequality

(1.15)\begin{equation} \int_{\mathbb{R}^{3}}|\nabla\times u|^{2}\,{\rm d}x\geq S_{\mathrm{curl,HL}}\mathop{\mathrm{inf}}\limits_{w\in \mathcal{W}_{\mathbb{R}^{3}}}\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u+w|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}. \end{equation}

holds for any $u\in W^{\alpha,2^{\ast }_{\alpha }}(\mathrm {curl};\mathbb {R}^{3})\setminus \mathcal {W}_{\mathbb {R}^{3}}$. Then the achievement of the sharp constant $S_{\mathrm {curl},HL}$ is related to a certain least energy solution of the limiting problem

(1.16)\begin{equation} \nabla\times(\nabla\times u)=\left(I_{\alpha}\ast|u|^{2^{{\ast}}_{\alpha}}\right)|u|^{2^{{\ast}}_{\alpha}-2}u,\ \mathrm{in}\ \mathbb{R}^{3}, \end{equation}

where $u\in W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\mathbb {R}^{3})$. By setting the functional

(1.17)\begin{equation} J(u)=\frac{1}{2}\int_{\mathbb{R}^{3}}|\nabla\times u|^{2}\,{\rm d}x-\frac{1}{2\cdot 2^{{\ast}}_{\alpha}}\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x, \end{equation}

and introducing the following Nehari–Pankov manifold [see (3.1)]

\[ \mathcal{N}:=\left\{u\in W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\mathbb{R}^{3})\setminus \mathcal{W}_{\mathbb{R}^{3}}: J'(u)u=0\ \mathrm{and}\ J'(u)|_{\mathcal{W}_{\mathbb{R}^{3}}}=0\right\}, \]

then we have

Theorem 1.2 We have the following two conclusions:

  1. (a). $\mathop {\mathrm {inf}}\limits _{\mathcal {N}} J=\frac {2^{\ast }_{\alpha }-1}{2\cdot 2^{\ast }_{\alpha }} S_{\mathrm {curl},HL}^{\frac {2^{\ast }_{\alpha }}{2^{\ast }_{\alpha }-1}}$ and is attained. Moreover, if $u\in \mathcal {N}$ and $J(u)=\mathop {\mathrm {inf}}\limits _{\mathcal {N}}J$, then $u$ is a ground state solution to equation (1.16) and equality holds in (1.15) for this $u$. If $u$ satisfies equality (1.15), then there are unique $t>0$ and $w\in \mathcal {W}_{\mathbb {R}^{3}}$ such that $t(u+w)\in \mathcal {N}$ and $J(t(u+w))=\mathop {\mathrm {inf}}\limits _{\mathcal {N}}J$.

  2. (b). $S_{\mathrm {curl},HL}> S_{HL}$, where

    (1.18)\begin{equation} S_{HL}:=\mathop{\mathrm{inf}}\limits_{u\in\mathcal{D}^{1,2}(\mathbb{R}^{3},\mathbb{R}^{3})\setminus\{0\}}\frac{\int_{\mathbb{R}^{3}}|\nabla u|^{2}\,{\rm d}x}{\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}}. \end{equation}

Note that $S_{HL}$ is the best constant of the combination of the HLS inequality and the Sobolev inequality, see [Reference Du and Yang17, lemma 1.2] for example. It is not clear that whether the sharp constant $S_{\mathrm {curl},HL}$ is independent on shape of the domain $\Omega$ or not. Therefore, we may define another two constants $S_{\mathrm {curl},HL}(\Omega )$ and $\overline {S}_{\mathrm {curl},HL}(\Omega )$. $S_{\mathrm {curl},HL}(\Omega )$ is the largest possible constant such that the inequality

(1.19)\begin{equation} \int_{\mathbb{R}^{3}}|\nabla\times u|^{2}\,{\rm d}x\geq S_{\mathrm{curl},HL}(\Omega)\mathop{\mathrm{inf}}\limits_{w\in \mathcal{W}_{\mathbb{R}^{3}}}\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u+w|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}} \end{equation}

holds for any $u\in W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )\setminus \mathcal {W}_{\mathbb {R}^{3}}$ with a zero extending; $\overline {S}_{\mathrm {curl},HL}(\Omega )$ is another constant such that the inequality

(1.20)\begin{equation} \int_{\Omega}|\nabla\times u|^{2}\,{\rm d}x\geq \overline{S}_{\mathrm{curl},HL}(\Omega)\mathop{\mathrm{inf}}\limits_{w\in \mathcal{W}_{\Omega}}\left(\int_{\Omega}|I_{\alpha/2}\ast|u+w|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}. \end{equation}

holds for any $u\in W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )\setminus \mathcal {W}_{\Omega }$, and $\overline {S}_{\mathrm {curl},HL}(\Omega )$ is largest with this property. We compare the four constants as follow.

Theorem 1.3 Let $\Omega$ be a bounded domain, either convex or with $\mathcal {C}^{1,1}$ boundary. Then

\[ S_{\mathrm{curl},HL}=S_{\mathrm{curl},HL}(\Omega)\geq \overline{S}_{\mathrm{curl},HL}(\Omega). \]

Unfortunately, we don't have any information about the shape of the solutions of (1.16). Hence, the method of taking cut-off functions and comparing the energy levels does not work well any longer. Inspired by the idea in [Reference Mederski30], we are going to investigate the energy levels of the ground states. From [Reference Bartsch and Mederski5] we know that the spectrum of the curl–curl operator in $W^{2}_{0}(\mathrm {curl};\Omega )$ consists of the eigenvalue $\lambda _{0}=0$ with infinite multiplicity and of a sequence of eigenvalues

\[ 0<\lambda_{1}\leq\lambda_{1}\leq\lambda_{2}\leq{\cdot}\cdot{\cdot}\leq\lambda_{k}\longrightarrow\infty \]

with finite multiplicity $m(\lambda _{k})\in \mathbb {N}$.

The main results for the existence are as follow:

Theorem 1.4 Suppose $\Omega$ is a bounded domain, either convex or with $\mathcal {C}^{1,1}$ boundary. Let $\lambda \in (-\lambda _{\nu },-\lambda _{\nu -1}]$ for some $\nu \geq 1$. Then $c_{\lambda }>0$ and the following statements hold$:$

  1. (a) If $c_{\lambda }< c_{0}$, then there is ground state solution to (1.10) , i.e. $c_{\lambda }$ is attained by a critical point of $J_{\lambda }$. A sufficient condition for this inequality to hold is $\lambda \in (-\lambda _{\nu },-\lambda _{\nu }+\bar {S}_{\mathrm {curl},HL}(\Omega )|\mathrm {diam}\Omega |^{-\frac {3\cdot 2^{\ast }_{\alpha }-\alpha -3}{2^{\ast }_{\alpha }}})$, where $|\mathrm {diam}\Omega |=\mathop {max}\limits _{x,y\in \Omega }|x-y|$.

  2. (b) There exists $\varepsilon _{\nu }\geq \bar {S}_{\mathrm {curl},HL}(\Omega )|\ \mathrm {diam}\ \Omega |^{-\frac {3\cdot 2^{\ast }_{\alpha }-\alpha -3}{2^{\ast }_{\alpha }}}$ such that $c_{\lambda }$ is not attained for $\lambda \in (-\lambda _{\nu }+\varepsilon _{\nu },-\lambda _{\nu -1}]$, and $c_{\lambda }=c_{0}$ for $\lambda \in (-\lambda _{\nu }+\varepsilon _{\nu },-\lambda _{\nu -1}]$. We do not exclude that $\varepsilon >\lambda _{\nu }-\lambda _{\nu -1}$, so these intervals may be empty.

  3. (c) $c_{\lambda }\longrightarrow 0$ as $\lambda \longrightarrow -\lambda ^{-}_{\nu }$, and the function

    \[ (-\lambda_{\nu},-\lambda_{\nu}+\varepsilon_{\nu}]\cap (-\lambda_{\nu},-\lambda_{\nu-1}]\ni\lambda\mapsto c_{\lambda}\in(0,\infty) \]
    is continuous and strictly increasing.
  4. (d) There exist at least $\sharp \{k:-\lambda _{k}\!<\!\lambda \!<\!-\lambda _{k}\!+\!\frac {2^{\ast }_{\alpha }-1}{2\cdot 2^{\ast }_{\alpha }}\bar {S}_{\mathrm {curl},HL}(\Omega )|\ \mathrm {diam}\ \Omega |^{-\frac {3\cdot 2^{\ast }_{\alpha }-\alpha -3}{2^{\ast }_{\alpha }}}\}$ pairs of solutions $\pm u$ to (1.10).

The paper is organized as follow. In §2 we introduce some work spaces on bounded domains and entire space $\mathbb {R}^{3}$, and we adapt the concentration compactness lemma for the curl–curl problem with nonlocal nonlinearities. And an abstract critical point theorem is also recalled in this part for the readers’ convenience. In §3, we show that the sharp constant $S_{\mathrm {curl},HL}$ is attained provided $\Omega =\mathbb {R}^{3}$, and we compare the four constants as we introduced. In the last Section, we are devoted to the proof of theorem 1.4.

2. Preliminaries and variational setting

2.1. Preliminaries

Throughout this paper we assume that $\Omega \subset \mathbb {R}^{3}$ is a bounded domain, either convex or with $\mathcal {C}^{1,1}$ boundary. In some cases $\Omega$ is only required to be a Lipschitz domain, see [Reference Mederski and Szulkin33] for more details. We shall look for solutions to problem (1.10) and (1.16) in $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )$ and $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\mathbb {R}^{3})$ respectively. Now we are ready to introdcue the definitions of the working spaces.

2.1.1. Coulomb space involving curl operator.

Definition 2.1 [Reference Mercuri, Moroz and Van Schaftingen34, definition 1] Let $N\in \mathbb {N}$, $\alpha \in (0,N)$ and $p\geq 1$. We define the Coulomb space $Q^{\alpha,p}(\mathbb {R}^{N},\mathbb {R}^{N})$ as the vector space of measurable functions $u: \mathbb {R}^{N}\longrightarrow \mathbb {R}^{N}$ such that

\[ ||u||_{Q^{\alpha,p}(\mathbb{R}^{N},\mathbb{R}^{N})}=\left(\int_{\mathbb{R}^{N}}|I_{\alpha/2}\ast|u|^{p}|^{2}dx\right)^{\frac{1}{2p}}<{+}\infty. \]

It is not difficult to see that $||\cdot ||_{Q^{\alpha,p}(\mathbb {R}^{N},\mathbb {R}^{N})}$ defines a norm, see proposition 2.1 in [Reference Mercuri, Moroz and Van Schaftingen34], and the Coulomb space is complete with respect to this norm. By the same way, we also define $Q^{\alpha,p}(\Omega,\mathbb {R}^{N})$ as the Coulomb space on the bounded domain. For the the dual space of $Q^{\alpha,p}(\mathbb {R}^{N},\mathbb {R}^{N})$, it can be characterized by the following proposition, and it is also adopted in $Q^{\alpha,p}(\Omega,\mathbb {R}^{N})$.

Proposition 2.2 [Reference Mercuri, Moroz and Van Schaftingen34, proposition 2.11] Let $T$ be a distribution, then $T\in (Q^{\alpha,p}(\mathbb {R}^{N},\mathbb {R}^{N}))'$ if and only if there exists $G(x,y)\in L^{\frac {2p}{2p-1}}(\mathbb {R}^{N},L^{\frac {p}{p-1}}(\mathbb {R}^{N}))$ such that for every $\varphi \in \mathcal {C}_{0}^{\infty }(\mathbb {R}^{N},\mathbb {R}^{N})$,

\[ \langle T,\varphi\rangle=\int_{\mathbb{R}^{N}}\left(\int_{\mathbb{R}^{N}}G(x,y)I_{\alpha/2}(x-y)^{\frac{1}{p}}\,{\rm d}y\right)\varphi(x)\,{\rm d}x. \]

Proof. By the definition of the Coulomb space $Q^{\alpha,p}(\mathbb {R}^{N},\mathbb {R}^{N})$, one can observe that the map

\[ \mathcal{L}:Q^{\alpha,p}(\mathbb{R}^{N},\mathbb{R}^{N})\longrightarrow L^{2p}(\mathbb{R}^{N},L^{p}(\mathbb{R}^{N})) \]

defined by

\[ \mathcal{L}u(x,y)=(I_{\alpha/2}(x-y))^{\frac{1}{p}}u(y) \]

is a linear isometry from $Q^{\alpha,p}(\mathbb {R}^{N},\mathbb {R}^{N})$ into $L^{2p}(\mathbb {R}^{N},L^{p}(\mathbb {R}^{N}))$. Then any linear functional on $Q^{\alpha,p}(\mathbb {R}^{N})$ can be extended to a linear functional on $L^{2p}(\mathbb {R}^{N},L^{p}(\mathbb {R}^{N}))$. Namely, there exists $G(x,y)\in L^{\frac {2p}{2p-1}}\left (\mathbb {R}^{N},L^{\frac {p}{p-1}}(\mathbb {R}^{N})\right )$ such that

\[ \langle T,\varphi\rangle=\langle G(x,y),\mathcal{L}\varphi\rangle. \]

For the Coulomb space involving curl operator, we have the following definition.

Definition 2.3 Let $N=3$, $\alpha \in (0,3)$ and $p\geq 1$. We define the Coulomb space involving curl operator $W^{\alpha,p}(\mathrm {curl};\mathbb {R}^{3})$ as the vector space of functions $u\in Q^{\alpha,p}(\mathbb {R}^{3},\mathbb {R}^{3})$ such that u is weakly differentiable in $\mathbb {R}^{3}$, $\nabla \times u\in L^{2}(\mathbb {R}^{3},\mathbb {R}^{3})$ and

\[ ||u||_{W^{\alpha,p}(\mathrm{curl};\mathbb{R}^{3})}=\left(\int_{\mathbb{R}^{3}}|\nabla\times u|^{2}\,{\rm d}x+\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u|^{p}|^{2}\,{\rm d}x\right)^{\frac{1}{p}}\right)^{\frac{1}{2}} <{+}\infty. \]

The function $||\cdot ||_{W^{\alpha,p}(\mathrm {curl};\mathbb {R}^{3})}$ defines a norm in view of the proposition 2.1 in [Reference Mercuri, Moroz and Van Schaftingen34]. By the same way, we also define $W^{\alpha,p}(\mathrm {curl};\Omega )$ as the Coulomb space involve curl operator on the bounded spaces, namely,

(2.1)\begin{equation} W^{\alpha,p}(\mathrm{curl};\Omega):=\{u\in Q^{\alpha,p}(\Omega,\mathbb{R}^{3}):\nabla\times u\in L^{2}(\Omega,\mathbb{R}^{3})\}. \end{equation}

We are going to prove that $W^{\alpha,p}(\mathrm {curl};\mathbb {R}^{3})$ and $W^{\alpha,p}(\mathrm {curl};\Omega )$ are Banach spaces. The proof of completeness follows by the same arguments as in the proof of theorem 4.3 in [Reference Kirsch and Hettlich21] and proposition 2.2 in [Reference Mercuri, Moroz and Van Schaftingen34]. The first ingredient is the following Fatou property for locally converging sequences.

Lemma 2.4 Let $N=3$, $\alpha \in (0,3)$ and $p\geq 1$. If $(u_{n})_{n\in \mathbb {N}}$ is a bounded sequence in $W^{\alpha,p}(\mathrm {curl};\mathbb {R}^{3})$ that converges to a function $u:\mathbb {R}^{3}\longrightarrow \mathbb {R}^{3}$ in $L^{1}_{loc}(\mathbb {R}^{3},\mathbb {R}^{3})$, then $u\in W^{\alpha,p}(\mathrm {curl};\mathbb {R}^{3})$,

(2.2)\begin{equation} \int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast| u|^{p}|^{2}\,{\rm d}x\leq\mathop{\mathrm{lim\ inf}}\limits_{n\longrightarrow\infty}\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast| u_{n}|^{p}|^{2}\,{\rm d}x, \end{equation}

and

(2.3)\begin{equation} \int_{\mathbb{R}^{3}}|\nabla\times u|^{2}\,{\rm d}x\leq\mathop{\mathrm{lim\ inf}}\limits_{n\longrightarrow\infty}\int_{\mathbb{R}^{3}}|\nabla\times u_{n}|^{2}\,{\rm d}x. \end{equation}

Proof. Since $(u_{n})_{n\in \mathbb {N}}\longrightarrow u$ is bounded in $W^{\alpha,p}(\mathrm {curl};\mathbb {R}^{3})$, we have

(2.4)\begin{equation} \int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u_{n}|^{p}|^{2}\,{\rm d}x\leq \infty, \end{equation}

then by the Fatou lemma, we have

(2.5)\begin{equation} \int_{\mathbb{R}^{3}}\mathop{\mathrm{lim\ inf}}\limits_{n\longrightarrow\infty}|I_{\alpha/2}\ast| u_{n}|^{p}|^{2}dx\leq\mathop{\mathrm{lim\ inf}}\limits_{n\longrightarrow\infty}\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast| u_{n}|^{p}|^{2}\,{\rm d}x. \end{equation}

By the Fatou lemma again, we have

(2.6)\begin{equation} I_{\alpha}\ast(\mathop{\mathrm{lim\ inf}}\limits_{n\longrightarrow\infty} |u_{n}|^{p}) \leq\mathop{\mathrm{lim\ inf}}\limits_{n\longrightarrow\infty} I_{\alpha}\ast(|u_{n}|^{p}). \end{equation}

Since $(u_{n})_{n\in \mathbb {N}}\longrightarrow u$ in $L^{1}_{loc}(\mathbb {R}^{3},\mathbb {R}^{3})$, for almost every $x\in \mathbb {R}^{3}$, we have

(2.7)\begin{equation} I_{\alpha}\ast(\mathop{\mathrm{lim\ inf}}\limits_{n\longrightarrow\infty} |u_{n}|^{p})(x) \longrightarrow I_{\alpha}\ast(|u|^{p})(x). \end{equation}

Then (2.2) follows (2.5), (2.6) and (2.7).

We are going to prove (2.3). Define $f$ on $\mathcal {D}(\mathbb {R}^{3},\mathbb {R}^{3})$ by

(2.8)\begin{equation} \langle f,v\rangle=\int_{\mathbb{R}^{3}}u\cdot (\nabla\times v)\,{\rm d}x, \end{equation}

since $u_{n}\longrightarrow u$ in $L^{1}_{loc}(\mathbb {R}^{3},\mathbb {R}^{3})$, we have

(2.9)\begin{align} |\langle f,v\rangle|& =|\int_{\mathbb{R}^{3}}u\cdot (\nabla\times v)\mathrm{d}x|=\mathop{\mathrm{lim}}\limits_{n\longrightarrow\infty}|\int_{\mathbb{R}^{3}}u_{n}\cdot (\nabla\times v)\,{\rm d}x|\nonumber\\ & =\mathop{\mathrm{lim}}\limits_{n\longrightarrow\infty}|\int_{\mathbb{R}^{3}}(\nabla\times u_{n})\cdot v\,{\rm d}x|\leq \mathop{\mathrm{lim\ inf}}\limits_{n\longrightarrow\infty}||\nabla\times u_{n}||_{2}\left(\int_{\mathbb{R}^{3}}|v|^{2}\,{\rm d}x\right)^{\frac{1}{2}}, \end{align}

where we use the Cauchy–Schwarz inequality. Since $\mathcal {D}(\mathbb {R}^{3},\mathbb {R}^{3})$ is dense in $L^{2}(\mathbb {R}^{3},\mathbb {R}^{3})$, by the the Hahn-Banach theorem, the distribution $f$ can be continuously extend to a linear functional on $L^{2}(\mathbb {R}^{3},\mathbb {R}^{3})$. Therefore, by the Riesz representation theorem, there exists $F\in L^{2}(\mathbb {R}^{3},\mathbb {R}^{3})$ such that for every $v\in \mathcal {D}(\mathbb {R}^{3},\mathbb {R}^{3})$

(2.10)\begin{equation} \int_{\mathbb{R}^{3}}F\cdot v\,{\rm d}x=\langle f,v\rangle=\int_{\mathbb{R}^{3}}u\cdot (\nabla\times v)\,{\rm d}x. \end{equation}

Setting $\nabla \times u$ as the curl of $u$ in the following distribute sense

(2.11)\begin{equation} \int_{\mathbb{R}^{3}}u\cdot(\nabla\times v)\,{\rm d}x=\int_{\mathbb{R}^{3}}(\nabla\times u)\cdot v\,{\rm d}x, \end{equation}

we can see $F=\nabla \times u\in L^{2}(\mathbb {R}^{3},\mathbb {R}^{3})$ in the weak sense. Choosing $v=\nabla \times u$ we find that

(2.12)\begin{align} \int_{\mathbb{R}^{3}}|\nabla\times u|^{2}\,{\rm d}x& \leq \mathop{\mathrm{lim\ inf}}\limits_{n\longrightarrow\infty}||\nabla\times u_{n}||_{2}\left(\int_{\mathbb{R}^{3}}|v|^{2}\mathrm{d}x\right)^{\frac{1}{2}}\nonumber\\ & \leq \mathop{\mathrm{lim\ inf}}\limits_{n\longrightarrow\infty}||\nabla\times u_{n}||_{2}\left(\int_{\mathbb{R}^{3}}|\nabla\times u|^{2}\mathrm{d}x\right)^{\frac{1}{2}}. \end{align}

Therefore we have

(2.13)\begin{equation} \int_{\mathbb{R}^{3}}|\nabla\times u|^{2}\,{\rm d}x\leq\mathop{\mathrm{lim\ inf}}\limits_{n\longrightarrow\infty}\int_{\mathbb{R}^{3}}|\nabla\times u_{n}|^{2}\,{\rm d}x. \end{equation}

Lemma 2.5 Let N=3, $\alpha \in (0,3)$ and $p\geq 1$. The normed spaces $W^{\alpha,p}(\mathrm {curl};\mathbb {R}^{3})$ and $W^{\alpha,p}(\mathrm {curl};\Omega )$ are complete.

Proof. Let $(u_{n})_{n\in \mathbb {N}}$ be a Cauchy sequence in $W^{\alpha,p}(\mathrm {curl};\mathbb {R}^{3})$. By the local estimate of the Coulomb energy, $(u_{n})_{n\in \mathbb {N}}$ is also a Cauchy sequence in $L_{loc}^{p}(\mathbb {R}^{3},\mathbb {R}^{3})$. Hence there exists $u\in L_{loc}^{p}(\mathbb {R}^{3},\mathbb {R}^{3})$ such that $(u_{n})_{n\in \mathbb {N}}\longrightarrow u$ in $L_{loc}^{p}(\mathbb {R}^{3},\mathbb {R}^{3})$. In light of lemma 2.4, we conclude that $u\in W^{\alpha,p}(\mathrm {curl};\mathbb {R}^{3})$. Moreover, for every $n\in \mathbb {N}$ the sequence $(u_{n}-u_{m})_{m\in \mathbb {N}}$ converges to $(u_{n}-u)$ in $L_{loc}^{p}(\mathbb {R}^{3},\mathbb {R}^{3})$. Hence, by lemma 2.4 again, we have

\begin{align*} & \mathop{\mathrm{lim\ sup}}\limits_{n\longrightarrow\infty}\left(\int_{\mathbb{R}^{3}}|\nabla\times u_{n}-\nabla\times u|^{2}\,{\rm d}x+\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u_{n}-u|^{p}|^{2}\,{\rm d}x\right)\\ & \leq\mathop{\mathrm{lim\ sup}}\limits_{n\longrightarrow\infty}\mathop{\mathrm{lim\ sup}}\limits_{m\longrightarrow\infty} \left(\int_{\mathbb{R}^{3}}|\nabla\times u_{n}-\nabla\times u_{m}|^{2}\,{\rm d}x+\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u_{n}-u_{m}|^{p}|^{2}\,{\rm d}x\right)\\ & \leq\mathop{\mathrm{lim\ sup}}\limits_{m,n\longrightarrow\infty}\left(\int_{\mathbb{R}^{3}}|\nabla\times u_{n}-\nabla\times u_{m}|^{2}\,{\rm d}x+\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u_{n}-u_{m}|^{p}|^{2}\,{\rm d}x\right)\\ & \leq0. \end{align*}

This implies $W^{\alpha,p}(\mathrm {curl};\mathbb {R}^{3})$ is complete. The completeness of $W^{\alpha,p}(\mathrm {curl};\Omega )$ can be proved in the same way.

We also define

\[ W^{\alpha,p}_{0}(\mathrm{curl};\Omega)=\mathrm{closure}\ \mathrm{of}\ \mathcal{C}^{\infty}_{0}(\Omega;\mathbb{R}^{3})\ \mathrm{in}\ W^{\alpha,p}(\mathrm{curl};\Omega). \]

If $p$ lies in some suitable range, then the two Coulomb spaces are the same for the case $\Omega =\mathbb {R}^{3}$.

Lemma 2.6 Let $\alpha \in (0,3)$, $\frac {3+\alpha }{3}\leq p\leq 3+\alpha$, then $W^{\alpha,p}(\mathrm {curl};\mathbb {R}^{3})=W^{\alpha,p}_{0}(\mathrm {curl};\mathbb {R}^{3})$.

Proof. Let $\eta _{R}\in \mathcal {C}^{\infty }_{0}(\mathbb {R}^{3})$ be such that $|\nabla \eta _{R}|\leq \frac {2}{R}$ for $R\leq |x|\leq 2R$, $\eta _{R}=1$ for $|x|\leq R$ and $\eta _{R}=0$ for $|x|\geq 2R$. Then for $u=(u_{1},u_{2},u_{3})\in W^{\alpha,p}(\rm {curl},\mathbb {R}^{3})$, we have $\eta _{R}u\longrightarrow u$ in $Q^{\alpha,p}(\mathbb {R}^{3},\mathbb {R}^{3})$ as $R\longrightarrow \infty$. Note that

(2.14)\begin{equation} \nabla\times(\eta_{R}u_{i}) =(\partial_{i}\eta_{R})u_{j}-(\partial_{j}\eta_{R})u_{i}+\eta_{R}(\partial_{i}u_{j}-\partial_{j}u_{i}),\quad i\neq j. \end{equation}

If $p=2$, we have $(\partial _{i}\eta _{R})u_{j}\longrightarrow 0$ in $L^{2}(\mathbb {R}^{3},\mathbb {R}^{3})$ as $\alpha \in (0,3)$. Indeed, we have

\begin{align*} \int_{\mathbb{R}^{3}}(\partial\eta_{R})^{2}u_{j}^{2}\,{\rm d}x& =\int_{R\leq|x|\leq 2R}(\partial\eta_{R})^{2}u_{j}^{2}\,{\rm d}x\leq\left(\frac{2}{R}\right)^{2}\int_{R\leq|x|\leq 2R}u_{j}^{2}\,{\rm d}x\\ & \quad\leq\left(\frac{2}{R}\right)^{2}\int_{|x|\leq 2R}u_{j}^{2}\,{\rm d}x\\ & \leq CR^{{-}2}(2R)^{\frac{3-\alpha}{2}}\left(\int_{|x|\leq 2R}|I_{\alpha/2}\ast|u|^{2}|^{2}\mathrm{d}x\right)^{\frac{1}{2}}. \end{align*}

If $p\neq 2$, let $q$ be such that $\frac {1}{p}+\frac {1}{q}=\frac {1}{2}$, then applying the Hölder inequality we have

\begin{align*} \int_{\mathbb{R}^{3}}(\partial\eta_{R})^{2}u_{j}^{2}\,{\rm d}x& \leq\left(\int_{R\leq|x|\leq 2R}|\partial_{i}\eta_{R}|^{q}\,{\rm d}x\right)^{\frac{2}{q}}\left(\int_{R\leq|x|\leq 2R}|u_{j}|^{p}\,{\rm d}x\right)^{\frac{2}{p}}\\ & \leq C_{1}(R^{3-q})^{\frac{2}{q}}(2R^{\frac{3-\alpha}{2}})^{\frac{2}{p}}\left(\int_{|x|\leq 2R}|I_{\frac{\alpha}{2}}\ast|u|^{p}|^{2}\,{\rm d}x\right)^{\frac{1}{p}}\\ & \leq C_{2}R^{\frac{p-(3+\alpha)}{p}}\left(\int_{|x|\leq 2R}|I_{\alpha/2}\ast|u|^{p}|^{2}\,{\rm d}x\right)^{\frac{1}{p}}. \end{align*}

Then, for $p\leq 3+\alpha$, we have $(\partial _{i}\eta _{R})u_{j}\longrightarrow 0$ in $L^{2}(\mathbb {R}^{3},\mathbb {R}^{3})$ as $R\longrightarrow \infty$. As $\partial _{i}u_{j}-\partial _{j}u_{i}\in L^{2}(\mathbb {R}^{3})$, it follows that the left-hand side in (2.14) tends to $\partial _{i}u_{j}-\partial _{j}u_{i}$ in $L^{2}(\mathbb {R}^{3})$ as $R\longrightarrow \infty$. Hence $\eta _{R}u\longrightarrow u$ in $W^{\alpha,p}(\rm {curl};\mathbb {R}^{3})$ and functions of compact support are dense in $W^{\alpha,p}(\rm {curl};\mathbb {R}^{3})$. The rest of the proof is similar to the [Reference Mederski and Szulkin33, lemma 2.1].

Lemma 2.7

  1. (i) $J_{\lambda }(u)$ and $J(u)$ are well defined on $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )$ and $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\mathbb {R}^{3})$ respectively.

  2. (ii) Let

    \[ G(u)(x,y)=\left(\frac{1}{|x-y|^{\frac{2^{{\ast}}_{\alpha}-1}{2^{{\ast}}_{\alpha}}N-\frac{2\cdot2^{{\ast}}_{\alpha}-1}{2\cdot2^{{\ast}}_{\alpha}}\alpha}}\right)|u(y)|^{2^{{\ast}}_{\alpha}}|u(x)|^{2^{{\ast}}_{\alpha}-1}. \]
    Then, $G(u)(x,y)\in L^{\frac {2\cdot 2^{\ast }_{\alpha }}{2\cdot 2^{\ast }_{\alpha }-1}}(\mathbb {R}^{3};L^{\frac {2^{\ast }_{\alpha }}{2^{\ast }_{\alpha }-1}}(\mathbb {R}^{3}))$.
  3. (iii) $J_{\lambda }(u)$ and $J(u)$ are of class $\mathcal {C}^{1}$.

Proof.

  1. (i) From the definition of $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )$ and $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\mathbb {R}^{3})$ , we know that the functionals $J_{\lambda }(u)$ and $J(u)$ are well defined.

  2. (ii) Set

    \[ I(u)=\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x. \]
    We claim that $I'(u)\in (Q^{\alpha,2^{\ast }_{\alpha }}(\mathbb {R}^{3},\mathbb {R}^{3}))'$. Indeed, for any $\varphi \in Q^{\alpha,2^{\ast }_{\alpha }}(\mathbb {R}^{3},\mathbb {R}^{3})$, we have
    (2.15)\begin{align} \langle I'(u),\varphi\rangle& =\int_{\mathbb{R}^{3}}(I_{\alpha} \ast|u|^{2^{{\ast}}_{\alpha}})|u|^{2^{{\ast}}_{\alpha}-2}u\cdot\varphi\,{\rm d}x\leq\left(\int_{\mathbb{R}^{3}}(I_{\alpha}\ast|u|^{2^{{\ast}}_{\alpha}})|u|^{2^{{\ast}}_{\alpha}}\mathrm{d}x\right)^{\frac{2^{{\ast}}_{\alpha}-1}{2^{{\ast}}_{\alpha}}} \nonumber\\ & \qquad \cdot\left(\int_{\mathbb{R}^{3}}(I_{\alpha}\ast|u|^{2^{{\ast}}_{\alpha}})|\varphi|^{2^{{\ast}}_{\alpha}}\mathrm{d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}\nonumber\\ & =||u||^{\frac{2^{{\ast}}_{\alpha}-1}{2}}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}\cdot\left(\int_{\mathbb{R}^{3}}(I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}})(I_{\alpha/2}\ast|\varphi|^{2^{{\ast}}_{\alpha}}\mathrm{d}x)\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}\nonumber\\ & \leq||u||^{\frac{2^{{\ast}}_{\alpha}-1}{2}}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}\cdot\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2\cdot2^{{\ast}}_{\alpha}}} \nonumber\\ & \qquad \cdot\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|\varphi|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2\cdot2^{{\ast}}_{\alpha}}}\nonumber\\ & =||u||^{\frac{2^{{\ast}}_{\alpha}+1}{2}}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}\cdot ||\varphi||_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}. \end{align}
    Then, by the definition of the functional space on Coulomb space, we have $I'(u)\in (Q^{\alpha,2^{\ast }_{\alpha }}(\mathbb {R}^{3},\mathbb {R}^{3}))'$.

    On the other hand, $G(u)(x,y)$ obviously satisfies that

    (2.16)\begin{equation} \langle I'(u),\varphi\rangle=\int_{\mathbb{R}^{3}}\left(\int_{\mathbb{R}^{3}}G(u)(x,y)(I_{\alpha/2}(x-y))^{\frac{1}{2^{{\ast}}_{\alpha}}}\,{\rm d}y\right)\varphi(x)\,{\rm d}x, \end{equation}
    Therefore, by proposition 2.2, we have $G(u)(x,y)\in L^{\frac {2\cdot 2^{\ast }_{\alpha }}{2\cdot 2^{\ast }_{\alpha }-1}}(\mathbb {R}^{3};L^{\frac {2^{\ast }_{\alpha }}{2^{\ast }_{\alpha }-1}}(\mathbb {R}^{3}))$.
  3. (iii) We are going to show that $I'(u)$ is continuous. For any sequences $u_{n}$, $u$ and $\varphi \in Q^{\alpha,2^{\ast }_{\alpha }}(\mathbb {R}^{3},\mathbb {R}^{3})$, we have

    (2.17)\begin{align} & \langle(I'(u_{n})-I'(u)),\varphi\rangle\nonumber\\ & =\int_{\mathbb{R}^{3}}\left(I_{\alpha}\ast|u_{n}|^{2^{{\ast}}_{\alpha}}\right)|u_{n}|^{2^{{\ast}}_{\alpha}-2}u_{n}\cdot\varphi\,{\rm d}x-\int_{\mathbb{R}^{3}}\left(I_{\alpha}\ast|u|^{2^{{\ast}}_{\alpha}}\right)|u|^{2^{{\ast}}_{\alpha}-2}u\cdot\varphi\,{\rm d}x\nonumber\\ & =\int_{\mathbb{R}^{3}}\left(I_{\alpha}\ast(|u_{n}|^{2^{{\ast}}_{\alpha}}-|u|^{2^{{\ast}}_{\alpha}})\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}\left(I_{\alpha}\ast(|u_{n}|^{2^{{\ast}}_{\alpha}}-|u|^{2^{{\ast}}_{\alpha}})\right)^{\frac{2^{{\ast}}_{\alpha}-1}{2^{{\ast}}_{\alpha}}}|u_{n}|^{2^{{\ast}}_{\alpha}-2}u_{n}\cdot\varphi\,{\rm d}x\nonumber\\ & \quad+\int_{\mathbb{R}^{3}}\left(I_{\alpha}\ast|u|^{2^{{\ast}}_{\alpha}}\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}\left(I_{\alpha}\ast|u|^{2^{{\ast}}_{\alpha}}\right)^{\frac{2^{{\ast}}_{\alpha}-1}{2^{{\ast}}_{\alpha}}}(|u_{n}|^{2^{{\ast}}_{\alpha}-2}u_{n}-|u|^{2^{{\ast}}_{\alpha}-2}u)\cdot\varphi\,{\rm d}x\nonumber\\ & \leq\left(\int_{\mathbb{R}^{3}}\left(I_{\alpha}\ast(|u_{n}|^{2^{{\ast}}_{\alpha}}-|u|^{2^{{\ast}}_{\alpha}}) \right)|\varphi|^{2^{{\ast}}_{\alpha}}dx\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}\nonumber\\ & \qquad\cdot \left(\int_{\mathbb{R}^{3}}\left(I_{\alpha}\ast(|u_{n}|^{2^{{\ast}}_{\alpha}}-|u|^{2^{{\ast}}_{\alpha}}) \right)|u_{n}|^{2^{{\ast}}_{\alpha}}\,{\rm d}x\right)^{\frac{2^{{\ast}}_{\alpha}-1}{2^{{\ast}}_{\alpha}}}\nonumber\\ & \quad+\left(\int_{\mathbb{R}^{3}}\left(I_{\alpha}\ast|u|^{2^{{\ast}}_{\alpha}} \right)|\varphi|^{2^{{\ast}}_{\alpha}}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}\nonumber\\ & \qquad\cdot \left(\int_{\mathbb{R}^{3}}\left(I_{\alpha}\ast|u|^{2^{{\ast}}_{\alpha}} \right)(|u_{n}|^{2^{{\ast}}_{\alpha}-1}-|u|^{2^{{\ast}}_{\alpha}-1})^{\frac{2^{{\ast}}_{\alpha}}{2^{{\ast}}_{\alpha}-1}}\,{\rm d}x\right)^{\frac{2^{{\ast}}_{\alpha}-1}{2^{{\ast}}_{\alpha}}}\nonumber\\ & =A_{1}\cdot A_{2}+A_{3}\cdot A_{4}. \end{align}
    By the semi-group property and Hölder inequality, we have
    (2.18)\begin{align} A_{1}& =\left(\int_{\mathbb{R}^{3}}\left(I_{\alpha/2}\ast(|u_{n}|^{2^{{\ast}}_{\alpha}}-|u|^{2^{{\ast}}_{\alpha}})\right)\left(I_{\alpha/2}\ast|\varphi|^{2^{{\ast}}_{\alpha}}\right) {\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}\nonumber\\ & \leq\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast(|u_{n}|^{2^{{\ast}}_{\alpha}}-|u|^{2^{{\ast}}_{\alpha}})|^{2}\,{\rm d}x\right)^{\frac{1}{2\cdot2^{{\ast}}_{\alpha}}}\cdot \left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|\varphi|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2\cdot2^{{\ast}}_{\alpha}}}\nonumber\\ & =B_{1}^{\frac{1}{2\cdot2^{{\ast}}_{\alpha}}}\cdot||\varphi||_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}, \end{align}
    where $B_{1}=\int _{\mathbb {R}^{3}}|I_{\alpha /2}\ast (|u_{n}|^{2^{\ast }_{\alpha }}-|u|^{2^{\ast }_{\alpha }})|^{2}dx$. Recalling the mean value theorem, and noting that $|\xi |=|u|+\theta |u_{n}-u|$ for some $\theta \in (0,1)$, we have
    (2.19)\begin{align} & ||u_{n}|^{2^{{\ast}}_{\alpha}}-|u|^{2^{{\ast}}_{\alpha}}|=C(2^{{\ast}}_{\alpha})(|u|+\theta|u_{n}-u|)^{2^{{\ast}}_{\alpha}-1}|u_{n}-u|\nonumber\\ & \quad=|\xi|^{2^{{\ast}}_{\alpha}-1}|u_{n}-u|\quad\mathrm{for}\ 0\leq\theta\leq1. \end{align}
    Therefore, by linearity of the convolution and by positivity of the Riesz–kernel, we deduce that
    (2.20)\begin{align} B_{1}& =\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast(|\xi|^{2^{{\ast}}_{\alpha}-1}|u_{n}-u|)|^{2}dx\nonumber\\ & =\int_{\mathbb{R}^{3}}I_{\alpha}\ast(|\xi|^{2^{{\ast}}_{\alpha}-1}|u_{n}-u|)\cdot (|\xi|^{2^{{\ast}}_{\alpha}-1}|u_{n}-u|)\,{\rm d}x\nonumber\\ & \leq \left( \int_{\mathbb{R}^{3}}\left(I_{\alpha}\ast(|\xi|^{2^{{\ast}}_{\alpha}-1}|u_{n}-u|)\right) |\xi|^{2^{{\ast}}_{\alpha}}\,{\rm d}x\right)^{\frac{2^{{\ast}}_{\alpha}-1}{2^{{\ast}}_{\alpha}}}\nonumber\\ & \qquad \cdot\left( \int_{\mathbb{R}^{3}}\left(I_{\alpha}\ast(|\xi|^{2^{{\ast}}_{\alpha}-1}|u_{n}-u|)\right) |u_{n}-u|^{2^{{\ast}}_{\alpha}}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}\nonumber\\ & =\left( \int_{\mathbb{R}^{3}}\left(I_{\alpha/2}\ast(|\xi|^{2^{{\ast}}_{\alpha}-1}|u_{n}-u|)\right) \cdot \left(I_{\alpha/2}\ast|\xi|^{2^{{\ast}}_{\alpha}}\right){\rm d}x\right)^{\frac{2^{{\ast}}_{\alpha}-1}{2^{{\ast}}_{\alpha}}}\nonumber\\ & \quad\cdot\left( \int_{\mathbb{R}^{3}}\left(I_{\alpha/2}\ast(|\xi|^{2^{{\ast}}_{\alpha}-1}|u_{n}-u|)\right) \cdot \left(I_{\alpha/2}\ast|u_{n}-u|^{2^{{\ast}}_{\alpha}}\right){\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}\nonumber\\ & \leq\left( \int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast(|\xi|^{2^{{\ast}}_{\alpha}-1}|u_{n}-u|)|^{2}\,{\rm d}x\right)^{\frac{1}{2}\frac{2^{{\ast}}_{\alpha}-1}{2^{{\ast}}_{\alpha}}}\nonumber\\ & \qquad \cdot \left( \int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|\xi|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2}\frac{2^{{\ast}}_{\alpha}-1}{2^{{\ast}}_{\alpha}}}\nonumber\\ & \quad\cdot\left( \int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast(|\xi|^{2^{{\ast}}_{\alpha}-1}|u_{n}-u|)|^{2}\,{\rm d}x\right)^{\frac{1}{2}\frac{1}{2^{{\ast}}_{\alpha}}}\nonumber\\ & \qquad\cdot \left( \int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast(|u_{n}-u|^{2^{{\ast}}_{\alpha}})|^{2}\,{\rm d}x\right)^{\frac{1}{2}\frac{1}{2^{{\ast}}_{\alpha}}}\nonumber\\ & \leq B_{1}^{\frac{1}{2}}\left(||u_{n}||^{2^{{\ast}}_{\alpha}-1}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}+ ||u||^{2^{{\ast}}_{\alpha}-1}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}\right) \cdot||u_{n}-u||_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}. \end{align}
    This implies
    (2.21)\begin{equation} B_{1}\leq\left(||u_{n}||^{2(2^{{\ast}}_{\alpha}-1)}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}+ ||u||^{2(2^{{\ast}}_{\alpha}-1)}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}\right) \cdot||u_{n}-u||^{2}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}. \end{equation}
    Thus,
    (2.22)\begin{align} A_{1}& \leq \left(||u_{n}||^{\frac{2^{{\ast}}_{\alpha}-1}{2^{{\ast}}_{\alpha}}}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}+||u||^{\frac{2^{{\ast}}_{\alpha}-1}{2^{{\ast}}_{\alpha}}}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}\right)\cdot||u_{n}-u||^{\frac{1}{2^{{\ast}}_{\alpha}}}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}\nonumber\\ & \qquad \cdot||\varphi||_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}. \end{align}
    Similarly, we have
    (2.23)\begin{equation} \begin{aligned} A_{2} & \leq \left(||u_{n}||^{\frac{(2^{{\ast}}_{\alpha}-1)^{2}}{2^{{\ast}}_{\alpha}}}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}+||u||^{\frac{(2^{{\ast}}_{\alpha}-1)^{2}}{2^{{\ast}}_{\alpha}}}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}\right)\nonumber\\ & \quad \cdot||u_{n}-u||^{\frac{2^{{\ast}}_{\alpha}-1}{2^{{\ast}}_{\alpha}}}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}\cdot||u_{n}||^{2^{{\ast}}_{\alpha}-1}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}\\ A_{3} & \leq||u||_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}||\varphi||_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}\\ A_{4} & \leq \left(||u_{n}||^{2^{{\ast}}_{\alpha}-2}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}+||u||^{2^{{\ast}}_{\alpha}-2}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}\right)\cdot||u_{n}-u||_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}\nonumber\\ & \quad \cdot||u||^{2^{{\ast}}_{\alpha}-1}_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}. \end{aligned} \end{equation}
    Therefore, for any $u_{n}\longrightarrow u$ in $Q^{\alpha,2^{\ast }_{\alpha }}(\mathbb {R}^{3},\mathbb {R}^{3})$, we have $\langle (I'(u_{n})-I'(u)),\varphi \rangle \longrightarrow 0$. This implies that $I(u)$ is $\mathcal {C}^{1}$. Therefore, $J_{\lambda }(u)$ and $J(u)$ are of class $\mathcal {C}^{1}$.

To apply the concentration compactness arguments, we need to introduce the following Coulomb–Sobolev space.

Definition 2.8 Let $\Omega \subset \mathbb {R}^{N}$, $\alpha \in (0,N)$ and $p\geq 1$. We define $W^{1,\alpha,p}(\Omega )$ as the scalar space of measurable functions $u:\Omega \longrightarrow \mathbb {R}$ such that $u \in Q^{\alpha,p}(\Omega )$ and $u$ is weakly differentiable in $\Omega$, $Du\in Q^{\alpha,p}(\Omega,\mathbb {R}^{N})$ and

\[ ||u||_{W^{1,\alpha,p}(\Omega)}=\left( \left(\int_{\Omega}|I_{\alpha/2}\ast|u|^{p}|^{2}\,{\rm d}x\right)^{\frac{1}{2}}+\left(\int_{\Omega}|I_{\alpha/2}\ast|Du|^{p}|^{2}\,{\rm d}x\right)^{\frac{1}{2}}\right)^{\frac{1}{p}}<{+}\infty. \]

We are going to prove that the Coulomb–Sobolev space $W^{1,\alpha,p}(\Omega )$ is a Banach space. Firstly, We have the following Fatou property for locally converging sequence.

Lemma 2.9 Let $N\in \mathbb {N}$, $\alpha \in (0,N)$ and $p\geq 1$. If $(u_{n})_{n\in \mathbb {N}}$ is a bounded sequence in $W^{1,\alpha,p}(\Omega )$ such that $u_{n}\longrightarrow u$ in $L^{1}_{loc}(\Omega )$ and $Du_{n}\longrightarrow g$ in $L^{1}_{loc}(\Omega,\mathbb {R}^{N})$, then $g=Du$ and $u\in W^{1,\alpha,p}(\Omega )$,

(2.24)\begin{equation} \int_{\Omega}|I_{\alpha/2}\ast| u|^{p}|^{2}\,{\rm d}x\leq\mathop{\mathrm{lim\ inf}}\limits_{n\longrightarrow\infty}\int_{\Omega}|I_{\alpha/2}\ast| u_{n}|^{p}|^{2}\,{\rm d}x, \end{equation}

and

(2.25)\begin{equation} \int_{\Omega}|I_{\alpha/2}\ast|Du|^{p}|^{2}\,{\rm d}x\leq\mathop{\mathrm{lim\ inf}}\limits_{n\longrightarrow\infty}\int_{\Omega}|I_{\alpha/2}\ast|Du_{n}|^{p}|^{2}\,{\rm d}x. \end{equation}

Proof. The proof of (2.24) follows the same argument in the proof of (2.2). We are going to prove (2.25). For $v\in \mathcal {C}_{0}^{\infty }(\Omega )$ we conclude that

(2.26)\begin{equation} \int_{\Omega}u_{n}\ \mathrm{div}(v)\,{\rm d}x={-}\int_{\Omega}\nabla u_{n}\cdot v \,{\rm d}x. \end{equation}

Since $u_{n}\longrightarrow u$ in $L^{1}_{loc}(\Omega )$, we have

(2.27)\begin{equation} \int_{\Omega}u_{n}\ \mathrm{div}(v)\,{\rm d}x\longrightarrow\int_{\Omega}u\ \mathrm{div}(v)\,{\rm d}x. \end{equation}

Since $Du_{n}\longrightarrow g$ in $L^{1}_{loc}(\Omega,\mathbb {R}^{N})$, we have

(2.28)\begin{equation} -\int_{\Omega}\nabla u_{n}\cdot v\,{\rm d}x\longrightarrow -\int_{\Omega}g\cdot v\,{\rm d}x. \end{equation}

Setting $Du$ as the weak derivation of $u$ in the following distribute sense

(2.29)\begin{equation} \int_{\Omega}u\ \mathrm{div}(v)\,{\rm d}x={-}\int_{\Omega}Du\cdot v\,{\rm d}x, \end{equation}

we can see $g=Du\in L^{p}(\Omega,\mathbb {R}^{N})$ in the weak sense, and $Du_{n}\longrightarrow Du$ in $L^{1}_{loc}(\Omega,\mathbb {R}^{N})$. Based on this fact, we can obtain (2.25) by the same analysis in the proof of (2.2).

Lemma 2.10 Let $N\in \mathbb {N}$, $\alpha \in (0,N)$ and $p\geq 1$. The normed space $W^{1,\alpha,p}(\Omega )$ is complete.

Proof. Let $(u_{n})_{n\in \mathbb {N}}$ be a Cauchy sequence in $W^{1,\alpha,p}(\Omega )$. By the local estimate of the Coulomb energy, $(u_{n})_{n\in \mathbb {N}}$ and $(Du_{n})_{n\in \mathbb {N}}$ are also the Cauchy sequences in $L_{loc}^{p}(\Omega )$. Hence there exists $u\in L_{loc}^{p}(\Omega )$ such that $(u_{n})_{n\in \mathbb {N}}\longrightarrow u$ in $L_{loc}^{p}(\Omega )$ and $g\in L_{loc}^{p}(\Omega,\mathbb {R}^{N})$ such that $(Du_{n})_{n\in \mathbb {N}}\longrightarrow g$ in $L_{loc}^{p}(\Omega,\mathbb {R}^{N})$. In light of lemma 2.9, we conclude that $u\in W^{1,\alpha,p}(\Omega )$. Moreover, for every $n\in \mathbb {N}$ the sequence $(u_{n}-u_{m})_{m\in \mathbb {N}}$ converges to $(u_{n}-u)$ in $L_{loc}^{p}(\Omega )$. Hence, by lemma 2.9 again, we have

\begin{align*} & \mathop{\mathrm{lim\ sup}}\limits_{n\longrightarrow\infty}\left(\int_{\Omega}|I_{\alpha/2}\ast|Du_{n}-Du|^{p}|^{2}\,{\rm d}x+\int_{\Omega}|I_{\alpha/2}\ast|u_{n}-u|^{p}|^{2}\,{\rm d}x\right)\\ & \leq\mathop{\mathrm{lim\ sup}}\limits_{n\longrightarrow\infty}\mathop{\mathrm{lim\ sup}}\limits_{m\longrightarrow\infty}\left(\int_{\Omega}|I_{\alpha/2}\ast|Du_{n}-Du_{m}|^{p}|^{2}\,{\rm d}x+\int_{\Omega}|I_{\alpha/2}\ast|u_{n}-u_{m}|^{p}|^{2}\,{\rm d}x\right)\\ & \leq\mathop{\mathrm{lim\ sup}}\limits_{m,n\longrightarrow\infty}\left(\int_{\Omega}|I_{\alpha/2}\ast|Du_{n}-Du_{m}|^{p}|^{2}\,{\rm d}x+\int_{\Omega}|I_{\alpha/2}\ast|u_{n}-u_{m}|^{p}|^{2}\,{\rm d}x\right)\\ & \leq0. \end{align*}

This implies $W^{1,\alpha,p}(\Omega )$ is complete.

We show that the Coulomb–Sobolev space $W^{1,\alpha,p}(\Omega )$ can be naturally identified with the completion of the set of the test functions $\mathcal {C}_{0}^{\infty }(\Omega )$ under the norm $||\cdot ||_{W^{1,\alpha,p}}$.

Lemma 2.11 Let $N\in \mathbb {N}$, $\alpha \in (0,N)$ and $p\geq 1$. The space of test function $\mathcal {C}_{0}^{\infty }(\Omega )$ is dense in $W^{1,\alpha,p}(\Omega )$.

Proof. Since the test function $\mathcal {C}_{0}^{\infty }(\Omega )$ is dense in $Q^{\alpha,p}(\Omega )$, see proposition 2.6 in [Reference Mercuri, Moroz and Van Schaftingen34], then, by lemma 2.9 the conclusion also holds in $W^{1,\alpha,p}(\Omega )$.

Similar to the Poincaré inequality for the local case, we have the following Poincaré inequality for the nonlocal case.

Lemma 2.12 For all $N\in \mathbb {N}$ and $\alpha \in (0,N)$, there exist $p\in (\frac {N-\alpha }{2},\infty )$ if $\alpha \in (0,N-2)$, while $p\in [N,\infty )$ if $\alpha \in [N-2,N)$, such that for every $a\in \Omega$ and $\rho >0$

\[ \int_{B_{\rho}(a)}|I_{\alpha/2}\ast|u|^{p}|^{2}\,{\rm d}x\leq C\rho^{\frac{N-\alpha}{2}}\left(\int_{B_{\rho}(a)}|I_{\alpha/2}\ast|Du|^{p}|^{2}\,{\rm d}x\right)^{\frac{1}{p}}. \]

Proof. By the HLS inequality (1.11), we have

(2.30)\begin{equation} \int_{B_{\rho}(a)}|I_{\alpha/2}\ast|u|^{p}|^{2}\,{\rm d}x\leq C_{1}(\alpha,\rho,p,N)\left(\int_{B_{\rho}(a)}|u(x)|^{\frac{2Np}{N+\alpha}}\,{\rm d}x\right)^{\frac{N+\alpha}{N}}. \end{equation}

If $\alpha \in (0,N-2)$ and $p\in (\frac {N-\alpha }{2},N)\subset (1,N)$, then we have

(2.31)\begin{align} \left(\int_{B_{\rho}(a)}|u(x)|^{\frac{2Np}{N+\alpha}}\,{\rm d}x\right)^{\frac{N+\alpha}{N}}& \leq C_{2}(\alpha,\rho,p,N)\left(\int_{B_{\rho}(a)}|u(x)|^{\frac{Np}{N-p}}\,{\rm d}x\right)^{\frac{N-p}{Np}}\nonumber\\ & \leq C_{3}(\alpha,\rho,p,N)\left(\int_{B_{\rho}(a)}|Du|^{p}\,{\rm d}x\right)^{\frac{2}{p}}. \end{align}

On the other hand, if $\alpha \in (0,N-2)$ and $p\in [N,\infty )$, we know there exists $h\in [\frac {2Np}{N+\alpha +2p},N)$ such that

(2.32)\begin{align} \left(\int_{B_{\rho}(a)}|u(x)|^{\frac{2Np}{N+\alpha}}\,{\rm d}x\right)^{\frac{N+\alpha}{N}}& \leq C_{4}(\alpha,\rho,p,N)\left(\int_{B_{\rho}(a)}|Du|^{h}\,{\rm d}x\right)^{\frac{2}{h}}\nonumber\\ & \leq C_{5}(\alpha,\rho,p,N)\left(\int_{B_{\rho}(a)}|Du|^{p}\,{\rm d}x\right)^{\frac{2}{p}}, \end{align}

where the Hölder inequality was applied. Consequently, for $\alpha \in [N-2,N)$, there also exist $p\in [N,\infty )$ and $h\in [\frac {2Np}{N+\alpha +2p},N)$ such that (2.32) holds.

Then the conclusion follows from (2.30), (2.31), (2.32) and the local estimate of Coulomb energy [Reference Mercuri, Moroz and Van Schaftingen34, proposition 2.3], which says that

\[ \left(\int_{B_{\rho}(a)}|Du|^{p}\,{\rm d}x\right)^{\frac{2}{p}}\leq C\rho^{\frac{N-\alpha}{2}}\left(\int_{B_{\rho}(a)}|I_{\alpha/2}\ast|Du|^{p}|^{2}\,{\rm d}x\right)^{\frac{1}{p}}. \]

To establish the Helmholtz decomposition, we also define the following Coulomb–Sobolev space.

Definition 2.13 Let $\Omega \subset \mathbb {R}^{3}$, $\alpha \in (0,3)$ and $p\in (1,\infty )$. We define $W^{1,\alpha,p}_{0}(\mathbb {R}^{3})$ and $W^{1,\alpha,p}_{0}(\Omega )$ as the completion of $C_{0}^{\infty }(\mathbb {R}^{3})$ and $C_{0}^{\infty }(\Omega )$ with respect to the norm

\[ ||w||_{W^{1,\alpha,p}_{0}(\mathbb{R}^{3})}=|\nabla w|_{Q^{\alpha,p}(\mathbb{R}^{3})},\quad ||w||_{W^{1,\alpha,p}_{0}(\Omega)}=|\nabla w|_{Q^{\alpha,p}(\Omega)}. \]

Proposition 2.14 $W^{1,\alpha,p}_{0}(\mathbb {R}^{3})$ is linearly isometric to

\[ \nabla W^{1,\alpha,p}_{0}(\mathbb{R}^{3}):=\{\nabla w\in Q^{\alpha,p}(\mathbb{R}^{3},\mathbb{R}^{3}): w\in W^{1,\alpha,p}_{0}(\mathbb{R}^{3})\}, \]

and $W^{1,\alpha,p}_{0}(\Omega )$ is linearly isometric to

\[ \nabla W^{1,\alpha,p}_{0}(\Omega):=\{\nabla w\in Q^{\alpha,p}(\Omega,\mathbb{R}^{3}): w\in W^{1,\alpha,p}_{0}(\Omega)\}. \]

Proof. Set the map $\nabla :W^{1,\alpha,p}_{0}(\mathbb {R}^{3})\longrightarrow \nabla W^{1,\alpha,p}_{0}(\mathbb {R}^{3})$. Since the Coulomb space is complete, the map is obviously injective and surjective. We also easily check that the map is isometric by the definition of $W^{1,\alpha,p}_{0}(\mathbb {R}^{3})$, this implies our conclusion.

2.1.2 Helmholtz decomposition.

Let $\mathcal {D}^{1,2}(\mathbb {R}^{3},\mathbb {R}^{3})$ denote the completion of $\mathcal {C}^{\infty }_{0}(\mathbb {R}^{3},\mathbb {R}^{3})$ with respect to the norm $|\nabla \cdot |_{2}$. Recall the subspace $\mathcal {V}_{\mathbb {R}^{3}}$ and $\mathcal {W}_{\mathbb {R}^{3}}$ of $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\mathbb {R}^{3})$ in the introduction, we have the following Helmholtz decomposition on $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\mathbb {R}^{3})$.

Lemma 2.15 $\mathcal {V}_{\mathbb {R}^{3}}$ and $\mathcal {W}_{\mathbb {R}^{3}}$ are closed subspaces of $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\mathbb {R}^{3})$ and

(2.33)\begin{equation} W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\mathbb{R}^{3})=\mathcal{V}_{\mathbb{R}^{3}}\oplus\nabla W^{1,\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathbb{R}^{3})=\mathcal{V}_{\mathbb{R}^{3}}\oplus\mathcal{W}_{\mathbb{R}^{3}}.\quad(direct\ sum) \end{equation}

Moreover, $\mathcal {V}_{\mathbb {R}^{3}}\subset \mathcal {D}^{1,2}(\mathbb {R}^{3},\mathbb {R}^{3})$ and the norms $|\nabla \cdot |_{2}$ and $||\cdot ||_{W^{\alpha,2^{\ast }_{\alpha }}(\mathrm {curl};\mathbb {R}^{3})}$ are equivalent in $\mathcal {V}_{\mathbb {R}^{3}}$.

Proof. By the HLS inequality in proposition 1.1, there is a continuous embedding

\[ L^{2^{{\ast}}}(\mathbb{R}^{3},\mathbb{R}^{3})\hookrightarrow Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3}). \]

Then the conclusion follows from the argument in [Reference Mederski28, lemma 3.2]. Indeed, Since $W^{1,\alpha,2^{\ast }_{\alpha }}_{0}(\mathbb {R}^{3})$ is a complete space, then $\nabla W^{1,\alpha,2^{\ast }_{\alpha }}_{0}(\mathbb {R}^{3})$ is a closed subspace of $Q^{\alpha,2^{\ast }_{\alpha }}(\mathbb {R}^{3},\mathbb {R}^{3})$. Moreover $\mathrm {cl}\mathcal {V}_{\mathbb {R}^{3}}\cap \nabla W^{1,\alpha,2^{\ast }_{\alpha }}_{0}(\mathbb {R}^{3})=\{0\}$ in $Q^{\alpha,2^{\ast }_{\alpha }}(\mathbb {R}^{3},\mathbb {R}^{3})$, hence $\mathcal {V}_{\mathbb {R}^{3}}\cap \nabla W^{1,\alpha,2^{\ast }_{\alpha }}_{0}(\mathbb {R}^{3})=\{0\}$ in $W^{\alpha,2^{\ast }_{\alpha }}(\mathrm {curl};\mathbb {R}^{3})$. In view of the Helmholtz decomposition, and smooth function $\varphi \in \mathcal {C}_{0}^{\infty }(\mathbb {R}^{3},\mathbb {R}^{3})$ can be written as

\[ \varphi=\varphi_{1}+\nabla\varphi_{2} \]

such that $\varphi _{1}\in \mathcal {D}^{1,2}(\mathbb {R}^{3},\mathbb {R}^{3})\cap \mathcal {C}^{\infty }(\mathbb {R}^{3},\mathbb {R}^{3})$, $\mathrm {div}(\varphi _{1})=0$ and $\varphi _{2}\in \mathcal {C}^{\infty }(\mathbb {R}^{3})$ is the Newton potential of $\mathrm {div}(\varphi )$. Since $\varphi$ has compact support, then $\nabla \varphi _{2}\in L^{6}(\mathbb {R}^{3},\mathbb {R}^{3})\subset Q^{\alpha,2^{\ast }_{\alpha }}(\mathbb {R}^{3},\mathbb {R}^{3})$ and $\varphi _{1}=\varphi -\nabla \varphi _{2}\in \mathcal {V}_{\mathbb {R}^{3}}$. Observe that $\nabla \times \nabla \varphi _{1}=-\triangle \varphi _{1}$, hence

\[ |\nabla\times u|_{2}=|\nabla u|_{2}=||u||_{\mathcal{D}^{1,2}(\mathbb{R}^{3},\mathbb{R}^{3})} \]

for any $u\in \mathcal {V}_{\mathbb {R}^{3}}$. By the Sobolev embedding we have $\mathcal {V}_{\mathbb {R}^{3}}$ is continuously embedded in $L^{6}(\mathbb {R}^{3},\mathbb {R}^{3})$ and by the HLS inequality also in $Q^{\alpha,2^{\ast }_{\alpha }}(\mathbb {R}^{3},\mathbb {R}^{3})$. Therefore the norms $||\cdot ||_{\mathcal {D}^{1,2}(\mathbb {R}^{3},\mathbb {R}^{3})}$ and $||\cdot ||_{W^{\alpha,2^{\ast }_{\alpha }}}(\mathrm {curl};\mathbb {R}^{3})$ are equivalent on $\mathcal {V}_{\mathbb {R}^{3}}$ and by the density argument we get the decomposition (2.33).

For the bounded domains case, we recall the definition of $\mathcal {V}'_{\Omega }$ in [Reference Bartsch and Mederski5], that is

\[ \mathcal{V}'_{\Omega}=\{v\in W^{2}_{0}(\mathrm{curl};\Omega):\int_{\Omega}\langle v,\varphi\rangle\,{\rm d}x=0\ \mathrm{for\ every}\ \varphi\in C^{\infty}_{0}(\Omega,\mathbb{R}^{3})\ \mathrm{with}\ \nabla\times\varphi=0\}. \]

Indeed, if $\varphi$ is supported in a ball, we have $\varphi =\nabla \psi$ for some $\psi \in \mathcal {C}_{0}^{\infty }(\Omega )$, hence we have $\mathrm {div}(v)=0$ for $v\in \mathcal {V}'_{\Omega }$. This implies that

\begin{align*} \mathcal{V}'_{\Omega}& =\{v\in W_{0}^{2}(\mathrm{curl};\Omega):\mathrm{div}(v)=0\ \mathrm{in\ the\ sense\ of\ distributions}\}\\ & \subset \{v\in W^{2}_{0}(\mathrm{curl};\Omega):\mathrm{div}(v)\in L^{2}(\Omega,\mathbb{R}^{3})\}=:X_{N}(\Omega). \end{align*}

Furthermore, since $\Omega$ is a bounded domain, either convex or with $\mathcal {C}^{1,1}$ boundary, $X_{N}(\Omega )$ is continuously embedded in $H^{1}(\Omega,\mathbb {R}^{3})$, see [Reference Amrouche, Bernardi, Dauge and Girault1]. Therefore in view of the Rellich's theorem $\mathcal {V}'_{\Omega }$ is compactly embedded in $L^{2}(\Omega,\mathbb {R}^{3})$ and continuously in $L^{6}(\Omega,\mathbb {R}^{3})$, so is $Q^{\alpha,2^{\ast }_{\alpha }}(\Omega,\mathbb {R}^{3})$. This implies in particular that $\mathcal {V}'_{\Omega }\subset \mathcal {V}_{\Omega }$. On the other hand, since $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )\subset W^{2}_{0}(\mathrm {curl};\Omega )$, we have $\mathcal {V}_{\Omega }\subset \mathcal {V}'_{\Omega }$. Therefore, we can see that $\mathcal {V}_{\Omega }=\mathcal {V}'_{\Omega }$ is a Hilbert space with inner product

\[ (v,z)=\int_{\Omega}\langle \nabla\times v,\nabla\times z\rangle\,dx=\int_{\Omega}\langle\nabla v,\nabla z\rangle\,dx. \]

Also, one can easily observe that $\mathcal {V}_{\Omega }$ is a closed linear subspace of $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )$. Therefore, by theorem 4.21 (c) in [Reference Kirsch and Hettlich21], we have the following Helmholtz decomposition

(2.34)\begin{equation} W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\Omega)=\mathcal{V}_{\Omega}\oplus\mathcal{W}_{\Omega}.\quad (\mathrm{direct\ sum}) \end{equation}

and that

(2.35)\begin{equation} \int_{\Omega}\langle v,w\rangle\,{\rm d}x=0\ \mathrm{if}\ v\in\mathcal{V}_{\Omega},\ w\in \mathcal{W}_{\Omega}, \end{equation}

which means that $\mathcal {V}_{\Omega }$ and $\mathcal {W}_{\Omega }$ are orthogonal in $L^{2}(\Omega,\mathbb {R}^{3})$. Then the norm

\[ ||v+w||:=((v,v)+|w|_{Q^{\alpha,2^{{\ast}}_{\alpha}}}^{2})^{\frac{1}{2}},~v\in\mathcal{V}_{\Omega},\ w\in \mathcal{W}_{\Omega} \]

is equivalent to $||\cdot ||_{W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )}$.

For the setting of boundary condition, according to [Reference Monk35, theorem 3.33], there is a continuous tangential trace operator $\gamma _{t}:W^{2}(\mathrm {curl};\Omega )\longrightarrow H^{-1/2}(\partial \Omega )$ such that

\[ \gamma_{t}(u)=\nu\times u|_{\partial\Omega}\quad\mathrm{for}\ \mathrm{any}\ u\in C^{\infty}(\overline{\Omega},\mathbb{R}^{3}) \]

and

\[ W^{2}_{0}(\mathrm{curl};\Omega)=\{u\in W^{2}(\mathrm{curl};\Omega):\gamma_{t}(u)=0\}. \]

Hence the vector field $u\in W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )=\mathcal {V}_{\Omega }\oplus \mathcal {W}_{\Omega }\subset W^{2}_{0}(\mathrm {curl};\Omega )$ satisfies the boundary condition in (1.10).

On the other hand, $\mathcal {W}_{\Omega }$ contains all gradient vectors fields, i.e. $\nabla W^{1,\alpha,2^{\ast }_{\alpha }}_{0}(\Omega )\subset \mathcal {W}_{\Omega }$. However, for some general domains, $\{w\in \mathcal {W}_{\Omega }:\mathrm {div}(w)=0\}$ may be nontrivial (harmonic field) and hence $\nabla W^{1,\alpha,2^{\ast }_{\alpha }}_{0}(\Omega )\subsetneq \mathcal {W}_{\Omega }$, see [Reference Bartsch and Mederski6, pp. 4314–4315]. While in the topology domains as we supposed, we have the following conclusion, which is a trivial extended from Lemma 2.3 in [Reference Mederski and Szulkin33].

Lemma 2.16 There holds $\mathcal {W}_{\Omega }\!=\!W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )\cap \mathcal {W}_{\mathbb {R}^{3}}\!=\!W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )\cap \nabla W^{1,\alpha,2^{\ast }_{\alpha }} (\Omega )$. If $\partial \Omega$ is connected, then $\mathcal {W}_{\Omega }=\nabla W^{1,\alpha,2^{\ast }_{\alpha }}_{0}(\Omega )$. If $\Omega$ is unbounded, $\mathcal {W}_{\Omega }=W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )\cap \mathcal {W}_{\mathbb {R}^{3}}$ still holds.

2.2. Concentration–compactness lemma

In view of the Helmholtz decomposition, the work space is decomposed into a Hilbert space $\mathcal {V}_{\Omega }$ and a Banach space $\mathcal {W}_{\Omega }$. For a bounded sequence in the work space, one can obtain the a.e convergenc in $\mathcal {V}_{\Omega }$ by the Rellich compactness theorem, which is important to the weak-weak$^{\ast }$ continuity of $J'(u)$. While in the subspace $\mathcal {W}_{\Omega }$, $w_{n}=\nabla p_{n}\rightharpoonup \nabla p=w$ can not deduce the a.e convergenc. By setting the convex nonlinearity satisfied the coercive condition, Merderski [Reference Mederski and Szulkin33] connected the subspaces $\mathcal {V}_{\Omega }$ and $\mathcal {W}_{\Omega }$ by the global minimum argument, then the a.e. convergenc on $\mathcal {W}_{\Omega }$ can be recovered by the second concentration–compactness lemma, see Lions [Reference Lions25]. Since the nonlinearity becomes a nonlocal term, we make some minor modifications to the concentration–compactness lemma.

In this subsection, We work in some subspaces of $Q^{\alpha,2^{\ast }_{\alpha }}(\Omega,\mathbb {R}^{3})$ and $Q^{\alpha,2^{\ast }_{\alpha }}(\mathbb {R}^{3},\mathbb {R}^{3})$. Let $Z\subset \mathcal {V}_{\Omega }$ be a finite-dimension subspace of $Q^{\alpha,2^{\ast }_{\alpha }}(\Omega,\mathbb {R}^{N})$ such that $Z\cap \mathcal {W}_{\Omega }=\{0\}$ and put

\[ \widetilde{\mathcal{W}}:=\mathcal{W}_{\Omega}\oplus Z. \]

Correspondingly, in $\mathbb {R}^{3}$, we put $Z=\{0\}$ and $\widetilde {\mathcal {W}}=\mathcal {W}_{\mathbb {R}^{3}}$. For simplicity, we only show the discussion on bounded domains $\Omega$, and the case in the entire space $\mathbb {R}^{3}$ is similar. Note that we always assume that $v\in \mathcal {V}_{\mathbb {R}^{3}}\subset \mathcal {D}^{1,2}(\mathbb {R}^{3},\mathbb {R}^{3})$ but not $\mathcal {V}_{\Omega }$, we then have

Lemma 2.17 Assume $F(u)=(I_{\alpha }\ast |u|^{2^{\ast }_{\alpha }})|u|^{2^{\ast }_{\alpha }}$ and $f(u)=\partial _{u}F(u)$, then $F(u)$ is uniformly strictly convex with respect to $u\in \mathbb {R}^{N}$, i.e. for any compact $A\subset (\mathbb {R}^{3}\times \mathbb {R}^{3})\setminus \{(u,u):u\in \mathbb {R}^{3}\}$

(2.36)\begin{equation} \mathop{\mathrm{inf}}\limits_{x\in\Omega,(u_{1},u_{2})\in A}\left(\frac{1}{2}\left(F(u_{1})+F(u_{2})\right)-F\left(\frac{u_{1}+u_{2}}{2}\right)\right)>0; \end{equation}

Moreover, for any $v\in \mathcal {V}_{\mathbb {R}^{3}}$ we find a unique $\widetilde {w}_{\Omega }(v)\in \widetilde {\mathcal {W}}$ such that

(2.37)\begin{equation} \int_{\Omega}F(v+\widetilde{w}_{\Omega}(v))\,{\rm d}x\leq\int_{\Omega}F(v+\widetilde{w})\,{\rm d}x \quad\mathrm{for}\ \mathrm{all}\ \widetilde{w}\in\widetilde{\mathcal{W}}. \end{equation}

In other word,

(2.38)\begin{equation} \int_{\Omega}\langle f(v+\widetilde{w}),\zeta\rangle\,{\rm d}x=0\ \mathrm{for}\ \mathrm{all}\ \zeta\in\widetilde{\mathcal{W}}\ \mathrm{if}\ \mathrm{and}\ \mathrm{only}\ \mathrm{if}\ \widetilde{w}=\widetilde{w}_{\Omega}(v). \end{equation}

Proof. The uniformly convexity of $F(u)$ follows from the proposition 2.8 in [Reference Mercuri, Moroz and Van Schaftingen34]. Now, we prove that $F(u)$ is strictly convex. Set $I(u)=\int _{\mathbb {R}^{3}}F(u)\,{\rm d}x$ and $u(x)=(u_{1},u_{2},u_{3})$, then for any $(s_{1},s_{2},s_{3})\in \mathbb {R}^{3}$ we have

\begin{align*} I(u)& =\int_{\Omega}|I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x=\int_{\Omega}\left[I_{\alpha/2}\ast\left(\mathop{\sum}\limits_{i=1}^{3}s_{i}|u_{i}|^{2^{{\ast}}_{\alpha}}\right)\right]^{2}dx \nonumber\\ & =\int_{\Omega}\left[\mathop{\sum}\limits_{i=1}^{3}s_{j}\left(I_{\alpha/2}\ast |u_{i}|^{2^{{\ast}}_{\alpha}}\right)\right]^{2}\,{\rm d}x \end{align*}

Set

\[ g(s_{1},s_{2},s_{3})=\left[\mathop{\sum}\limits_{i=1}^{3}s_{j}\left(I_{\alpha/2}\ast |u_{i}|^{2^{{\ast}}_{\alpha}}\right)\right]^{2}=h(L(s_{1},s_{2},s_{3})), \]

where $h(t)=t^{2}$ ia a strict convex function and

\[ L(s_{1},s_{2},s_{3})=\mathop{\sum}\limits_{i=1}^{3}s_{j}\left(I_{\alpha/2}\ast |u_{i}|^{2^{{\ast}}_{\alpha}}\right). \]

is a linear functional. Then, for each $x\in \mathbb {R}^{3}$, $g(s_{1},s_{2},s_{3})$ is convex.

Indeed, fix $\lambda \in (0,1)$ and $(s_{1},s_{2},s_{3}),(r_{1},r_{2},r_{3})\in \mathbb {R}^{3}$, we have

\begin{align*} & g((1-\lambda)(s_{1},s_{2},s_{3})+\lambda(r_{1},r_{2},r_{3})) =h(L((1-\lambda)(s_{1},s_{2},s_{3})+\lambda(r_{1},r_{2},r_{3})))\\ & =h((1-\lambda)L(s_{1},s_{2},s_{3})\!+\!\lambda L(r_{1},r_{2},r_{3}))\leq(1\!-\!\lambda)h(L(s_{1},s_{2},s_{3}))\!+\!\lambda h(L(r_{1},r_{2},r_{3}))\\ & =(1-\lambda)g(s_{1},s_{2},s_{3})+\lambda g(r_{1},r_{2},r_{3}). \end{align*}

Moreover, since $L$ is an injective function, we deduce that $g$ is strictly convex. Hence, $I(u)$ is strictly convex, so is $F(u)$. On the other hand, $I(u)$ is coercive in $Q^{\alpha,2^{\ast }_{\alpha }}(\Omega,\mathbb {R}^{3})$. Then, by the global minimum theorem, we have (2.37) and (2.38).

Denote the space of finite measures in $\mathbb {R}^{3}$ by $\mathcal {M}(\mathbb {R}^{3})$. Then we have the following concentration–compactness lemma, see [Reference Mederski and Szulkin33, lemma 3.1] for the local case.

Lemma 2.18 Assume $F(u)=(I_{\alpha }\ast |u|^{2^{\ast }_{\alpha }})|u|^{2^{\ast }_{\alpha }}$. Suppose $(v_{n})\subset \mathcal {V}_{\mathbb {R}^{3}}$, $v_{n}\rightharpoonup v_{0}$ in $\mathcal {V}_{\mathbb {R}^{3}}$, $v_{n}\longrightarrow v_{0}$ a.e. in $\mathbb {R}^{3}$, $|\nabla v_{n}|^{2}\rightharpoonup \mu$ and $(I_{\alpha }\ast |v_{0}|^{2^{\ast }_{\alpha }})|v_{0}|^{2^{\ast }_{\alpha }}\rightharpoonup \rho$ in $\mathcal {M}(\mathbb {R}^{3})$. Then there exists an at most countable set $I\subset \mathbb {R}^{3}$ and nonnegative weights $\{\mu _{x}\}_{x\in I}$, $\{\rho _{x}\}_{x\in I}$ such that

\[ \mu\geq |\nabla v_{0}|^{2}+\mathop{\Sigma}\limits_{x\in I}\mu_{x}\delta_{x},\quad\rho=\left(I_{\alpha}\ast|v_{0}|^{2^{{\ast}}_{\alpha}}\right)|v_{0}|^{2^{{\ast}}_{\alpha}}+\mathop{\Sigma}\limits_{x\in I}\rho_{x}\delta_{x}, \]

and passing to a subsequence, $\widetilde {w}_{\Omega }(v_{n})\rightharpoonup \widetilde {w}_{\Omega }(v_{0})$ in $\widetilde {\mathcal {W}}$, $\widetilde {w}_{\Omega }(v_{n})\longrightarrow \widetilde {w}_{\Omega }(v_{0})$ a.e. in $\Omega$ and in $L^{p}_{loc}(\Omega )$ for any $1\leq p\leq 2^{\ast }_{\alpha }$.

Remark 2.19 If $\Omega =\mathbb {R}^{3}$, $\widetilde {\mathcal {W}}=\mathcal {W}_{\mathbb {R}^{3}}$, we have the same conclusion, that is $\widetilde {w}_{\mathbb {R}^{3}}(v_{n})\rightharpoonup \widetilde {w}_{\mathbb {R}^{3}}(v_{0})$ in $\widetilde {\mathcal {W}}$, $\widetilde {w}_{\mathbb {R}^{3}}(v_{n})\longrightarrow \widetilde {w}_{\mathbb {R}^{3}}(v_{0})$ a.e. in $\mathbb {R}^{3}$.

Proof. Step 1. Let $\varphi \in \mathcal {C}^{\infty }_{0}(\mathbb {R}^{3})$, then by the definition of $S_{HL}$ in (1.18), we have

\[ \left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|\varphi(v_{n}-v_{0})|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}} S_{HL}\leq \int_{\mathbb{R}^{3}}|\nabla(\varphi (v_{n}-v_{0}))|^{2}\,{\rm d}x. \]

This means that

\begin{align*} & \left(\int_{\mathbb{R}^{3}}|\varphi|^{2\cdot2^{{\ast}}_{\alpha}}\left(I_{\alpha}\ast|v_{n}-v_{0}|^{2^{{\ast}}_{\alpha}}\right)|(v_{n}-v_{0})|^{2^{{\ast}}_{\alpha}}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}} S_{HL}\nonumber\\ & \quad\leq \int_{\mathbb{R}^{3}}|\varphi|^{2}|\nabla(v_{n}-v_{0})|^{2}\,{\rm d}x+o(1). \end{align*}

Using the Brezis-Lieb lemma for the nonlocal case on the left-hand side, see [Reference Gao and Yang19, pp. 1226], we then obtain

(2.39)\begin{equation} \left(\int_{\mathbb{R}^{3}}|\varphi|^{2\cdot 2^{{\ast}}_{\alpha}}\,{\rm d}\overline{\rho}\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}S_{HL}\leq \left(\int_{\mathbb{R}^{3}}|\varphi|^{2}\,{\rm d}\overline{\mu}\right)^{1/2}, \end{equation}

where $\overline {\mu }:=\mu -|\nabla v_{0}|^{2}$ and $\overline {\rho }=\rho -(I_{\alpha }\ast |v_{0}|^{2^{\ast }_{\alpha }})|v_{0}|^{2^{\ast }_{\alpha }}$. Set $I=\{x\in \mathbb {R}^{3}:\mu (\{x\}) >0\}$. Since $\mu$ is finite and $\mu,\overline {\mu }$ have the same singular set, $I$ is at most countable and $\mu \geq |\nabla v_{0}|^{2}+\Sigma _{x\in I}\mu _{x}\delta _{x}$. As in the proof of lemma 2.5 in [Reference Gao, Silva, Yang and Zhou18] it follows from (2.39) that $\overline {\rho }=\Sigma _{x\in I}\rho _{x}\delta _{x}$. So $\mu$ and $\rho$ are as claimed.

Step 2. To recover the a.e. convergence of the sequence on $\mathcal {W}_{\Omega }$, we consider the global minimum argument which connects $\mathcal {V}_{\mathbb {R}^{3}}$ and $\mathcal {W}_{\Omega }$. Using (2.37) we infer that

(2.40)\begin{equation} |v_{n}+\widetilde{w}_{\Omega}(v_{n})|^{2\cdot2^{{\ast}}_{\alpha}}_{Q^{\alpha,2^{{\ast}}_{\alpha}}}\leq\int_{\Omega}F(v_{n}+\widetilde{w}_{\Omega}(v_{n}))\mathrm{d}x\leq\int_{\Omega}F(v_{n})\,{\rm d}x\leq |v_{n}|_{Q^{\alpha,2^{{\ast}}_{\alpha}}}^{2\cdot2^{{\ast}}_{\alpha}}. \end{equation}

Since the right-hand side above is bounded, so is $(|\widetilde {w}_{\Omega }(v_{n})|_{Q^{\alpha,2^{\ast }_{\alpha }}})$. Hence, by the uniform convexity and reflexivity of Coulomb space, see [Reference Mercuri, Moroz and Van Schaftingen34, § 2.4.1], up to a subsequence, $\widetilde {w}_{\Omega }(v_{n})\rightharpoonup \widetilde {w}_{0}$ for some $\widetilde {w}_{0}$.

In the following we are going to prove that $\widetilde {w}_{\Omega }(v_{n})\longrightarrow \widetilde {w}_{0}$ a.e. in $\Omega$ after taking subsequence. The convexity of $F$ in $u$ implies that

\[ F\left(\frac{u_{1}+u_{2}}{2}\right)\geq F(u_{1})+\left\langle f(u_{1}),\frac{u_{2}-u_{1}}{2}\right\rangle, \]

applying (2.36), we obtain for any $k\geq 1$ and $|u_{1}-u_{2}|\geq \frac {1}{k}$, $|u_{1}|,|u_{2}|\leq k$ that

(2.41)\begin{equation} m_{k}\leq\frac{1}{2}(F(u_{1})+F(u_{2}))-F\left(\frac{u_{1}+u_{2}}{2}\right)\leq\frac{1}{4}\langle f(u_{1})-f(u_{2}),u_{1}-u_{2}\rangle, \end{equation}

where

\begin{align*} m_{k}& :=\mathop{\mathrm{inf}}\limits_{x\in\Omega,u_{1},u_{2}\in\mathbb{R}^{3}}\frac{1}{2}(F(u_{1})+F(u_{2}))-F\left(\frac{u_{1}+u_{2}}{2}\right)>0\quad\mathrm{for}\ \frac{1}{k}\\ & \quad \leq|u_{1}-u_{2}|,~|u_{1}|,|u_{2}|\leq k. \end{align*}

Now we decompose by $\widetilde {w}_{\Omega }(v_{n})=w_{n}+z_{n}$, $\widetilde {w}_{0}=w_{0}+z_{0}$ where $w_{n}$, $w_{0}\in \mathcal {W}_{\Omega }$ and $z_{n}$, $z_{0}\in Z$. Obviously, since $Z$ is a finite dimension space, we may assume $z_{n}\longrightarrow z_{0}$ in $Z$ and a.e. in $\Omega$. Notice that $v_{n}+\widetilde {w}_{\Omega }(v_{n})$ is bounded in $Q^{\alpha,2^{\ast }_{\alpha }}(\Omega,\mathbb {R}^{3})$, we may introduce

(2.42)\begin{align} \Omega_{n,k}& :=\{x\in\Omega:|v_{n}+\widetilde{w}_{\Omega}(v_{n})-v_{0}-w_{0}-z_{0}|\nonumber\\ & \quad\geq\frac{1}{k}~\mathrm{and}~|v_{n}+\widetilde{w}_{\Omega}(v_{n})|,|v_{0}+w_{0}+z_{0}|\leq k\}. \end{align}

Then, by (2.42) and (2.41), we have

(2.43)\begin{align} & 4m_{k}\int_{\Omega_{n,k}}|\varphi|^{2\cdot2^{{\ast}}_{\alpha}}\,{\rm d}x\nonumber\\ & \leq\int_{\Omega}|\varphi|^{2\cdot2^{{\ast}}_{\alpha}}\langle f(v_{n}+\widetilde{w}_{\Omega}(v_{n}))-f(v_{0}+w_{0}+z_{0}),v_{n}+\widetilde{w}_{\Omega}(v_{n})-v_{0}-w_{0}-z_{0} \rangle\,{\rm d}x\nonumber\\ & =\int_{\Omega}|\varphi|^{2\cdot2^{{\ast}}_{\alpha}}\langle f(v_{n}+\widetilde{w}_{\Omega}(v_{n}))-f(v_{0}+w_{0}+z_{0}),v_{n}-v_{0} \rangle\,{\rm d}x\nonumber\\ & \quad+\int_{\Omega}|\varphi|^{2\cdot2^{{\ast}}_{\alpha}}\langle f(v_{n}+\widetilde{w}_{\Omega}(v_{n}))-f(v_{0}+w_{0}+z_{0}),\widetilde{w}_{\Omega}(v_{n})-w_{0}-z_{0}\rangle {\rm d}x=I_{1}+I_{2}. \end{align}

Since $|v_{n}+\widetilde {w}_{\Omega }(v_{n})|\leq k$ and $|v_{0}+w_{0}+z_{0}|\leq k$ on $\Omega _{n,k}$, we have $|v_{n}+\widetilde {w}_{\Omega }(v_{n})|\leq C_{1}|v_{n}|$ and $|v_{0}+w_{0}+z_{0}|\leq C_{2}|v_{0}|$. Then, by the similar estimation in (iii) of lemma 2.7 we have

(2.44)\begin{align} I_{1}& =\int_{\Omega}|\varphi|^{2\cdot2^{{\ast}}_{\alpha}}\left\langle (I_{\alpha}\ast|v_{n}+\widetilde{w}_{\Omega}(v_{n})|^{2^{{\ast}}_{\alpha}})|v_{n}+\widetilde{w}_{\Omega}(v_{n})|^{2^{{\ast}}_{\alpha}-2}(v_{n}+\widetilde{w}_{\Omega}(v_{n}))\right.\nonumber\\ & \quad\left.-(I_{\alpha}\ast|v_{0}+w_{0}+z_{0}|^{2^{{\ast}}_{\alpha}})|v_{0}+w_{0}+z_{0}|^{2^{{\ast}}_{\alpha}-2}(v_{0}+w_{0}+z_{0}),v_{n}-v_{0} \right\rangle {\rm d}x\nonumber\\ & =\int_{\Omega}|\varphi|^{2\cdot2^{{\ast}}_{\alpha}}\left\langle (I_{\alpha}\ast|v_{n}+\widetilde{w}_{\Omega}(v_{n})|^{2^{{\ast}}_{\alpha}})|v_{n}+\widetilde{w}_{\Omega}(v_{n})|^{2^{{\ast}}_{\alpha}-2}(v_{n}+z_{n})\right.\nonumber\\ & \quad\left.-(I_{\alpha}\ast|v_{0}+w_{0}+z_{0}|^{2^{{\ast}}_{\alpha}})|v_{0}+w_{0}+z_{0}|^{2^{{\ast}}_{\alpha}-2}(v_{0}+z_{0}),v_{n}-v_{0} \right\rangle {\rm d}x\nonumber\\ & \leq C\int_{\Omega}|\varphi|^{2\cdot2^{{\ast}}_{\alpha}}\left\langle (I_{\alpha}\ast|v_{n}|^{2^{{\ast}}_{\alpha}})|v_{n}|^{2^{{\ast}}_{\alpha}-2}v_{n}-(I_{\alpha}\ast|v_{0}|^{2^{{\ast}}_{\alpha}})|v_{0}|^{2^{{\ast}}_{\alpha}-2}v_{0},v_{n}-v_{0} \right\rangle {\rm d}x\nonumber\\ & \leq C\left(\int_{\Omega}|\varphi|^{2\cdot2^{{\ast}}_{\alpha}}(I_{\alpha}\ast|v_{n}-v_{0}|^{2^{{\ast}}_{\alpha}})|v_{n}-v_{0}|^{2^{{\ast}}_{\alpha}}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}} =C\left(\int_{\Omega}|\varphi|^{2\cdot2^{{\ast}}_{\alpha}}\,{\rm d}\overline{\rho}\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}. \end{align}

where we use the fact that $Z$ is a finite dimension space and $\int _{\Omega }\langle v,w\rangle \,{\rm d}x=0$ see (4.2).

Next, we are going to show that $I_{2}=o(1)$. Fix $l\geq 1$. In view of lemmas 2.11, 2.12 and lemma 1.1 in [Reference Leinfelder23], there exists $\xi _{n}\in W^{1,\alpha,2^{\ast }_{\alpha }}(B_{l})$ such that $w_{n}=\nabla \xi _{n}$ and we may assume without loss of generality that $\int _{B_{l}}\xi _{n}\,{\rm d}x=0$. Then by the Poincaré inequality in lemma 2.12

\[ ||\xi_{n}||_{W^{1,\alpha,2^{{\ast}}_{\alpha}}(B_{l})}\leq C|w_{n}|_{Q^{\alpha,2^{{\ast}}_{\alpha}}(B_{l},\mathbb{R}^{3})}\leq C|w_{n}|_{Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3})}, \]

and passing to a subsequence, $\xi _{n}\rightharpoonup \xi$ for some $\xi \in W^{1,\alpha,2^{\ast }_{\alpha }}(B_{l})$. So by the natural compactly embedding, $\xi _{n}\longrightarrow \xi$ in $Q^{\alpha,2^{\ast }_{\alpha }}(B_{l})$. Now take any $\varphi \in C^{\infty }_{0}(B_{l})$. Since $\nabla (|\varphi |^{2\cdot 2^{\ast }_{\alpha }}(\xi _{n}-\xi ))\in \widetilde {\mathcal {W}}$, in view of (2.38) we get

\[ \int_{\Omega}\langle f(v_{n}+\widetilde{w}_{\Omega}(v_{n})),\nabla(|\varphi|^{2\cdot2^{{\ast}}_{\alpha}}(\xi_{n}-\xi))\rangle\,{\rm d}x=0. \]

That is

\begin{align*} & \int_{\Omega}|\varphi|^{2\cdot2^{{\ast}}_{\alpha}}\langle f(v_{n}+\widetilde{w}_{\Omega}(v_{n})),w_{n}-\nabla\xi\rangle\,{\rm d}x\nonumber\\ & \quad=\int_{\Omega}\langle f(v_{n}+\widetilde{w}_{\Omega}(v_{n})),\nabla(|\varphi|^{2\cdot2^{{\ast}}_{\alpha}}) (\xi-\xi_{n})\rangle\,{\rm d}x \end{align*}

where the right-hand side tends to 0 as $n\longrightarrow \infty$. Since $w_{n}\rightharpoonup \nabla \xi$ in $Q^{\alpha,2^{\ast }_{\alpha }}(B_{l})$,

\[ \int_{\Omega}|\varphi|^{2\cdot2^{{\ast}}_{\alpha}}\langle f(v_{0}+\nabla\xi+z_{0}),w_{n}-\nabla\xi\rangle\,{\rm d}x=o(1). \]

Hence, recalling that $\widetilde {w}_{\Omega }(v_{n})=w_{n}+z_{n}$ and $z_{n}\longrightarrow z_{0}$, we obtain

(2.45)\begin{equation} I_{2}=\int_{\Omega}|\varphi|^{2\cdot2^{{\ast}}_{\alpha}}\langle f(v_{n}+\widetilde{w}_{\Omega}(v_{n}))-f(v_{0}+\nabla\xi+z_{0}),\widetilde{w}_{\Omega}(v_{n})-\nabla\xi-z_{0}\rangle\,{\rm d}x=o(1). \end{equation}

Since $\varphi \in C^{\infty }_{0}(B_{l})$ is arbitrary, it follows from (2.43) and (2.45) that

(2.46)\begin{equation} 4m_{k}|\Omega_{n,k}\cap E|\leq(\overline{\rho}(E))^{1/2^{{\ast}}_{\alpha}}+o(1) \end{equation}

for any Borel set $E\subset B_{l}$. On the other hand, we can find an open set $E_{k}\supset I$ such that $|E_{k}|<\frac {1}{2^{k+1}}$. Then, taking $E=B_{l}\setminus E_{k}$ in (2.46), we have $4m_{k}|\Omega _{n,k}\cap (B_{l}\setminus E_{k})|=o(1)$ as $n\longrightarrow \infty$ because $\mathrm {supp}(\overline {\rho })\subset I$; hence we can find a sufficiently large $n_{k}$ such that $|\Omega _{n_{k},k}\cap B_{l}|<\frac {1}{2^{k}}$ and we obtain

\[ |\mathop{\cap}\limits_{j=1}^{\infty}\mathop{\cup}\limits_{k=j}^{\infty}\Omega_{n_{k},k}\cap B_{l}|\leq \mathop{\mathrm{lim}}\limits_{j\longrightarrow\infty}\mathop{\Sigma}\limits_{k=j}^{\infty}|\Omega_{n_{k},k}\cap B_{l}|\leq\mathop{\mathrm{lim}}\limits_{j\longrightarrow\infty}\frac{1}{2^{j-1}}=0. \]

According to the fact that $\widetilde {w}_{\Omega }(v_{n})\rightharpoonup \widetilde {w}_{0}$, one can employ the diagonal procedure and hence find a subsequence of $\widetilde {w}_{\Omega }(v_{n})$ which converges to $\widetilde {w}_{0}$ a.e. in $\Omega =\cup _{l=1}^{\infty }B_{l}$.

Let $p\in [1,2^{\ast }_{\alpha }]$. For $\Omega '\subset \Omega$ such that $|\Omega '|<+\infty$ we have

\begin{align*} & \int_{\Omega'}|v_{n}-v_{0}+\widetilde{w}_{\Omega}(v_{n})-\widetilde{w}_{0}|^{p}\,{\rm d}x\leq |\Omega'|^{1-\frac{p}{2^{{\ast}}_{\alpha}}} \left(\int_{\Omega}|v_{n}-v_{0}+\widetilde{w}_{\Omega}(v_{n})-\widetilde{w}_{0}|^{2^{{\ast}}_{\alpha}}\,{\rm d}x\right)^{\frac{p}{2^{{\ast}}_{\alpha}}}\\ & \leq |\Omega'|^{1-\frac{p}{2^{{\ast}}_{\alpha}}}|\mathrm{diam}\Omega|^{\frac{3-\alpha}{2}\cdot\frac{p}{2^{{\ast}}_{\alpha}}}\left(\int_{\Omega}|I_{\alpha/2}\ast|v_{n}-v_{0}+\widetilde{w}_{\Omega}(v_{n})-\widetilde{w}_{0}|^{2^{{\ast}}_{\alpha}}|^{2}\mathrm{d}x\right)^{\frac{1}{2}\cdot\frac{p}{2^{{\ast}}_{\alpha}}}, \end{align*}

where $\mathrm {diam}\Omega =\mathop {max}\limits _{x,y\in \Omega }|x-y|$. Hence by the Vitali convergence theorem, $v_{n}-v_{0}+\widetilde {w}_{\Omega }(v_{n})-\widetilde {w}_{0}\longrightarrow 0$ in $L^{p}_{loc}(\Omega )$ after passing to a subsequence.

Step 3. We show that $\widetilde {w}_{\Omega }(v_{0})=\widetilde {w}_{0}$. Take any $\widetilde {w}\in \widetilde {\mathcal {W}}$ and observe that by the Vitali convergence theorem,

\[ 0=\int_{\Omega}\langle f(v_{n}+\widetilde{w}_{\Omega}(v_{n})),\widetilde{w}\rangle dx\longrightarrow \int_{\Omega}\langle f(v_{0}+\widetilde{w}_{0}),\widetilde{w}\rangle\,{\rm d}x, \]

up to a subsequence. Now (2.38) implies that $\widetilde {w}_{0} =\widetilde {w}_{\Omega }(v_{0})$ which completes the proof.

2.3. Abstract critical point theory

For readers convenient, we end this section with recalling the abstract critical point lemma, see [Reference Bartsch and Mederski5, § 4] and [Reference Mederski30, § 3] for more details. Let $X$ be a reflexive Banach space with norm $||\cdot ||$ and with a topological direct sum decomposition $X=X^+\oplus \widetilde {X}$, where $X^+$ is a Hilbert space with a scalar product. For $u\in X$ we denote by $u^+\in X^+$ and $\widetilde {u}\in \widetilde {X}$ the corresponding summands so that $u=u^++\widetilde {u}$. We may assume that $\langle u,v\rangle =||u||^{2}$ for any $u\in X^+$ and that $||u||^{2}=||u^+||^{2}+||\widetilde {u}||^{2}$. The topology $\mathcal {T}$ on $X$ is defined as the product of the norm topology in $X^+$ and the weak topology in $\widetilde {X}$. Thus $u_{n}\mathop {\longrightarrow }\limits ^{\mathcal {T}}u$ is equivalent to $u^+_{n}\longrightarrow u^+$ and $\widetilde {u}\rightharpoonup \widetilde {u}$.

Let $J\in \mathcal {C}^{1}(X,\mathbb {R})$ be a functional on $X$ of the form

\[ J(u)=\frac{1}{2}||u^+||-I(u)\ \mathrm{for}\ u=u^+{+}\widetilde{u}\in X^+{\oplus}\widetilde{X} \]

such that the following assumptions hold

  1. (A1) $I\in \mathcal {C}^{1}(X,\mathbb {R})$ and $I(u)\geq I(0)=0$ for any $u\in X$.

  2. (A2) $I$ is $\mathcal {T}$-sequentially lower semi-continuous: $u_{n}\mathop {\longrightarrow }\limits ^{\mathcal {T}}u\Longrightarrow \mathrm {lim}~\mathrm {inf}~I(u_{n})\geq I(u)$.

  3. (A3) If $u_{n}\mathop {\longrightarrow }\limits ^{\mathcal {T}} u$ and $I(u_{n})\longrightarrow I(u)$ then $u_{n}\longrightarrow u$.

  4. (A4) There exists $r>0$ such that $a:=\mathop {\mathrm {inf}}\limits _{u\in X^+:||u||=r}J(u)>0$.

  5. (B1) $||u^+||+I(u)\longrightarrow \infty$ as $||u||\longrightarrow \infty$.

  6. (B2) $I(t_{n}u_{n})/t_{n}^{2}\longrightarrow \infty$ if $t_{n}\longrightarrow \infty$ and $u^+_{n}\longrightarrow u^+$ for some $u^+\neq 0$ as $n\longrightarrow \infty$.

  7. (B3) $\frac {t^{2}-1}{2}I'(u)(u)+tI'(u)(v)+I(u)-I(tu+v)<0$ for every $u\in \mathcal {N},t>0,v\in X$ such that $u\neq tu+v$.

We defined the following Nehari–Pankov

\[ \mathcal{N}:=\{u\in X\setminus\widetilde{X}:J'(u)|_{\mathbb{R}u\oplus \widetilde{X}}=0\}. \]

Correspondingly, we defined the (PS)$^{\mathcal {T}}_{c}$ condition for $J$.

Definition 2.20 We say that $J$ satisfies the $(PS)_{c}^{\mathcal {T}}$ condition in $\mathcal {N}$ if every $(PS)_{c}$ sequence $(u_{n})\in \mathcal {N}$ has a subsequence which convergence in the $\mathcal {T}$ topology:

\[ u_{n}\in\mathcal{N},\quad J(u_{n})\longrightarrow 0,\quad J'(u_{n})\longrightarrow c\quad\Longrightarrow\quad u_{n}\mathop{\longrightarrow}\limits^{\mathcal{T}}u\in X~~along~a~subsequence. \]

We also recall the compactly perturbed problem with respect to another decomposition of $X$. Namely, suppose that

(2.47)\begin{equation} \widetilde{X}=X^{0}\oplus X^{1}, \end{equation}

where $X^{0},X^{1}$ are closed in $\widetilde {X}$, and $X^{0}$ is a Hilbert space. For $u\in \widetilde {X}$ we denote $u^{0}\in X^{0}$ and $u^{1}\in X^{1}$ the corresponding summands so that $u=u^{0}+u^{1}$. We use the same notation for the scalar product in $X^+\oplus X^{0}$ and $\langle u,u\rangle =||u||^{2}=||u^+||^{2}+||u^{0}||^{2}$ for any $u=u^++u^{0}\in X^+\oplus X^{0}$, hence $X^+$ and $X^{0}$ are orthogonal. We consider another functional $J_{cp}\in \mathcal {C}^{1}(X,\mathbb {R})$ of the form

\[ J_{cp}=\frac{1}{2}||u^+{+}u^{0}||^{2}-I_{cp}(u)\ \mathrm{for}\ u=u^+{+}u^{0}+u^{1}\in X^+{\oplus} X^{0}\oplus X^{1}. \]

We define the corresponding Nehari–Pankov manifold for $J_{cp}$

\[ \mathcal{N}_{cp}:=\{u\in X\setminus X^{1}:J'_{cp}(u)|_{\mathbb{R}u\oplus X^{1}}=0\}, \]

and assume that $J_{cp}$ satisfies all corresponding assumption (A1)–(A4), (B1)–(B3), where we replace $X^+\oplus X^{0}$, $X^{1}$ and $I_{cp}$ instead of $X^+, X$ and $I$ respectively. Moreover, we enlist new additional conditions:

  1. (C1) $J_{cp}(u_{n})-J_{u_{n}}\longrightarrow 0$ if $(u_{n})\subset \mathcal {N}_{cp}$ is bounded and $(u^+_{n}+u^{0}_{n})\rightharpoonup 0$. Moreover there is $M>0$ such that $J_{cp}(u)-J(u)\leq M||u^++u^{0}||^{2}$ for $u\in \mathcal {N}_{cp}$.

  2. (C2) $I(t_{n}u_{n})\setminus t^{2}_{n}\longrightarrow \infty$ and $t_{n}\longrightarrow \infty$ and $(I(tu^+_{n}))_{n}$ is bounded away from 0 for any $t>1$.

  3. (C3) $J'$ is weak-to-weak$^{\ast }$ continuous on $\mathcal {N}$, i.e. if $(u_{n})_{n}\subset \mathcal {N}$, $u_{n}\rightharpoonup u$, then $J'(u_{n})\mathop {\rightharpoonup }\limits ^{\ast } J'(u)$ in $X^{\ast }$. Moreover $J$ is weakly sequentially lower semi-continuous on $\mathcal {N}$, i.e. if $(u_{n})_{n}\subset \mathcal {N}$, $u_{n}\rightharpoonup u$ and $u\in \mathcal {N}$, then $\mathop {\mathrm {lim}~\mathrm {inf}}\limits _{n\longrightarrow \infty }J(u_{n})\geq J(u)$.

There we present the abstract critical point theorem:

Lemma 2.21 [Reference Mederski30, theorem 3.2]: Let $J\in \mathcal {C}^{1}(X,\mathbb {R})$ be coercive on $\mathcal {N}$ and let $J_{cp}\in \mathcal {C}^{1}(X,\mathbb {R})$ be coercive on $\mathcal {N}_{cp}$. Suppose that $J$ and $J_{cp}$ satisfy (A1)-(A4),(B1)-(B3) and set $c=\mathop {\mathrm {inf}}\limits _{\mathcal {N}}J$ and $d=\mathop {\mathrm {inf}}\limits _{\mathcal {N}_{cp}}J_{cp}$. Then the following statements hold$:$

  1. (a) If (C1)–(C2) hold and $\beta < d$, then any $(PS)_{\beta }$-sequence in $\mathcal {N}$ contains a weakly convergent subsequence with a nontrivial limit point.

  2. (b) If (C1)–(C3) hold and $c< d$, then $c$ is achieved by a critical point (ground state) of $J$.

  3. (c) Suppose that $J$ is even and satisfies the $(PS)_{\beta }^{\mathcal {T}}$-condition in $\mathcal {N}$ for any $\beta <\beta _{0}$ for some fixed $\beta _{0}\in (c,\infty ]$. Let

    \[ m(\mathcal{N},\beta_{0})=sup\{\gamma(J^{{-}1}((0,\beta)\cap\mathcal{N}):\beta<\beta_{0}\}\in \mathbb{N}_{0}, \]
    where $\gamma$ stands for the Krasnoselskii genus for closed and symmetric subsets of $X$. Then $J$ has at least $m(\mathcal {N},\beta _{0})$ pairs of critical points $u$ and $-u$ such that $u\neq 0$ and $c\leq J(u)<\beta _{0}$.

3. Sharp constant $S_{\mathrm {curl},HL}(\mathbb {R}^{3})$

3.1. Proof of theorem 1.2

In this subsection, we consider functional (1.17), which is associated to equation (1.16), and we work on the following Nehari–Pankov manifold

\[ \mathcal{N}:=\left\{u\in W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\mathbb{R}^{3})\setminus \mathcal{W}_{\mathbb{R}^{3}}: J'(u)u=0\ \mathrm{and}\ J'(u)|_{\mathcal{W}_{\mathbb{R}^{3}}}=0\right\}. \]

Lemma 3.1 There exists a continuous mapping $m:\mathcal {V}_{\mathbb {R}^{3}}\setminus \{0\}\longrightarrow \mathcal {N}$.

Proof. By lemma 2.15, $W_{0}^{\alpha,2^{\ast }_{\alpha }}(\mathrm {curl};\mathbb {R}^{3})=\mathcal {V}_{\mathbb {R}^{3}}\oplus \mathcal {W}_{\mathbb {R}^{3}}$. It follows from (2.37) and (2.38) that if $v\in \mathcal {V}_{\mathbb {R}^{3}}$ and $\widetilde {w}_{\mathbb {R}^{3}}(v)\in \widetilde {\mathcal {W}}=\mathcal {W}_{\mathbb {R}^{3}}$, then we have ${J'(v+\widetilde {w}_{\mathbb {R}^{3}}(v))|_{\mathcal {W}_{\mathbb {R}^{3}}}\,{=}\,0}$. And as

(3.1)\begin{equation} J(t(v+\widetilde{w}(v)))=\frac{t^{2}}{2}\int_{\mathbb{R}^{3}}|\nabla v|^{2}\,{\rm d}x-\frac{t^{2\cdot2^{{\ast}}_{\alpha} }}{2\cdot 2^{{\ast}}_{\alpha}}\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|v+\widetilde{w}_{\mathbb{R}^{3}}(v)|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x, \end{equation}

there is a unique $t(v)>0$ such that

(3.2)\begin{equation} t(v)(v+\widetilde{w}_{\mathbb{R}^{3}}(v))\in\mathcal{N}\ \mathrm{for}\ v\in\mathcal{V}_{\mathbb{R}^{3}}\setminus\{0\}. \end{equation}

Setting $m(v):=t(v)(v+\widetilde {w}_{\mathbb {R}^{3}}(v))$, we then note that

(3.3)\begin{equation} J(m(v))\geq J(t(v+\widetilde{w}_{\mathbb{R}^{3}}))\ \mathrm{for}\ \mathrm{all}\ t>0\ \mathrm{and}\ \widetilde{w}_{\mathbb{R}^{3}}\in\mathcal{W}_{\mathbb{R}^{3}}. \end{equation}

Since $J(m(v))\geq J(v)$ and there exist $a,r>0$ such that $J(v)\geq a$ if $||v||=r$, this implies that $\mathcal {N}$ is bounded away from $\mathcal {W}_{\mathbb {R}^{3}}$ and hence closed. Therefore, by the similar analysis in [Reference Mederski and Szulkin33, lemma 4.4 ], the mapping $m$ is continuous.

Lemma 3.2 Set $\mathcal {S}:=\{v\in \mathcal {V}_{\mathbb {R}^{3}}:||v||=1\}$, there exist a $(PS)_{c}$ sequence $(v_{n})$ for $J\circ m$, and a $(PS)_{c}$ sequence $(m(v_{n}))$ for $J$ on $\mathcal {N}$.

Proof. By the continuity of mapping $m$, we easily observe that $m|_{\mathcal {S}}:\mathcal {S}\longrightarrow \mathcal {N}$ is a homeomorphism with the inverse $u=v+m(v)\mapsto \frac {v}{||v||}$. Recall the argument in [Reference Mederski28, proposition 4.4(b)], we know that $J\circ m|_{\mathcal {S}}:\mathcal {S}\longrightarrow \mathcal {R}$ is of class $\mathcal {C}^{1}$ and is bounded from below by the constant $a>0$. By the Ekeland variational principle, there is a $(PS)_{c}$ sequence $(v_{n})\subset \mathcal {S}$ such that

(3.4)\begin{equation} (J\circ m)(v_{n})\longrightarrow \mathop{\mathrm{inf}}\limits_{\mathcal{S}}J\circ m=\mathop{\mathrm{inf}}\limits_{\mathcal{N}}J\geq a>0. \end{equation}

Again, by the argument in [Reference Mederski28, proposition 4.4(b)], we have $(m(v_{n}))$ is a $(PS)_{c}$ sequence for $J$ on $\mathcal {N}$.

Proof. Complete of the proof of theorem 1.2

Firstly, we prove part (a). Taking a minimizing sequence $(u_{n})=(m(v_{n}))\subset \mathcal {N}$ and set $u_{n}=t(v_{n})(v_{n}+\widetilde {w}_{\mathbb {R}^{3}}(v_{n}))=v'_{n}+\widetilde {w}_{\mathbb {R}^{3}}(v'_{n})\in \mathcal {V}_{\mathbb {R}^{3}}\oplus \mathcal {W}_{\mathbb {R}^{3}}$. Then we have

\[ J(u_{n})=J(u_{n})-\frac{1}{2\cdot 2^{{\ast}}_{\alpha}}J'(u_{n})u_{n}=\frac{2^{{\ast}}_{\alpha}-1}{2\cdot2^{{\ast}}_{\alpha}}|\nabla\times u_{n}|^{2}_{2}=\frac{2^{{\ast}}_{\alpha}-1}{2\cdot2^{{\ast}}_{\alpha}}|\nabla v'_{n}|^{2}_{2}. \]

Since the norm $|\nabla \cdot |_{2}$ is an equivalent norm in $\mathcal {V}_{\mathbb {R}^{3}}$, it follows that $J(u_{n})$ is coercive on $\mathcal {N}$, hence $(v'_{n})$ is bounded. On the other hand, we also have

\[ J(u_{n})=J(u_{n})-\frac{1}{2}J'(u_{n})u_{n}=\frac{2^{{\ast}}_{\alpha}-1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u_{n}|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x. \]

By (3.4), $J(u_{n})$ is bounded away from $0$, so is $|u_{n}|_{Q^{\alpha,2^{\ast }_{\alpha }}}\nrightarrow 0$, and hence by (2.40), we also have $|v'_{n}|_{Q^{\alpha,2^{\ast }_{\alpha }}}\nrightarrow 0$.

Denote $T_{s,y}(v'):=s^{1/2}v'(s\cdot +y)$, where $s>0$, $y\in \mathbb {R}^{3}$. Then, passing to a subsequence and using the argument in [Reference Solimini42, theorem 1], we have $\overline {v}_{n}=T_{s_{n},y_{n}}(v'_{n})\rightharpoonup v_{0}$ for some $v_{0}\neq 0$, where $(s_{n})\subset \mathbb {R}^+$ and $(y_{n})\subset \mathbb {R}^{3}$. Taking subsequence again, we also have $\overline {v}_{n}\longrightarrow v_{0}$ a.e. in $\mathbb {R}^{3}$ and in view of the concentration–compactness lemma 2.18, we deduce $\widetilde {w}_{\mathbb {R}^{3}}(\overline {v}_{n})\rightharpoonup \widetilde {w}_{\mathbb {R}^{3}}(v_{0})$ and $\widetilde {w}_{\mathbb {R}^{3}}(\overline {v}_{n})\longrightarrow \widetilde {w}_{\mathbb {R}^{3}}(v_{0})$ a.e. in $\mathbb {R}^{3}$. Setting $u:=v_{0}+\widetilde {w}_{\mathbb {R}^{3}}(v_{0})$ and assume without loss of generality that $s_{n}=1$ and $y_{n}=0$, then by lemma 4.6 in [Reference Mederski and Szulkin33], we have $u_{n}\rightharpoonup u$ and $u_{n}\longrightarrow u$ a.e. in $\mathbb {R}^{3}$. Moreover, by lemma 2.7 we have

\[ (I_{\alpha}\ast|u_{n}|^{2^{{\ast}}_{\alpha}})|u_{n}|^{2^{{\ast}}_{\alpha}-2}u_{n}\rightharpoonup(I_{\alpha}\ast|u|^{2^{{\ast}}_{\alpha}})|u|^{2^{{\ast}}_{\alpha}-2}u\ \mathrm{in}\ (Q^{\alpha,2^{{\ast}}_{\alpha}}(\mathbb{R}^{3},\mathbb{R}^{3}))', \]

Therefore, for any $z\in W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\mathbb {R}^{3})$, using weak and a.e. convergence, we have

\begin{align*} \langle J'(u_{n}),z\rangle& =\int_{\mathbb{R}^{3}}\langle\nabla\times u_{n},z\rangle\,{\rm d}x-\int_{\mathbb{R}^{3}}\left\langle\left(I_{\alpha}\ast|u_{n}|^{2^{{\ast}}_{\alpha}}\right)|u_{n}(x)|^{2^{{\ast}}_{\alpha}-2}u_{n}(x),z\right\rangle\,{\rm d}x \nonumber\\ & \quad\longrightarrow \langle J'(u),z\rangle. \end{align*}

This implies that $u$ is a solution to (1.16). Using Fatou's lemma, we deduce that

\begin{align*} \mathop{\mathrm{inf}}\limits_{\mathcal{N}}J& =J(u_{n})+o(1)=J(u_{n})-\frac{1}{2}J'(u_{n})u_{n}+o(1)\\ & =\frac{2^{{\ast}}_{\alpha}-1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u_{n}|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x+o(1)\geq \frac{2^{{\ast}}_{\alpha}-1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x +o(1)\\ & =J(u)-\frac{1}{2}J'(u)u+o(1)=J(u)+o(1). \end{align*}

Hence $J(u)\leq \mathop {\mathrm {inf}}\limits _{\mathcal {N}} J\leq J(u)$ and as a solution, $u\in \mathcal {N}$.

Next, we show $\mathop {\mathrm {inf}}\limits _{\mathcal {N}} J=\frac {2^{\ast }_{\alpha }-1}{2\cdot 2^{\ast }_{\alpha }} S_{\mathrm {curl},HL}^{\frac {2^{\ast }_{\alpha }}{2^{\ast }_{\alpha }-1}}$, where $S_{\mathrm {curl},HL}$ is the sharp constant in (1.15), which can be rewritten as follow

(3.5)\begin{equation} S_{\mathrm{curl},HL}=\mathop{\mathrm{inf}}\limits_{\mathop{u\in W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\mathbb{R}^{3})}\limits_{\nabla\times u\neq0}}\frac{\int_{\mathbb{R}^{3}}|\nabla\times u|^{2}\,{\rm d}x}{\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u+\widetilde{w}_{\mathbb{R}^{3}}(u)|^{2^{{\ast}}_{\alpha}}|^{2}dx\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}}. \end{equation}

In fact, by (2.37), it is clear that a minimize $\widetilde {w}_{\mathbb {R}^{3}}(u)$ exists uniquely for any $u\in W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )$, not only $u\in \mathcal {V}_{\mathbb {R}^{3}}$. So by lemma 2.15, $u+\widetilde {w}_{\mathbb {R}^{3}}(u)=v+\widetilde {w}_{\mathbb {R}^{3}}(v)\in \mathcal {V}_{\mathbb {R}^{3}}\oplus \mathcal {W}_{\mathbb {R}^{3}}$ for some $v\in \mathcal {V}_{\mathbb {R}^{3}}$ and therefore

(3.6)\begin{align} \mathop{\mathrm{inf}}\limits_{w\in\mathcal{W}_{\mathbb{R}^{3}}}\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u+w|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x & =\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u+\widetilde{w}_{\mathbb{R}^{3}}(u)|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\nonumber\\ & \quad =\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|v+\widetilde{w}_{\mathbb{R}^{3}}(v)|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x. \end{align}

On the other hand, since $u+\widetilde {w}_{\mathbb {R}^{3}}(u)\in \mathcal {N}$, $J'(u)u=0$, i.e.

\[ \int_{\mathbb{R}^{3}}|\nabla\times u|^{2}\,{\rm d}x=\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u+\widetilde{w}_{\mathbb{R}^{3}}(u)|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x. \]

Then we can easily calculate that

\[ \mathop{\mathrm{inf}}\limits_{\mathcal{N}} J=\frac{2^{{\ast}}_{\alpha}-1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\mathbb{R}^{3}}|\nabla\times u|^{2}dx =\frac{2^{{\ast}}_{\alpha}-1}{2\cdot2^{{\ast}}_{\alpha}} S_{\mathrm{curl},HL}^{\frac{2^{{\ast}}_{\alpha}}{2^{{\ast}}_{\alpha}-1}}. \]

As we can see, if $u$ satisfies equality (1.15), then $t(u)(u+\widetilde {w}_{\mathbb {R}^{3}}(u))\in \mathcal {N}$ and is a minimizer for $J|_{\mathcal {N}}$ and the corresponding point $v$ in $\mathcal {S}$ is a minimizer for $J\circ m|_{\mathcal {S}}$, see (3.4). Hence $v$ is a critical point of $J\circ m|_{\mathcal {S}}$ and $m(v)=u$ is a critical point of $J$. This completes the proof of (a).

(b) To compare the constants $S_{\mathrm {curl},HL}$ and $S_{HL}$, see (3.5) and (1.18), we firstly claim that $S_{\mathrm {curl},HL}\geq S_{HL}$. In fact, by (3.6) and $\mathrm {div}(v)=0$, we have

(3.7)\begin{equation} S_{\mathrm{curl},HL}=\mathop{\mathrm{inf}}\limits_{v\in\mathcal{V}_{\mathbb{R}^{3}}\setminus\{0\}}\frac{\int_{\mathbb{R}^{3}}|\nabla v|^{2}\,{\rm d}x}{\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|v+\widetilde{w}_{\mathbb{R}^{3}}(v)|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}}. \end{equation}

Then given $\varepsilon >0$, we can find $v\neq 0$ such that

\[ \int_{\mathbb{R}^{3}}|\nabla v|^{2}\,{\rm d}x\leq (S_{\mathrm{curl},HL}+\varepsilon)\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|v+\widetilde{w}_{\mathbb{R}^{3}}(v)|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}. \]

Since $\widetilde {w}_{\mathbb {R}^{3}}(v)$ is a minimizer, we deduce that

\[ \int_{\mathbb{R}^{3}}|\nabla v|^{2}\,{\rm d}x\leq (S_{\mathrm{curl},HL}+\varepsilon)\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|v|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}. \]

On the other hand, let $v=(v_{1},v_{2},v_{3})$, then $|v|=(v_{1}^{2}+v_{2}^{2}+v_{3}^{2})^{\frac {1}{2}}$. Since $\frac {2^{\ast }_{\alpha }}{2}>1$, then by the second inequality in [Reference Mercuri, Moroz and Van Schaftingen34, proposition 2.1], we have

\begin{align*} \left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|v|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}} & =\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast(v_{1}^{2}+v_{2}^{2}+v_{3}^{2})^{\frac{2^{{\ast}}_{\alpha}}{2}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}\\ & \quad \leq \mathop{\sum}\limits_{i}^{3}\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|v_{i}|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}. \end{align*}

Moreover, by the definition of $S_{HL}$, see (1.18), we have

\begin{align*} \int_{\mathbb{R}^{3}}|\nabla v|^{2}\,{\rm d}x& \leq\frac{(S_{\mathrm{curl},HL}+\varepsilon)}{S_{HL}}\cdot S_{HL}\cdot\mathop{\sum}\limits_{i}^{3}\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|v_{i}|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}\\ & \quad\leq\frac{(S_{\mathrm{curl},HL}+\varepsilon)}{S_{HL}}\int_{\mathbb{R}^{3}}|\nabla v|^{2}\,{\rm d}x. \end{align*}

Hence we get our claim by $S_{\mathrm {curl},HL}+\varepsilon \geq S_{HL}$.

Secondly, we exclude the case $S_{\mathrm {curl},HL}=S_{HL}$. Otherwise, all inequalities above become equalities with $\varepsilon =0$. Particularly, $S_{HL}(\int _{\mathbb {R}^{3}}|I_{\alpha /2}\ast |v_{i}|^{2^{\ast }_{\alpha }}|^{2}\,{\rm d}x)^{\frac {1}{2^{\ast }_{\alpha }}} =\int _{\mathbb {R}^{3}}|\nabla v_{i}|^{2}\,{\rm d}x$. This implies that all $v_{i}$ are instantons and $v_{i}=C(\frac {b}{b^{2}+|x-a|^{2}})^{\frac {N-2}{2}}$, up to multiplicative constants, see [Reference Gao and Yang19, lemma 1.2] for the optimal function of $S_{HL}$. A simple calculation shows that $\mathrm {div}(v)\neq 0$. However, this is impossible because $v\in \mathcal {V}\setminus \{0\}$. Hence, $S_{\mathrm {curl},HL}\neq S_{HL}$.

3.2 Proof of theorem 1.3

To compare the sharp constants $S_{\mathrm {curl},HL}(\mathbb {R}^{3})$ and $\bar {S}_{\mathrm {curl},HL}(\Omega )$, we have introduced another constant $S_{\mathrm {curl},HL}(\Omega )$. Recall from § 2.1 that we have the following Helmholtz decomposition in entire space $\mathbb {R}^{3}$ and in the bounded domain $\Omega$:

\[ W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\mathbb{R}^{3})=\mathcal{V}_{\mathbb{R}^{3}}\oplus\mathcal{W}_{\mathbb{R}^{3}}\quad \mathrm{and}\quad W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\Omega)=\mathcal{V}_{\Omega}\oplus\mathcal{W}_{\Omega}. \]

Then, as (3.5), we note that $S_{\mathrm {curl},HL}(\Omega )$ [see (1.19)] can be characterized as

(3.8)\begin{align} S_{\mathrm{curl},HL}(\Omega)& =\mathop{\mathrm{inf}}\limits_{\mathop{u\in W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\Omega)}\limits_{\nabla\times u\neq0}}\mathop{\mathrm{sup}}\limits_{w\in\mathcal{W}_{\mathbb{R}^{3}}}\frac{\int_{\mathbb{R}^{3}}|\nabla\times u|^{2}\,{\rm d}x}{\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u+w|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}}\nonumber\\ & =\mathop{\mathrm{inf}}\limits_{\mathop{u\in W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\Omega)}\limits_{\nabla\times u\neq0}}\frac{\int_{\mathbb{R}^{3}}|\nabla\times u|^{2}\,{\rm d}x}{\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u+\widetilde{w}_{\mathbb{R}^{3}}(u)|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}}, \end{align}

where $u\in W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )$ is extended by $0$ outside $\Omega$. For constant $\bar {S}_{\mathrm {curl},HL}(\Omega )$ in domains $\Omega \neq \mathbb {R}^{3}$, it also can be characterized as

(3.9)\begin{align} \bar{S}_{\mathrm{curl},HL}(\Omega)& =\mathop{\mathrm{inf}}\limits_{\mathop{u\in W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\Omega)}\limits_{\nabla\times u\neq0}}\mathop{\mathrm{sup}}\limits_{w\in\mathcal{W}_{\Omega}}\frac{\int_{\Omega}|\nabla\times u|^{2}\,{\rm d}x}{\left(\int_{\Omega}|I_{\alpha/2}\ast|u+w|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}}\nonumber\\ & =\mathop{\mathrm{inf}}\limits_{\mathop{u\in W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\Omega)}\limits_{\nabla\times u\neq0}}\frac{\int_{\Omega}|\nabla\times u|^{2}\,{\rm d}x}{\left(\int_{\Omega}|I_{\alpha/2}\ast|u+\widetilde{w}_{\Omega}(u)|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}}. \end{align}

To compare these sharp constants, we introduce the following set

(3.10)\begin{equation} \mathcal{N}_{\Omega}:=\{u\in W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\Omega)\setminus\mathcal{W}_{\Omega} ;J'(u)u=0\ \mathrm{and}\ J'(u)|_{\mathcal{W}_{\Omega}}=0\}. \end{equation}

According to the argument in [Reference Mederski and Szulkin33, lemma 4.2], we have $tu+\widetilde {w}_{\mathbb {R}^{3}}(tu)=t(u+\widetilde {w}_{\mathbb {R}^{3}}(u))$ , then we may assume without loss of generality that $u+\widetilde {w}_{\mathbb {R}^{3}}(u)\in \mathcal {N}$ in (3.8). By the maximality and uniqueness of $\widetilde {w}_{\Omega }(u)$, we easily deduce that the mapping $u\mapsto \widetilde {w}_{\Omega }(u)$ is also continuous. Therefore, we may assume that $u+\widetilde {w}_{\Omega }(u)\in \mathcal {N}_{\Omega }$ in (3.9). Then easily calculate that

(3.11)\begin{align} \mathop{\mathrm{inf}}\limits_{\mathcal{N}} J& =\frac{2^{{\ast}}_{\alpha}-1}{2\cdot2^{{\ast}}_{\alpha}} S_{\mathrm{curl},HL}^{\frac{2^{{\ast}}_{\alpha}}{2^{{\ast}}_{\alpha}-1}},\quad \mathop{\mathrm{inf}}\limits_{\mathcal{N}} J|_{W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\Omega)}\nonumber\\ & \quad =\frac{2^{{\ast}}_{\alpha}-1}{2\cdot2^{{\ast}}_{\alpha}} S^{\frac{2^{{\ast}}_{\alpha}}{2^{{\ast}}_{\alpha}-1}}_{\mathrm{curl},HL}(\Omega),\quad \mathop{\mathrm{inf}}\limits_{\mathcal{N}_{\Omega}} J=\frac{2^{{\ast}}_{\alpha}-1}{2\cdot2^{{\ast}}_{\alpha}} \bar{S}^{\frac{2^{{\ast}}_{\alpha}}{2^{{\ast}}_{\alpha}-1}}_{\mathrm{curl},HL}(\Omega). \end{align}

Lemma 3.3 $S_{\mathrm {curl},HL}(\Omega )\geq S_{\mathrm {curl},HL}$, $S_{\mathrm {curl},HL}(\Omega )\geq \bar {S}_{\mathrm {curl},HL}(\Omega )$.

Proof. In view of lemma 2.16, $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )\subset W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\mathbb {R}^{3})$, we can easily observe from (3.8) and (3.5) that $S_{\mathrm {curl},HL}(\Omega )\geq S_{\mathrm {curl},HL}$. Similarly, since $\mathcal {W}_{\Omega }\subset \mathcal {W}_{\mathbb {R}^{3}}$, we can deduce that $S_{\mathrm {curl},HL}(\Omega )\geq \bar {S}_{\mathrm {curl},HL}(\Omega )$ from (3.8) and (3.9).

To complete theorem 1.3, we shall need the following inequality, which corresponds to the condition (B3), and the proof follows a similar argument in [Reference Mederski30, lemma 4.1].

Lemma 3.4 If $u\in W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )\setminus \{0\},u\in \mathcal {W}_{\Omega }$ and $t\geq 0$, then

\[ J(u)\geq J(tu+w)-J'(u)\left[\frac{t^{2}-1}{2}u+tw\right]. \]

Moreover, strict inequality holds provided $t=1$ and $w=0$. $(\Omega =\mathbb {R}^{3}\ admitted.)$

Proof. By an explicit computation and using $\nabla \times w=0$, we show that

\[ J(u)-J(tu+w)+J'(u)\left[\frac{t^{2}-1}{2}u+tw\right]=\int_{\Omega}\varphi(t,x)\,{\rm d}x, \]

where

\begin{align*} \varphi(t,x)& ={-}\left\langle\left(I_{\alpha}\ast|u|^{2^{{\ast}}_{\alpha}}\right)|u(x)|^{2^{{\ast}}_{\alpha}-2}u(x) ,\frac{t^{2}-1}{2}u(x)+tw(x)\right\rangle\\ & \quad-\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\Omega}|I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x \\ & \quad+\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\Omega}|I_{\alpha/2}\ast|tu+w|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x. \end{align*}

It is easy to check that $\varphi (0,x)>0$ as $t=0$ and $\varphi (t,x)\longrightarrow \infty$ as $t\longrightarrow \infty$. Therefore, if there exist $t$ such that $\varphi (t,x)\leq 0$, then there exists $t_{0}>0$ such that $\partial _{t}\varphi (t_{0},x)=0$, namely

\begin{align*} \partial_{t}\varphi(t_{0},x)& ={-}\left\langle\left(I_{\alpha}\ast|u|^{2^{{\ast}}_{\alpha}}\right)|u(x)|^{2^{{\ast}}_{\alpha}-2}u(x) ,t_{0}u(x)+w(x)\right\rangle\\ & \quad+\left\langle\left(I_{\alpha}\ast|t_{0}u+w|^{2^{{\ast}}_{\alpha}}\right)|t_{0}u(x)+w(x)|^{2^{{\ast}}_{\alpha}-2}\left(t_{0}u(x)+w(x)\right),u(x)\right\rangle=0, \end{align*}

then either $\langle u,t_{0}u+w\rangle$=0, i.e. $-\langle u,w\rangle =t_{0}\langle u,u\rangle =t_{0}|u|^{2}$, or $|u|=|t_{0}+w|$, i.e., $-t_{0}\langle u,w\rangle =\frac {t_{0}^{2}-1}{2}|u|^{2}+\frac {1}{2}|w|^{2}$. In the first case, we obtain that

\begin{align*} \varphi(t_{0},x)& =\left(\frac{t_{0}^{2}+1}{2}-\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\right)\int_{\Omega}|I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\\ & \quad +\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\mathbb{R}{3}}|I_{\alpha/2}\ast|t_{0}u+w|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x>0. \end{align*}

And in the second case, we deduce that

\[ \varphi(t_{0},x)=\frac{1}{2}\int_{\Omega}(I_{\alpha}\ast|u|^{2^{{\ast}}_{\alpha}})|u(x)|^{2^{{\ast}}_{\alpha}-2}|w(x)|^{2}\,{\rm d}x\geq0. \]

Hence $\varphi (t,x)\geq 0$ for all $t\geq 0$ and the inequality is strict if $w\neq 0$. If $w=0$, we can see

\[ \varphi(t,x)=\left(\frac{t^{2\cdot2^{{\ast}}_{\alpha}}}{2\cdot2^{{\ast}}_{\alpha}}-\frac{t^{2}}{2}+\frac{1}{2}-\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\right)\int_{\Omega}(I_{\alpha}\ast|u|^{2^{{\ast}}_{\alpha}})|u(x)|^{2^{{\ast}}_{\alpha}-2}|w(x)|^{2}dx>0 \]

provided $t\neq 1$.

Lemma 3.5 $S_{\mathrm {curl},HL}(\Omega )\leq S_{\mathrm {curl},HL}$.

Proof. By theorem 1.2(a), $u$ is a minimizer for $J$ on $\mathcal {N}$, then we can find a sequence $(u_{n})\subset \mathcal {C}_{0}^{\infty }(\mathbb {R}^{3},\mathbb {R}^{3})$ such that $u_{n}\longrightarrow u$. By the Helmholtz decomposition, we have $u_{n}=v_{n}+w_{m}, v_{n}\in \mathcal {V}_{\mathbb {R}^{3}},w_{n}\in \mathcal {W}_{\mathbb {R}^{3}}$. Since $u_{n}=v_{n}+w_{n}\longrightarrow u=v_{0}+\widetilde {w}_{\mathbb {R}^{3}}(v_{0})$ and therefore $v_{n}\longrightarrow v_{0}$, $w_{n}\longrightarrow \widetilde {w}_{\mathbb {R}^{3}}(v_{0})$. So $v_{0}\neq 0$ and $v_{n}$ are bounded away from $0$ in $Q^{\alpha,2^{\ast }_{\alpha }}(\mathbb {R}^{3},\mathbb {R}^{3})$ due to $u\in \mathcal {N}$.

Assume without loss of generality that $0\in \Omega$. There exist $\lambda _{n}$ such that $\overline {u}_{n}$ given by $\overline {u}_{n}(x):=\lambda _{n}^{1\setminus 2}u_{n}(\lambda _{n}x)$ are supported in $\Omega$, that is $\overline {u}_{n}(x)\in W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )$. Set $\widetilde {w}_{\mathbb {R}^{3}}(\overline {u}_{n})\in \mathcal {W}_{\mathbb {R}^{3}}$ and choose $t_{n}$ so that $t_{n}(\overline {u}_{n}+\widetilde {w}_{\mathbb {R}^{3}}(\overline {u}_{n}))\in \mathcal {N}$, then

(3.12)\begin{equation} t_{n}^{2}=\frac{\left(\int_{\mathbb{R}^{3}}|\nabla\times \overline{u}_{n}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}-1}}} {\left(\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|\overline{u}_{n}+\widetilde{w}_{\mathbb{R}^{3}}(\overline{u}_{n})|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}-1}}}. \end{equation}

Since the Riesz potential is invariant with respect to translation, we have $||\overline {u}_{n}||=||u_{n}||$ and

\begin{align*} \int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|\overline{u}_{n}+\widetilde{w}_{\mathbb{R}^{3}}(\overline{u}_{n})|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x& =\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|u_{n}+\widetilde{w}_{\mathbb{R}^{3}}(u_{n})|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\\ & =\int_{\mathbb{R}^{3}}|I_{\alpha/2}\ast|v_{n}+\widetilde{w}_{\mathbb{R}^{3}}(v_{n})|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x. \end{align*}

Therefore, as $(u_{n})$ is bounded, we have $(\overline {u}_{n})$ and $(\widetilde {w}_{\mathbb {R}^{3}}(\overline {u}_{n}))$ are bounded away from $0$, so is $||\overline {u}_{n}(x)+\widetilde {w}_{\mathbb {R}^{3}}(\overline {u}_{n})(x)||_{Q^{\alpha,2^{\ast }_{\alpha }}}$. Then we deduce that $(t_{n})$ is bounded, hence so is $(t_{n}^{2})$. Moreover, since $J(\overline {u}_{n})=J(u_{n})\longrightarrow \frac {2^{\ast }_{\alpha }-1}{2\cdot 2^{\ast }_{\alpha }} S_{\mathrm {curl},HL}^{\frac {2^{\ast }_{\alpha }}{2^{\ast }_{\alpha }-1}}$ and $||J'(\overline {u}_{n})||=||J'(u_{n})||\longrightarrow 0$, it follows from lemma 3.4 that

\begin{align*} \frac{2^{{\ast}}_{\alpha}-1}{2\cdot2^{{\ast}}_{\alpha}} S_{\mathrm{curl},HL}^{\frac{2^{{\ast}}_{\alpha}}{2^{{\ast}}_{\alpha}-1}} & =\mathop{\mathrm{lim}}\limits_{n\longrightarrow\infty}J(\overline{u}_{n}) \geq\mathop{\mathrm{lim}}\limits_{n\longrightarrow\infty} \left(\vphantom{\left. -J'(\overline{u}_{n})\left[\frac{t^{2}_{n}-1}{2}\overline{u}_{n} +t^{2}_{n}\widetilde{w}_{\mathbb{R}^{3}}(\overline{u}_{n}))\right]\right)}J(t_{n}(\overline{u}_{n}+\widetilde{w}_{\mathbb{R}^{3}}(\overline{u}_{n})))\right.\\ & \left.\quad -\,J'(\overline{u}_{n})\left[\frac{t^{2}_{n}-1}{2}\overline{u}_{n} +t^{2}_{n}\widetilde{w}_{\mathbb{R}^{3}}(\overline{u}_{n}))\right]\right)\\ & =\mathop{\mathrm{lim}}\limits_{n\longrightarrow\infty}J(t_{n}(\overline{u}_{n}+\widetilde{w}_{\mathbb{R}^{3}}(\overline{u}_{n})))\geq\frac{2^{{\ast}}_{\alpha}-1}{2\cdot2^{{\ast}}_{\alpha}} S^{\frac{2^{{\ast}}_{\alpha}}{2^{{\ast}}_{\alpha}-1}}_{\mathrm{curl},HL}(\Omega). \end{align*}

The last inequality follows from the fact that $\overline {u}_{n}\in W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )$.

Proof. Complete of the proof of theorem 1.3

Repeating the proof of theorem 1.2 (b) with obvious changes, namely, change the domain $\mathbb {R}^{3}$ into $\Omega$, change $S_{\mathrm {curl},HL}$ into $\bar {S}_{\mathrm {curl},HL}(\Omega )$, we have $\bar {S}_{\mathrm {curl},HL}(\Omega )\geq S_{HL}$. Since the optimal function for $\bar {S}_{\mathrm {curl},HL}(\Omega )$ is not found in our process, we can not exclude the case $\bar {S}_{\mathrm {curl},HL}(\Omega )=S_{HL}$. As a consequently, we complete the proof of theorem 1.3 by lemmas 3.3 and 3.5.

4. Proof of theorem 1.4

According to the spectrum analysis of the curl–curl operator in the introduction, for $\lambda \leq 0$, we find two closed and orthogonal subspaces $\mathcal {V}_{\Omega }^+$ and $\widetilde {\mathcal {V}}_{\Omega }$ of $\mathcal {V}_{\Omega }$ such that the quadratic form $Q:\mathcal {V}_{\Omega }\longrightarrow \mathbb {R}$ given by

(4.1)\begin{equation} Q(v):=\int_{\Omega}(|\nabla\times v|^{2}+\lambda|v|^{2})\,{\rm d}x=\int_{\Omega}(|\nabla v|^{2}+\lambda|v|^{2})\,{\rm d}x \end{equation}

is positive defined on $\mathcal {V}_{\Omega }^+$ and negative semi-definite on $\widetilde {\mathcal {V}}_{\Omega }$ where $\mathrm {dim}\widetilde {\mathcal {V}}_{\Omega }<\infty$. Writing $u=v+w=v^++\widetilde {v}+w\in \mathcal {V}_{\Omega }^+\oplus \widetilde {\mathcal {V}}_{\Omega }\oplus \mathcal {W}_{\Omega }$, the functional $J_{\lambda }$ [see (1.13)] can be expressed as

\begin{align*} J_{\lambda}(u) & =\frac{1}{2}||v^+||^{2}+\frac{1}{2}||\widetilde{v}||^{2}+\frac{\lambda}{2}\int_{\Omega}(|v|^{2}+|w|^{2})\,{\rm d}x-\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\Omega}|I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\\ & =\frac{1}{2}||v^+||^{2}-I_{\lambda}(v+w), \end{align*}

where

\[ I_{\lambda}(v+w)={-}\frac{1}{2}||\widetilde{v}||^{2}-\frac{\lambda}{2}\int_{\Omega}(|v|^{2}+|w|^{2})\,{\rm d}x+\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\Omega}|I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x. \]

Similarly as in [Reference Bartsch and Mederski5], we shall show that $J_{\lambda }$ satisfies the assumptions (A1)–(A4), (B1)–(B3) and (C1)–(C3) from § 2.2.

Lemma 4.1 Conditions (A1)–(A4), (B1)–(B3) and (C2) in lemma 2.21 hold for $J_{\lambda }$.

Proof.

  1. (i) By lemma 2.7, we have $I_{\lambda }$ is of class $\mathcal {C}^{1}$. Since $Q(v)$ is negative on $\widetilde {\mathcal {V}}_{\Omega }$, $2^{\ast }_{\alpha }$ is a upper critical index, we have $I_{\lambda }(u)\geq I_{\lambda }(0)=0$ for any $u\in W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )$.

  2. (ii) Since $I_{\lambda }$ is convex, $I_{\lambda }$ is $\mathcal {T}-$sequentialy lower semicontinuous. Hence, (A2) holds.

  3. (iii) We easily check (A3), since $u_{n}\rightharpoonup u_{0}$ in $Q^{\alpha,2^{\ast }_{\alpha }}(\Omega,\mathbb {R}^{3})$, and $I_{\lambda }(u_{n})\longrightarrow I_{\lambda }(u_{0})$ imply $|u_{n}|_{Q^{\alpha,2^{\ast }_{\alpha }}}\longrightarrow |u_{0}|_{Q^{\alpha,2^{\ast }_{\alpha }}}$, thus $u_{n}\longrightarrow u_{0}$ in $Q^{\alpha,2^{\ast }_{\alpha }}(\Omega,\mathbb {R}^{3})$.

  4. (iv) Since $\mathcal {V}_{\Omega }$ is a Hilbert space, the HLS inequality is still valid on there, then for any $u\in \mathcal {V}_{\Omega }^+$, we have

    \begin{align*} J(u)=J(v,0)& =\frac{1}{2}||v||_{\mathcal{V}_{\Omega}}^{2}+\frac{\lambda}{2}|v|^{2}_{2}-\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\Omega}|I_{\alpha/2}\ast|v|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\\ & \geq\frac{1}{2}||v||_{\mathcal{V}_{\Omega}}^{2}+\frac{\lambda}{2}|v|^{2}_{2}-\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\left(\int_{\Omega}|v|^{6}\,{\rm d}x\right)^{\frac{1}{6}\ast2}\\ & \geq\frac{\delta}{2}||v||_{\mathcal{V}_{\Omega}}^{2}-\varepsilon|v|^{2}-c_{\varepsilon}|v|^{3}_{6}\geq\frac{\delta}{4}||v||^{2}_{\mathcal{V}_{\Omega}}-C_{1}||v||_{\mathcal{V}_{\Omega}}^{3} \end{align*}
    for some $\delta, C_{1}>0$.
  5. (v) Condition (B1) follows from lemma 5.1 (c) in [Reference Bartsch and Mederski5]. Suppose that $(||v^+_{n}||_{\mathcal {V}_{\Omega }})_{n}$ is bounded and $||(v_{n},w_{n})||\longrightarrow \infty$ as $n\longrightarrow \infty$. Since $\mathrm {dim}(\widetilde {\mathcal {V}}_{\Omega })<\infty$ there holds $|v_{n}+w_{n}|_{Q^{\alpha,2^{\ast }_{\alpha }}}\longrightarrow \infty$. Moreover by the orthogonality $\mathcal {V}_{\Omega }^+\perp \widetilde {\mathcal {V}}_{\Omega }$ in $L^{2}(\Omega,\mathbb {R}^{3})$ and $\mathcal {V}_{\Omega }\perp \mathcal {W}_{\Omega }$ in $Q^{\alpha,2^{\ast }_{\alpha }}(\Omega,\mathbb {R}^{3})$, we have

    (4.2)\begin{equation} ||\widetilde{v}_{n}||_{\mathcal{V}_{\Omega}}^{2}\leq C_{1}|\widetilde{v}_{n}|_{2}^{2}\leq C_{1}|v_{n}|^{2}_{2}\leq C_{1}|v_{n}+ w_{n}|_{2}^{2}\leq C_{2}|v_{n}+ w_{n}|_{Q^{\alpha,2^{{\ast}}_{\alpha}}}^{2} \end{equation}
    for some $0< C_{1}< C_{2}$. This implies
    \begin{align*} I(v_{n},w_{n})& ={-}\frac{1}{2}||\widetilde{v}_{n}||-\frac{\lambda}{2}|v_{n}+ w_{n}|^{2}_{2}+\frac{1}{2\cdot2^{{\ast}}_{\alpha}}|v_{n}+ w_{n}|_{Q^{\alpha,2^{{\ast}}_{\alpha}}}^{2^{{\ast}}_{\alpha}}\\ & \geq{-}\frac{C_{2}}{2}|v_{n}+ w_{n}|_{Q^{\alpha,2^{{\ast}}_{\alpha}}}^{2}+\frac{1}{2\cdot2^{{\ast}}_{\alpha}}|v_{n}+ w_{n}|_{Q^{\alpha,2^{{\ast}}_{\alpha}}}^{2^{{\ast}}_{\alpha}}\longrightarrow\infty, \end{align*}
    because $|v_{n}+\nabla w_{n}|_{Q^{\alpha,2^{\ast }_{\alpha }}}\longrightarrow \infty$.
  6. (vi) This part we check condition (B2) and (C2). By (4.2), we have

    \begin{align*} & I(t_{n}(v_{n}+w_{n}))\\ & =\frac{1}{2}||t_{n}\widetilde{v}_{n}||_{\mathcal{V}_{\Omega}}^{2}-\frac{\lambda}{2}|t_{n}(v_{n}+w_{n})|^{2}_{2}+\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\Omega}|I_{\alpha/2}\ast|t_{n}(v_{n}+w_{n})|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\\ & \geq{-}\frac{1}{2}t^{2}_{n}||\widetilde{v}_{n}||^{2}_{\mathcal{V}_{\Omega}}-\frac{\lambda}{2}t^{2}_{n}|v_{n}+w_{n}|^{2}_{2}+t_{n}^{2\cdot2^{{\ast}}_{\alpha}}\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\Omega}|I_{\alpha/2}\ast|v_{n}+w_{n}|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\\ & \geq{-}\frac{C_{2}}{2}t^{2}_{n}||{v}_{n}+w_{n}||^{2}_{Q_{\alpha,2^{{\ast}}_{\alpha}}}+t_{n}^{2\cdot2^{{\ast}}_{\alpha}}\frac{1}{2\cdot2^{{\ast}}_{\alpha}}||{v}_{n}+w_{n}||^{2\cdot2^{{\ast}}_{\alpha}}_{Q_{\alpha,2^{{\ast}}_{\alpha}}}. \end{align*}
    Then
    \[ I(t_{n}(v_{n}+w_{n}))/ t^{2}_{n}\geq{-}\frac{C_{2}}{2}||\widetilde{v}_{n}||^{2}_{Q_{\alpha,2^{{\ast}}_{\alpha}}}+t_{n}^{2\cdot2^{{\ast}}_{\alpha}-2}\frac{1}{2\cdot2^{{\ast}}_{\alpha}}||\widetilde{v}_{n}||^{2\cdot2^{{\ast}}_{\alpha}}_{Q_{\alpha,2^{{\ast}}_{\alpha}}}. \]
    If $||(v_{n},w_{n})||\longrightarrow \infty$ then $I(t_{n}(v_{n}+w_{n}))/ t^{2}_{n}\longrightarrow \infty$. If $(||(v_{n},w_{n})||)_{n}$ is bounded. Then $(|v_{n}+w_{n}|_{Q^{\alpha,2^{\ast }_{\alpha }}})_{n}$ is bounded. If $|v_{n}+w_{n}|_{Q^{\alpha,2^{\ast }_{\alpha }}}\longrightarrow 0$, then $|v_{n}+w_{n}|_{2}\longrightarrow 0$ and by the orthogonality in $L^{2}(\Omega,\mathbb {R}^{3})$ which contradicts $u_{0}\neq 0$. Therefore $\frac {t_{n}^{2\cdot 2^{\ast }_{\alpha }-2}}{2\cdot 2^{\ast }_{\alpha }}||\widetilde {v}_{n}||^{2\cdot 2^{\ast }_{\alpha }}_{Q_{\alpha,2^{\ast }_{\alpha }}}\longrightarrow \infty$ as $n\longrightarrow \infty$ and again $I(t_{n}(v_{n}+w_{n}))/ t^{2}_{n}\longrightarrow \infty$.
  7. (vii) Condition (B3) follows from lemma 3.4 by changing $J(u)$ into $J_{\lambda }(u)$.

To apply the concentration–compactness lemma, we set $\widetilde {\mathcal {W}}:=\widetilde {\mathcal {V}}_{\Omega }\oplus \mathcal {W}_{\Omega }$ with $\widetilde {w}_{\Omega }=\widetilde {v}+w$, where $\widetilde {\mathcal {V}}_{\Omega }=Z$, see §2.2. On the other hand, we shall extend $\mathcal {V}_{\Omega }^+$ into $\mathcal {V}_{\mathbb {R}^{3}}$, which is a closed subspaces of $\mathcal {D}^{1,2}(\mathbb {R}^{3},\mathbb {R}^{3})$. Indeed, let $U$ be a bounded domain in $\mathbb {R}^{3}$, $\bar {\Omega }\subset U$. Since $\mathcal {V}_{\Omega }\subset H^{1}(\Omega,\mathbb {R}^{3})$, then each $v\in \mathcal {V}_{\Omega }$ may be extended to $v'\in H^{1}_{0}(U,\mathbb {R}^{3})$ such that $v'|_{\Omega } =v$. This extension is bounded as a mapping from $\mathcal {V}_{\Omega }$ to $H^{1}_{0}(U,\mathbb {R}^{3})$. Since

\[ \mathcal{V}':=\{v'\in H^{1}_{0}(U,\mathbb{R}^{3}):v'|_{\Omega}\in\mathcal{V}_{\Omega}\} \]

is a closed subspace of $H^{1}_{0}(U,\mathbb {R}^{3})$, and hence of $\mathcal {D}^{1,2}(\mathbb {R}^{3},\mathbb {R}^{3})$, we then can apply lemma 2.18 with $\mathcal {V}_{\Omega }^+$ replacing $\mathcal {V}_{\mathbb {R}^{3}}$. Set the generalized Nehari–Pankov manifold as follow

(4.3)\begin{equation} \mathcal{N}_{\lambda}:=\{u\in W^{\alpha,2^{{\ast}}_{\alpha}}_{0}(\mathrm{curl};\Omega)\setminus(\widetilde{\mathcal{V}}_{\Omega}\oplus\mathcal{W}_{\Omega}) :J'_{\lambda}(u)|_{\mathcal{R}u\oplus\widetilde{\mathcal{V}}_{\Omega}\oplus\widetilde{\mathcal{W}}_{\Omega}}=0\}. \end{equation}

Lemma 4.2 $J'$ is weak-to-weak$^{\ast }$ continuous on $\mathcal {N_{\lambda }}$ and condition (C3) in lemma 2.21 holds.

Proof. Suppose that $u_{n}\rightharpoonup u_{0}$ in $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )$. Set $u_{n}=m_{\lambda }(v^+_{n})=v^++\widetilde {w}_{\Omega }(v^+_{n})$. Since $\mathcal {V}_{\Omega }^+$ and $\widetilde {\mathcal {W}}_{\Omega }$ are complementary subspaces, $v^+_{n}$ is bounded in $\mathcal {V}_{\Omega }^+$. Then passing to a subsequence, we have $v^+_{n}\rightharpoonup v^+_{0}$ in $\mathcal {V}_{\Omega }^+$ , $v^+_{n}\longrightarrow v^+_{0}$ in $L^{2}(\Omega,\mathbb {R}^{3})$ and a.e. in $\Omega$. Therefore, by the concentration–compactness lemma 2.18, we have $\widetilde {w}_{\Omega }(v^+_{n})\longrightarrow \widetilde {w}_{\Omega }(v^+_{0})$ in $L^{2}(\Omega,\mathbb {R}^{3})$ and also a.e. in $\Omega$. Hence, we also have $u_{n}\longrightarrow u_{0}$ a.e. in $\Omega$. Then by the Viltali convergence principle, $J'_{\lambda }$ is weak-to-weak$^{\ast }$ continuous. Moreover, by the lower semi-continuity of $I_{\lambda }$, (C3) holds.

Now, we set a compactly perturbed problem. Take $X^{0}:=\widetilde {V}_{\Omega }$, $X^{1}:=\mathcal {W}_{\Omega }$ and let us consider the functional $J_{cp}: X =\mathcal {V}_{\Omega }\oplus \mathcal {W}_{\Omega }\longrightarrow \mathbb {R}$ given by

\[ J_{cp}(u)=J_{0}(u)=\frac{1}{2}\int_{\Omega}|\nabla\times u|^{2}\,{\rm d}x+\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\Omega}|I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x. \]

Moreover we define the corresponding Nehari–Pankov manifold

(4.4)\begin{equation} \mathcal{N}_{cp} = \{E\in (\mathcal{V}_{\Omega}\oplus\mathcal{W}_{\Omega})\setminus\mathcal{W}_{\Omega}: J'_{cp}(u)|_{\mathbb{R}u\oplus\mathcal{W}_{\Omega}}=0\}. \end{equation}

Observe that as in lemma 4.1 we show that $J_{cp}$ satisfies the corresponding condition (A1)–(A4) and (B1)–(B3).

Lemma 4.3 Condition (C1) in lemma 2.21 holds.

Proof. For any bounded sequence $u_{n}\subset \mathcal {N}_{cp}$, we have $u_{n}\rightharpoonup u$ in $\mathcal {N}_{cp}$. By the concentrated– compactness lemma 2.18, we have $u_{n}\longrightarrow u$ in $L^{2}(\Omega,\mathbb {R}^{3})$. Since $J_{\lambda }(u)-J_{cp}(u)=\frac {\lambda }{2}\int _{\Omega }|u|^{2}dx$, we have condition (C1) holds.

Lemma 4.4 $J_{\lambda }$ is coercive on $\mathcal {N}_{\lambda }$ and $J_{cp}$ is coercive on $\mathcal {N}_{cp}$.

Proof. The proof is similar to lemma 4.6 in [Reference Mederski30]. Let $u_{n}=v_{n}+w_{n}\in \mathcal {N}_{\lambda }$ and suppose that $||u_{n}||\longrightarrow \infty$. Observe that

\[ J_{\lambda}(u_{n})=J_{\lambda}(u_{n})-\frac{1}{2}J'_{\lambda}(u_{n})(u_{n})=\left(\frac{1}{2}-\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\right)|u_{n}|_{Q^{\alpha,2^{{\ast}}_{\alpha}}}^{2\cdot2^{{\ast}}_{\alpha}}\geq C_{1}|w_{n}|_{Q^{\alpha,2^{{\ast}}_{\alpha}}}^{2\cdot2^{{\ast}}_{\alpha}} \]

for some constant $C_{1}>0$, since $\mathcal {W}_{\Omega }$ is closed, $\mathrm {cl}\mathcal {V}_{\Omega }\cap \mathcal {W}_{\Omega }=\{0\}$ in $Q^{\alpha,2^{\ast }_{\alpha }}(\Omega,\mathbb {R}^{3})$ and the projection $\mathrm {cl}\mathcal {V}_{\Omega }\oplus \mathcal {W}_{\Omega }=\{0\}$ onto $\mathcal {W}_{\Omega }$ is continuous. Hence, if $|u_{n}|_{Q^{\alpha,2^{\ast }_{\alpha }}}\longrightarrow \infty$, then $J_{\lambda }(u_{n})\longrightarrow \infty$ as $n\longrightarrow \infty$. Suppose that $|u_{n}|_{Q^{\alpha,2^{\ast }_{\alpha }}}$ is bounded, Then $||v_{n}||\longrightarrow \infty$ and

\begin{align*} J_{\lambda}(u_{n})& =J_{\lambda}(u_{n})-\frac{1}{2\cdot2^{{\ast}}_{\alpha}}J'_{\lambda}(u_{n})(u_{n})\\ & =\left(\frac{1}{2}-\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\right)\left(\int_{\Omega}|\nabla\times v_{n}|^{2}\,{\rm d}x+\lambda\int_{\Omega}|v_{n}+w_{n}|^{2}\,{\rm d}x\right)\\ & \geq\left(\frac{1}{2}-\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\right)\left(\int_{\Omega}|\nabla\times v_{n}|^{2}\,{\rm d}x+\lambda C_{2}|u_{n}|_{Q^{\alpha,2^{{\ast}}_{\alpha}}}^{2}\right), \end{align*}

for some constant $C_{2}>0$. Thus $J_{\lambda }(u_{n})\longrightarrow \infty$. Similarly, we show that $J_{cp}$ is coercive on $\mathcal {N}_{cp}$.

Lemma 4.5 Let $c_{\lambda }< c_{0}$ and $(u_{n})_{n\in \mathbb {N}}\subset \mathcal {N}_{\lambda }$ be a Palais-Smale sequence at $c_{\lambda }$, i.e. $J_{\lambda }(u_{n})\longrightarrow c_{\lambda }$ and $J'_{\lambda }(u_{n})\longrightarrow 0$ as $n\longrightarrow \infty$. Then $u_{n}\rightharpoonup u_{0}\neq 0$ for some $u_{0}$ in $W^{\alpha,2^{\ast }_{\alpha }}(\mathrm {curl};\Omega )$. Moreover, $c_{\lambda }$ is achieved by a critical point of $J_{\lambda }$.

Proof. The conclusion follows from lemmas 4.1, 4.3, 4.2, 4.4 and 2.21(a)(b).

As we introduced before, we shall verify the $(PS)_{c}$ condition. Similar to [Reference Mederski and Szulkin33, lemma 6.4] we need the following version of the Brezis-Lieb lemma. Setting

\[ N(u)=\left(I_{\alpha}\ast|u|^{2^{{\ast}}_{\alpha}}\right)|u(x)|^{2^{{\ast}}_{\alpha}-2}u(x), \]

then we have the following lemma.

Lemma 4.6 Suppose $(u_{n})$ is bounded in $Q^{\alpha,2^{\ast }_{\alpha }}(\Omega,\mathbb {R}^{3})$ and $u_{n}\longrightarrow u$ a.e. in $\Omega$. Then

\[ N(u_{n})-N(u_{n}-u)\longrightarrow N(u)\ in\ (Q^{\alpha,2^{{\ast}}_{\alpha}}(\Omega,\mathbb{R}^{3}))'\ as\ n\longrightarrow\infty. \]

Proof. By the proof of lemma 2.7, we have $N(u):Q^{\alpha,2^{\ast }_{\alpha }}(\Omega,\mathbb {R}^{3})\longrightarrow (Q^{\alpha,2^{\ast }_{\alpha }}(\Omega,\mathbb {R}^{3}))'$. Therefore, it turns to prove that $G(u_{n})-G(u_{n}-u)\longrightarrow G(u)$ in $L^{\frac {2\cdot 2^{\ast }_{\alpha }}{2\cdot 2^{\ast }_{\alpha }-1}}(\Omega,L^{\frac {2^{\ast }_{\alpha }}{2^{\ast }_{\alpha }-1}} (\Omega ))$. Since $u_{n}\longrightarrow u$ a.e. in $\Omega$, we have $G(u_{n})-G(u_{n}-u)\longrightarrow G(u)$ a.e. in $\Omega$. Since $u_{n}$ is bounded in $Q^{\alpha,2^{\ast }_{\alpha }}(\Omega,\mathbb {R}^{3})$, we have $G(u_{n})$ is bounded in $L^{\frac {2\cdot 2^{\ast }_{\alpha }}{2\cdot 2^{\ast }_{\alpha }-1}}\left (\Omega,L^{\frac {2^{\ast }_{\alpha }}{2^{\ast }_{\alpha }-1}}(\Omega )\right )$ , so is $G(u_{n})-G(u_{n}-u)$. Then we have $G(u_{n})-G(u_{n}-u)\rightharpoonup G(u)$. Therefore, we only need to prove that

\[ |G(u_{n})-G(u_{n}-u)|_{L^{\frac{2\cdot2^{{\ast}}_{\alpha}}{2\cdot2^{{\ast}}_{\alpha}-1}}(\Omega,L^{\frac{2^{{\ast}}_{\alpha}}{2^{{\ast}}_{\alpha}-1}}(\Omega))}\rightarrow |G(u)|_{L^{\frac{2\cdot2^{{\ast}}_{\alpha}}{2\cdot2^{{\ast}}_{\alpha}-1}}\left(\Omega,L^{\frac{2^{{\ast}}_{\alpha}}{2^{{\ast}}_{\alpha}-1}}(\Omega)\right)}. \]

Indeed, let $A=\left (\frac {1}{|x-y|^{\frac {2^{\ast }_{\alpha }}{2^{\ast }_{\alpha }-1}N-\frac {2\cdot 2^{\ast }_{\alpha }}{2\cdot 2^{\ast }_{\alpha }}\alpha }}\right )^{\frac {2^{\ast }_{\alpha }}{2^{\ast }_{\alpha }-1}}$. Then by using Vitali's convergence theorem we obtain

\begin{align*} & \int_{\Omega}\left(\int_{\Omega}|G(u_{n})-G(u_{n}-u)|^{\frac{2^{{\ast}}_{\alpha}}{2^{{\ast}}_{\alpha}-1}}\,{\rm d}y\right)^{\frac{2\cdot2^{{\ast}}_{\alpha}}{2\cdot2^{{\ast}}_{\alpha}-1}}\,{\rm d}x\\ & \quad=\int_{\Omega}\left(\int_{\Omega}(A)\left(|u_{n}(y)|^{2^{{\ast}}_{\alpha}}|u_{n}(x)|^{2^{{\ast}}_{\alpha}-1}\right.\right.\\ & \quad\left.\left. -\,|u_{n}(y)-u(y)|^{2^{{\ast}}_{\alpha}}|u_{n}(x)-u (x)|^{2^{{\ast}}_{\alpha}-1}\right)^{\frac{2^{{\ast}}_{\alpha}}{2^{{\ast}}_{\alpha}-1}}\,{\rm d}y\right)^{\frac{2\cdot2^{{\ast}}_{\alpha}}{2\cdot2^{{\ast}}_{\alpha}-1}}\,{\rm d}x\\ & \quad=\int_{\Omega}\left(\int_{\Omega}(A)\left(\int_{0}^{1}\frac{\mathrm{d}}{\mathrm{d}t}\left(|u_{n}(y)+(t-1)u(y)|^{2^{{\ast}}_{\alpha}}|u_{n}(x)\right.\right.\right.\\ & \quad\left.\left.\left. -\,(t-1)u(x)|^{2^{{\ast}}_{\alpha}-1}\right)^{\frac{2^{{\ast}}_{\alpha}}{2^{{\ast}}_{\alpha}-1}}\,{\rm d}t\right){\rm d}y\right)^{\frac{2\cdot2^{{\ast}}_{\alpha}}{2\cdot2^{{\ast}}_{\alpha}-1}}\,{\rm d}x\\ & \quad=\int_{0}^{1}\left[\int_{\Omega}\left(\int_{\Omega}(A)\left(\frac{(2^{{\ast}}_{\alpha})^{2}}{2^{{\ast}}_{\alpha}-1}\left\langle|u_{n}(y)+(t-1)u(y)|^{\frac{(2^{{\ast}}_{\alpha})^{2}}{2^{{\ast}}_{\alpha}-1}-2}|u_{n}(x)\right.\right.\right.\right.\\ & \quad\left.\left.\left.\left. +\,(t-1)u(x)|^{2^{{\ast}}_{\alpha}}(u_{n}(y)+(t-1)u(y)),u(y)\vphantom{\left[\int_{\Omega}\left(\int_{\Omega}(A)\left(\frac{(2^{{\ast}}_{\alpha})^{2}}{2^{{\ast}}_{\alpha}-1}\left\langle|u_{n}(y)+(t-1)u(y)|^{\frac{(2^{{\ast}}_{\alpha})^{2}}{2^{{\ast}}_{\alpha}-1}-2}|u_{n}(x)\right.\right.\right.\right.}\right\rangle\right.\right.\right.\\ & \quad\left.\left.\left.+2^{{\ast}}_{\alpha}\left\langle|u_{n}(y)+(t-1)u(y)|^{\frac{(2^{{\ast}}_{\alpha})^{2}}{2^{{\ast}}_{\alpha}-1}}|u_{n}(x)\right.\right.\right.\right.\\ & \quad\left.\left.\left.\left. +(t-1)u(x)|^{2^{{\ast}}_{\alpha}-2}(u_{n}(x)+(t-1)u(x)),u(x)\right\rangle\right){\rm d}y\right)^{\frac{2\cdot2^{{\ast}}_{\alpha}}{2\cdot2^{{\ast}}_{\alpha}-1}}\,{\rm d}x\right]{\rm d}t\\ & \longrightarrow\int_{0}^{1}\left[\int_{\Omega}\left(\int_{\Omega}(A) \left(\frac{(2^{{\ast}}_{\alpha})^{2}}{2^{{\ast}}_{\alpha}-1}\left\langle|tu(y)|^{\frac{(2^{{\ast}}_{\alpha})^{2}}{2^{{\ast}}_{\alpha}-1}-2}|tu(x)|^{2^{{\ast}}_{\alpha}}(tu(y)),u(y)\right\rangle\right.\right.\right.\\ & \quad\left.\left.\left.+2^{{\ast}}_{\alpha}\left\langle|tu(y)|^{\frac{(2^{{\ast}}_{\alpha})^{2}}{2^{{\ast}}_{\alpha}-1}}|tu(x)|^{2^{{\ast}}_{\alpha}-2}(tu(x)),u(x)\right\rangle\right){\rm d}y\right)^{\frac{2\cdot2^{{\ast}}_{\alpha}}{2\cdot2^{{\ast}}_{\alpha}-1}}\,{\rm d}x\right]{\rm d}t\\ & \quad=\int_{\Omega}\left(\int_{\Omega}\left(A\right)\left(|u(y)|^{2^{{\ast}}_{\alpha}}|u(x)|^{2^{{\ast}}_{\alpha}-1}\right)^{\frac{2^{{\ast}}_{\alpha}}{2^{{\ast}}_{\alpha}-1}}\,{\rm d}y\right)^{\frac{2\cdot2^{{\ast}}_{\alpha}}{2\cdot2^{{\ast}}_{\alpha}-1}}\,{\rm d}x\\ & \quad =\int_{\Omega}\left(\int_{\Omega}|G(u)|^{\frac{2^{{\ast}}_{\alpha}}{2^{{\ast}}_{\alpha}-1}}\,{\rm d}y\right)^{\frac{2\cdot2^{{\ast}}_{\alpha}}{2\cdot2^{{\ast}}_{\alpha}-1}}\,{\rm d}x. \end{align*}

Lemma 4.7 Let $c_{\lambda }< c_{0}$ and $(u_{n})_{n\in \mathbb {N}}\subset \mathcal {N}_{\lambda }$ be the Palais-Smale sequence at $c_{\lambda }$. Then $u_{n}\longrightarrow u_{0}\neq 0$ in $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\rm {curl};\Omega )$ along a subsequence, where $u_{0}$ is the nontrivial weak limit in lemma 4.5.

Proof. Let $(u_{n})$ be a $(PS)_{c_{\lambda }}-$sequence such that $(u_{n})\subset \mathcal {N}_{\lambda }$. By lemma 4.4, $(u_{n})$ is bounded and we can assume that $u_{n}\rightharpoonup u_{0}$ in $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )$. Then as in the proof of lemma 4.2, we have $J^{'}_{\lambda }(u_{0})=0$, this implies that $u_{0}$ is a solution for (1.10). Moreover, by the concentration–compactness lemma, we have $u_{n}\longrightarrow u_{0}$ in $L^{2^{\ast }_{\alpha }}_{loc}(\Omega )$, see the same analysis in lemma 4.2. On the other hand, By the compactly perturbed analysis in lemma 4.5, the weak limits $u_{0}\neq 0$. Then by the general principle for the refined nonlocal Brezis-Lieb identity in [Reference Mercuri, Moroz and Van Schaftingen34, proposition 4.3 (ii) (iii)], we have

\begin{align*} & \mathop{\mathrm{lim}}\limits_{n\longrightarrow\infty}\left(\int_{\Omega}(I_{\alpha}\ast|u_{n}|^{2^{{\ast}}_{\alpha}})|u_{n}|^{2^{{\ast}}_{\alpha}}\,{\rm d}x-\int_{\Omega}(I_{\alpha}\ast|u_{n}-u_{0}|^{2^{{\ast}}_{\alpha}})|u_{n}-u_{0}|^{2^{{\ast}}_{\alpha}}\,{\rm d}x\right)\\ & \quad \longrightarrow \int_{\Omega}(I_{\alpha}\ast|u_{0}|^{2^{{\ast}}_{\alpha}})|u_{0}|^{2^{{\ast}}_{\alpha}}\,{\rm d}x, \end{align*}

we hence have

\[ \mathop{\mathrm{lim}}\limits_{n\longrightarrow\infty}\left(J_{\lambda}(u_{n})-J_{\lambda}(u_{n}-u_{0})\right)=J_{\lambda}(u_{0})\geq0, \]

and by lemma 4.6

\[ \mathop{\mathrm{lim}}\limits_{n\longrightarrow\infty}\left(J'_{\lambda}(u_{n})-J'_{\lambda}(u_{n}-u_{0})\right)=J'_{\lambda}(u_{0})=0. \]

Since $J'(u_{n})\longrightarrow 0$ and $u_{n}\longrightarrow u_{0}$ in $L^{2}(\Omega,\mathbb {R}^{3})$, we have

(4.5)\begin{equation} \mathop{\mathrm{lim}}\limits_{n\longrightarrow\infty}J'_{0}(u_{n}-u_{0})=0. \end{equation}

Suppose $\mathop {\mathrm {lim}~\mathrm {inf}}\limits _{n\longrightarrow \infty }||u_{n}-u_{0}||>0$. Since $\mathop {\mathrm {lim}}\limits _{n\longrightarrow \infty }J'_{0}(u_{n}-u_{0})(u_{n}-u_{0})=0$, we infer that

\[ \mathop{\mathrm{lim}~\mathrm{inf}}\limits_{n\longrightarrow\infty}|\nabla\times(u_{n}-u_{0})|_{2}>0. \]

Let $u_{n}-u_{0}=v_{n}+\widetilde {w}_{\Omega }(v_{n})\in \mathcal {V}_{\Omega }\oplus \mathcal {W}_{\Omega }$ according to the Helmholtz decomposition in $W^{\alpha,2^{\ast }_{\alpha }}_{0}(\mathrm {curl};\Omega )$. If $v_{n}\longrightarrow 0$ in $Q^{\alpha,2^{\ast }_{\alpha }}(\Omega,\mathbb {R}^{3})$, then by (4.5) we have $J'_{0}(u_{n}-u_{0})v_{n}\longrightarrow 0$, thus

\begin{align*} |\nabla\times(u_{n}-u_{0})|^{2}_{2}& =|\nabla\times v_{n}|^{2}_{2}=J'_{0}(u_{n}-u_{0})v_{n}\\ & \quad +\int_{\Omega}\left\langle\left(I_{\alpha}\ast|u_{n}-u_{0}|^{2^{{\ast}}_{\alpha}}\right)|u_{n}-u_{0}|^{2^{{\ast}}_{\alpha}-2}(u_{n}-u_{0}),v_{n}\right\rangle\,{\rm d}x\longrightarrow0 \end{align*}

as $n\longrightarrow \infty$, which is a contradiction. Therefore $|v_{n}|_{Q^{\alpha,2^{\ast }_{\alpha }}}$ is bounded away from 0. If $w_{n}:=\widetilde {w}_{\Omega }(u_{n}-u_{0})\in \mathcal {W}_{\Omega }$, then $(w_{n})$ is bounded and since $u_{n}-u_{0}+w_{n}=v_{n}+\widetilde {w}_{\Omega }(v_{n})\in \mathcal {V}_{\Omega }\oplus \mathcal {W}_{\Omega }$, $|u_{n}-u_{0}+w_{n}|_{Q^{\alpha,2^{\ast }_{\alpha }}}$ is bounded away from 0. Choose $t_{n}$ so that $t_{n}(u_{n}-u_{0}+w_{n})\in \mathcal {N}_{\Omega }$, see (3.10). As in (3.12) we have

\[ t_{n}^{2}=\frac{\left(\int_{\Omega}|\nabla\times (u_{n}-u_{0})|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}-1}}} {\left(\int_{\Omega}|I_{\alpha/2}\ast|u_{n}-u_{0}+w_{n}|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}-1}}}, \]

and so $(t_{n})$ is bounded. Then using lemma 3.4, we have

\[ J_{0}(u_{n}-u_{0})\geq J_{0}(t_{n}(u_{n}-u_{0}+w_{n}))-J'_{0}(u_{n}-u_{0})\left[\frac{t^{2}_{n}-1}{2}(u_{n}-u_{0}+t^{2}_{n}w_{n})\right], \]

so by (4.5) and since $u_{n}\longrightarrow u_{0}$ in $L^{2}(\Omega,\mathbb {R}^{3})$,

\[ c_{\lambda}=\mathop{\mathrm{lin}}\limits_{n\longrightarrow\infty} J_{\lambda}(u_{n}-u_{0})=\mathop{\mathrm{lim}}\limits_{n\longrightarrow\infty}J_{0}(u_{n}-u_{0})\geq\mathop{\mathrm{lim}}\limits_{n\longrightarrow\infty}J_{0}(t_{n}(u_{n}-u_{0}+w_{n}))\geq c_{0}, \]

which is a contradiction. Therefore, passing to a subsequence, $u_{n}\longrightarrow u_{0}$, hence also in the $\mathcal {T}-$topology.

Finally, we shall compare $c_{\lambda }$ and $c_{0}$ in some ranges of $\lambda$. Recall from the third identity in (3.11), we note that $c_{0}=\frac {2^{\ast }_{\alpha }-1}{2\cdot 2^{\ast }_{\alpha }} \bar {S}^{\frac {2^{\ast }_{\alpha }}{2^{\ast }_{\alpha }-1}}_{\mathrm {curl},HL}(\Omega )\geq \frac {2^{\ast }_{\alpha }-1}{2\cdot 2^{\ast }_{\alpha }}S_{HL}^{\frac {2^{\ast }_{\alpha }}{2^{\ast }_{\alpha }-1}}$.

Lemma 4.8 Let $\lambda \in (-\lambda _{\nu },-\lambda _{\nu -1}]$ for some $\nu \geq 1$. There holds

\begin{align*} & c_{\lambda}=\mathop{\mathrm{inf}}\limits_{\mathcal{N}_{\lambda}}J_{\lambda}\leq\frac{2^{{\ast}}_{\alpha}-1}{2\cdot2^{{\ast}}_{\alpha}}(\lambda+\lambda_{\nu})^{\frac{2^{{\ast}}_{\alpha}}{2^{{\ast}}_{\alpha}-1}}|\mathrm{diam}\Omega|^{\frac{3\cdot2^{{\ast}}_{\alpha}-\alpha-3}{2^{{\ast}}_{\alpha}-1}},\\ & c_{\lambda}< c_{0}\ if\ \lambda<{-}\lambda_{\nu}+\bar{S}_{\mathrm{curl},HL}(\Omega)|\mathrm{diam}\Omega|^{-\frac{3\cdot2^{{\ast}}_{\alpha}-\alpha-3}{2^{{\ast}}_{\alpha}}}. \end{align*}

Proof. Let $e_{\nu }$ be an eigenvector corresponding to $\lambda _{\nu }$. Then $e_{\nu }\in \mathcal {V}_{\Omega }^+$. Choose $t>0$, $\widetilde {v}\in \widetilde {\mathcal {V}}_{\Omega }$ and $w\in \mathcal {W}_{\Omega }$ so that $u=v+w=t e_{\nu }+\widetilde {\nu }+w\in \mathcal {N}_{\lambda }$. Since $\lambda _{k}\leq \lambda _{\nu }$ for $k<\nu$,

\begin{align*} c_{\lambda}& \leq J_{\lambda}(te_{\nu}+\widetilde{v}+w)\\ & =\frac{\lambda_{\nu}}{2}\int_{\Omega}|te_{v}|^{2}\,{\rm d}x+ \frac{1}{2}\int_{\Omega}|\nabla\times \widetilde{v}|^{2}\,{\rm d}x+\frac{\lambda}{2}\int_{\Omega}|u|^{2}\,{\rm d}x-\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\Omega}|I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\\ & \leq\frac{\lambda_{\nu}}{2}\int_{\Omega}|v|^{2}\,{\rm d}x+\frac{\lambda}{2}\int_{\Omega}|u|^{2}\,{\rm d}x-\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\Omega}|I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x\\ & \leq\frac{\lambda+\lambda_{\nu}}{2}\int_{\Omega}|u|^{2}\,{\rm d}x-\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\int_{\Omega}|I_{\alpha/2}\ast|u|^{2^{{\ast}}_{\alpha}}|^{2}\,{\rm d}x,\\ & \leq\frac{\lambda+\lambda_{\nu}}{2}\int_{\Omega}|u|^{2}\,{\rm d}x-\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\frac{1}{|\mathrm{diam}\Omega|^{3-\alpha}}\left(\int_{\Omega}|u|^{2^{{\ast}}_{\alpha}}\,{\rm d}x\right)^{2}, \end{align*}

where $|\mathrm {diam}\Omega |=\mathop {max}\limits _{x,y\in \Omega }|x-y|$. Then using the Hölder inequality, we get

\begin{align*} c_{\lambda}& \leq\frac{\lambda+\lambda_{\nu}}{2}\left[\left(\int_{\Omega}|u|^{2^{{\ast}}_{\alpha}}dx\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}\right]^{2}|\Omega|^{\frac{2^{{\ast}}_{\alpha}-2}{2^{{\ast}}_{\alpha}}} -\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\frac{1}{|\mathrm{diam}\Omega|^{3-\alpha}}\left(\int_{\Omega}|u|^{2^{{\ast}}_{\alpha}}\,{\rm d}x\right)^{2}\\ & \leq\frac{\lambda\!+\!\lambda_{\nu}}{2}\left[\left(\int_{\Omega}|u|^{2^{{\ast}}_{\alpha}}\,{\rm d}x\right)^{\frac{1}{2^{{\ast}}_{\alpha}}}\right]^{2}|\mathrm{diam}\Omega|^{3\cdot\frac{2^{{\ast}}_{\alpha}-2}{2^{{\ast}}_{\alpha}}} \!-\!\frac{1}{2\cdot2^{{\ast}}_{\alpha}}\frac{1}{|\mathrm{diam}\Omega|^{3-\alpha}}\left(\int_{\Omega}|u|^{2^{{\ast}}_{\alpha}}\,{\rm d}x\right)^{2}\\ & \leq\frac{2^{{\ast}}_{\alpha}-1}{2\cdot2^{{\ast}}_{\alpha}}(\lambda+\lambda_{\nu})^{\frac{2^{{\ast}}_{\alpha}}{2^{{\ast}}_{\alpha}-1}}|\mathrm{diam}\Omega|^{\frac{3\cdot2^{{\ast}}_{\alpha}-\alpha-3}{2^{{\ast}}_{\alpha}-1}}, \end{align*}

where the last inequality follows from the inequality $\frac {A}{2}t^{2}-\frac {1}{p}t^{p}\leq (\frac {1}{2}-\frac {1}{p})A^{\frac {p}{p-2}} (A>0)$.

Since $c_{0}=\frac {2^{\ast }_{\alpha }-1}{2\cdot 2^{\ast }_{\alpha }} \bar {S}^{\frac {2^{\ast }_{\alpha }}{2^{\ast }_{\alpha }-1}}_{\mathrm {curl},HL}(\Omega )$, the second inequality follows immediately.

Proof. Complete of the Proof of theorem 1.4

Note that if $\lambda <-\lambda _{\nu }+\bar {S}_{\mathrm {curl},HL}(\Omega )|\mathrm {diam}\Omega |^{-\frac {3\cdot 2^{\ast }_{\alpha }-\alpha -3}{2^{\ast }_{\alpha }}}$, then $c_{\lambda }< c_{0}$, and by lemma 4.7, $J_{\lambda }$ satisfies the $(PS)_{c_{\lambda }}$ condition, hence satisfies the $(PS)_{c_{\lambda }}^{\mathcal {T}}$ condition. Then statement (a) follows from lemma 4.5, and the remaining statements (b)–(d) are similar to [Reference Mederski and Szulkin33, theorem 1.4] and can be proved by the same strategy.

Funding

The first author Minbo Yang was partially supported by National Natural Science Foundation of China (No. 11971436) and Zhejiang Provincal Natural Science Foundation (No. LZ22A010001). The second author Weiwei Ye was partially supported by Natural Science Research key Projects in Universities in Anhui Province of China (No. 2023AH050425).

Author contribution

The authors declare that they contribute to the paper equally, they all joined in the work of analysis, calculation and organizing the paper.

Conflict of interest

The authors declare that they have no conflict of interest between each other.

Data availability statement

All of the data for the research is included in the manuscript.

References

Amrouche, C., Bernardi, C., Dauge, M. and Girault, V.. Vector potentials in three-dimensional non-smooth domains. Math. Methods Appl. Sci. 21 (1998), 823864.3.0.CO;2-B>CrossRefGoogle Scholar
Azzollini, A., Benci, V., D'Aprile, T. and Fortunato, D.. Existence of static solutions of the semilinear Maxwell equations. Ric. Mat. 55 (2006), 123137.CrossRefGoogle Scholar
Bang, O., Krolikowski, W., Wyller, J. and Rasmussen, J.. Collapse arrest and soliton stabilization in nonlocal nonlinear media. Phys. Rev. E 66 (2002), 046619.CrossRefGoogle ScholarPubMed
Bartsch, T., Dohnal, T., Plum, M. and Reichel, W.. Ground states of a nonlinear curl–curl problem in cylindrically symmetric media. Nonlinear Differ. Equ. Appl. 34 (2016), 2352.Google Scholar
Bartsch, T. and Mederski, J.. Ground and bound state solutions of semilinear time-harmonic Maxwell equations in a bounded domain. Arch. Ration. Mech. Anal. 215 (2015), 283306.CrossRefGoogle Scholar
Bartsch, T. and Mederski, J.. Nonlinear time-harmonic Maxwell equations in an anisotropic bounded medium. J. Funct. Anal. 272 (2017), 43044333.CrossRefGoogle Scholar
Benci, V. and Fortunato, D.. Towards a unified field theory for classical electrodynamics. Arch. Ration. Mech. Anal. 173 (2004), 379414.CrossRefGoogle Scholar
Benci, V. and Rabinowitz, P. H.. Critical point theorems for indefinite functionals. Invent. Math. 52 (1979), 241273.CrossRefGoogle Scholar
Bergé, L. and Couairon, A.. Nonlinear propagation of self-guided ultra-short pulses in ionized gases. Phys. Plasmas 7 (2000), 210230.CrossRefGoogle Scholar
Brezis, H. and Nirenberg, L.. Positive solutions of nonlinear elliptic equations involving critical Sobolev exponents. Commun. Pure Appl Math. 36 (1983), 437477.CrossRefGoogle Scholar
Capozzi, A., Fortunato, D. and Palmieri, G.. An existence result for nonlinear elliptic problems involving critical Sobolev exponent. Ann. Inst. H. Poincaré Anal. Non Linéaire 2 (1985), 463470.CrossRefGoogle Scholar
Cerami, G., Solimini, S. and Struwe, M.. Some existence results for superlinear elliptic boundary value problems involving critical exponents. J. Funct. Anal. 69 (1986), 289306.CrossRefGoogle Scholar
D'Aprile, T. and Siciliano, G.. Magnetostatic solutions for a semilinear perturbation of the Maxwell equations. Adv. Differ. Equ. 16 (2011), 435466.Google Scholar
Dörfler, W., Lechleiter, A., Plum, M., Schneider, G. and Wieners, C.. Photonic crystals: mathematical analysis and numerical approximation (Basel: Springer, 2012).Google Scholar
Dalfovo, F. et al. Theory of Bose-Einstein condensation in trapped gases. Rev. Mod. Phys. 71 (1999), 463512.CrossRefGoogle Scholar
du Plessis, N.. An introduction to potential theory. University Mathematical Monographs vol. 7 (Edinburgh: Oliver and Boyd, 1970).Google Scholar
Du, Le. and Yang, M. B.. Uniqueness and nondegeneracy of solutions for a critical nonlocal equation. Discrete Contin. Dyn. Syst. 39 (2019), 58475866.CrossRefGoogle Scholar
Gao, F. S., Silva, E., Yang, M. B. and Zhou, J. Z.. Existence of solutions for critical Choquard equations via the concentration compactness method. Proc. R. Soc. Edinburgh 150 (2020), 921954.CrossRefGoogle Scholar
Gao, F. S. and Yang, M. B.. The Brezis-Nirenberg type critical problem for the nonlinear Choquard equation. Sci. China Math. 61 (2018), 12191242.CrossRefGoogle Scholar
Guo, Q. Q. and Mederski, J.. Ground states of nonlinear Schrödinger equations with sum of periodic and inverse-square potentials. J. Differ. Equ. 260 (2016), 41804202.CrossRefGoogle Scholar
Kirsch, A. and Hettlich, F.. The mathematical theory of time-harmonic Maxwell's equations: expansion, integral, and variational methods, Applied Mathematical Science vol. 190 (Cham: Springer, 2015).CrossRefGoogle Scholar
Krolikowski, W., Bang, O., Rasmussen, J. J. and Wyller, J.. Modulational instability in nonlocal nonlinear kerr media. Phys. Rev. E 64 (2001), 016612.CrossRefGoogle ScholarPubMed
Leinfelder, H.. Gauge invariance of Schrödinger operators and related spectral properties. J. Oper. Theorey 9 (1983), 163179.Google Scholar
Lieb, E. and Loss, M.. Analysis, graduate studies in mathematics (Providence: American Mathematical Society, 2001).Google Scholar
Lions, P. L.. The concentration-compactness principle in the calculus of variations. The limit case. Part I and II. Rev. Mat. Iberoam. 1 (1985), 145201 [2 (1985) 45–121].CrossRefGoogle Scholar
Litvak, A. G.. Self-focusing of powerful light beams by thermal effects. JETP Lett. 4 (1966), 230232.Google Scholar
Mandel, R.. Ground states for Maxwell's equation in nonlocal nonlinear media. Partial Differ. Equ. Appl. 3 (2022), Paper No. 22, 16pp.CrossRefGoogle Scholar
Mederski, J.. Ground states of time-harmonic semilinear Maxwell equations in $\mathbb {R}^3$ with vanishing permittivity. Arch. Ration. Mech. Anal. 218 (2015), 825861.CrossRefGoogle Scholar
Mederski, J.. Ground states of a system of nonlinear Schrodinger equations with periodic potentials. Commun. Partial Differ. Equ. 41 (2016), 14261440.CrossRefGoogle Scholar
Mederski, J.. The Brezis-Nirenberg problem for the curl–curl operator. J. Funct. Anal. 274 (2018), 13451380.CrossRefGoogle Scholar
Mederski, J.. Nonlinear time-harmonic Maxwell equations in a bounded domain Lack of compactness. Sci. China Math. 61 (2018), 19631970.CrossRefGoogle Scholar
Mederski, J., Schino, J. and Szulkin, A.. Multiple solutions to a nonlinear curl–curl problem in $\mathbb {R}^3$. Arch. Ration. Mech. Anal. 236 (2019), 253288.CrossRefGoogle Scholar
Mederski, J. and Szulkin, A.. A Sobolev-type inequality for the curl operator and ground states for the curl–curl equation with critical Sobolev exponent. Arch. Ration. Mech. Anal. 241 (2021), 18151842.CrossRefGoogle Scholar
Mercuri, C., Moroz, V. and Van Schaftingen, J.. Groundstates and radial solutions to nonlinear Schrödinger-Poisson-Slater equations at the critical frequency. Calc. Var. 55 (2016), Art. 146, 58pp.CrossRefGoogle Scholar
Monk, P.. Numerical Mathamatics and Scientific Computation: Finite element methods for Maxwell's equations (New York: Oxford University Press, 2003), xiv+450.CrossRefGoogle Scholar
Moroz, V. and Schaftingen, J. V.. A guide to the Choquard equation. J. Fixed Point Theory Appl. 19 (2017), 773813.CrossRefGoogle Scholar
Nikolov, N. I., Neshev, D., Bang, O. and Królikowski, W. Z.. Quadratic solitons as nonlocal solitons. Phys. Rev. E 68 (2003), 036614.CrossRefGoogle ScholarPubMed
Padilla, W. J., Basov, D. N. and Smith, D. R.. Negative refractive index metamaterials. Mater. Today 9 (2006), 2835.CrossRefGoogle Scholar
Picard, R., Weck, N. and Witsch, K. J.. Time-harmonic Maxwell equations in the exterior of perfectly conducting, irregular obstacles. Analysis 21 (2001), 231263.CrossRefGoogle Scholar
Qin, D. D. and Tang, X. H.. Time-harmonic Maxwell equations with asymptotically linear polarization. Z. Angew. Math. Phys. 67 (2016), 3967.CrossRefGoogle Scholar
Reimbert, C., Minzoni, A. and Smyth, N.. Spatial soliton evolution in nematic liquid crystals in the nonlinear local regime. J. Opt. Soc. Am. B: Opt. Phys. 23 (2006), 294301.CrossRefGoogle Scholar
Solimini, S.. A note on compactness-type properties with respect to Lorentz norms of bounded subsets of a Sobolev space. Ann. Inst. H. Poincaré Anal. Non Linéaire 12 (1995), 319337.CrossRefGoogle Scholar
Stuart, C. A. and Zhou, H. S.. Existence of guided cylindrical TM-models in a homogeneous self-trapping dielectric. Ann. Inst. H. Poincare Anal. Non Lineaire 18 (2001), 6996.CrossRefGoogle Scholar
Stuart, C. A. and Zhou, H. S.. Axisymmetric TE-modes in a self-focusing dielectric. SIAM J. Math. Anal. 37 (2005), 218237.CrossRefGoogle Scholar
Szulkin, A. and Weth, T.. Ground state solutions for some indefinite variational problems. J. Funct. Anal. 257 (2009), 38023822.CrossRefGoogle Scholar
Zeng, X. Y.. Cylindrically symmetric ground state solutions for curl–curl equations with critical exponent. Z. Angew. Math. Phys. 68 (2017), Paper No. 135, 12pp.CrossRefGoogle Scholar