2.1 Moments and deviations

We already alluded to how the full rank problem for the random matrix $\mathbb{A}$ over $\mathbb{F}_q$ can be viewed as a random constraint satisfaction problem. Indeed, suppose we draw a right-hand side vector ${\boldsymbol{y}}\in \mathbb{F}_q^{{\boldsymbol{m}}}$ independently of $\mathbb{A}$ . Then $\mathbb{A}$ has full row rank w.h.p. iff the random linear system $\mathbb{A} x={\boldsymbol{y}}$ admits a solution w.h.p. For if ${\text{rk}}\mathbb{A}\lt {\boldsymbol{m}}$ , then the image $\mathbb{A} \mathbb{F}_q^n$ is a proper subspace of $\mathbb{F}_q^{{\boldsymbol{m}}}$ and thus the random linear system $\mathbb{A} x={\boldsymbol{y}}$ has a solution with probability at most $1/q$ . Naturally, the random linear system is nothing but a random constraint satisfaction problem with ${\boldsymbol{m}}$ constraints and $n$ variables.

Over the past two decades the second moment method has emerged as the default approach to pinpointing satisfiability thresholds of random constraint satisfaction problems [Reference Achlioptas and Moore3, Reference Achlioptas, Naor and Peres4]. Indeed, one of the first success stories was the random $3$ -XORSAT problem, which boils down directly to a full rank problem over $\mathbb{F}_2$ [Reference Dubois and Mandler21]. In fact, as we saw in Example 1.3, to mimic $3$ -XORSAT we just set $q=2$ , ${\boldsymbol{d}}=\text{Po}(d)$ for some $d\gt 0$ and ${\boldsymbol{k}}=3$ deterministically. In addition, draw ${\boldsymbol{y}}\in \mathbb{F}_2^{{\boldsymbol{m}}}$ uniformly and independently of everything else.

We try the second moment method on the number ${\boldsymbol{Z}}={\boldsymbol{Z}}(\mathbb{A},{\boldsymbol{y}})$ of solutions to $\mathbb{A} x={\boldsymbol{y}}$ given $\mathfrak{A}$ . Since ${\boldsymbol{y}}$ is independent of $\mathbb{A}$ , for any fixed vector $x\in \mathbb{F}_2^n$ the event $\mathbb{A} x={\boldsymbol{y}}$ has probability $2^{-{\boldsymbol{m}}}$ . Consequently,

(2.1) \begin{align} \mathbb{E}_{\mathfrak{A}}[{\boldsymbol{Z}}]=2^{n-{\boldsymbol{m}}}. \end{align}

Hence, (2.1) recovers the obvious condition that we cannot have more rows than columns. Since ${\boldsymbol{m}}\sim \text{Po}(dn/3)$ , (2.1) boils down to $d\lt 3$ .

The second moment method now rests on the hope that we may be able to show that $\mathbb{E}_{\mathfrak{A}}[{\boldsymbol{Z}}^2]\sim \mathbb{E}_{\mathfrak{A}}[{\boldsymbol{Z}}]^2$ . Then Chebyshev’s inequality would imply ${\boldsymbol{Z}}\sim \mathbb{E}_{\mathfrak{A}}[{\boldsymbol{Z}}]$ w.h.p., and thus, in light of (2.1), that $\mathbb{A} x={\boldsymbol{y}}$ has a solution w.h.p.

Concerning the computation of $\mathbb{E}_{\mathfrak{A}}[{\boldsymbol{Z}}^2]$ , because the set of solutions is either empty or a translation of the kernel, we obtain

(2.2) \begin{align} \mathbb{E}_{\mathfrak{A}}[{\boldsymbol{Z}}^2]&=\sum _{\sigma,\tau \in \mathbb{F}_q^n}{\mathbb{P}}_{\mathfrak{A}}\left \lbrack{\mathbb{A}\sigma =\mathbb{A}\tau ={\boldsymbol{y}}}\right \rbrack =\sum _{\sigma,\tau \in \mathbb{F}_q^n}{\mathbb{P}}_{\mathfrak{A}}\left \lbrack{\mathbb{A}\sigma ={\boldsymbol{y}}}\right \rbrack{\mathbb{P}}_{\mathfrak{A}}\left \lbrack{\sigma -\tau \in \ker \mathbb{A}}\right \rbrack \nonumber \\[5pt] &=\mathbb{E}_{\mathfrak{A}}\left \lbrack{{\boldsymbol{Z}}}\right \rbrack \mathbb{E}_{\mathfrak{A}}|\ker \mathbb{A}|. \end{align}

To calculate the expected kernel size we notice that the probability that a vector $x$ is in the kernel depends on its Hamming weight. For instance, the zero vector always belongs to the kernel, while the all-ones vector ${\mathbb{1}}$ does not w.h.p. More systematically, invoking inclusion/exclusion, we find that for a vector $x$ of Hamming weight $w$ we have ${\mathbb{P}}_{\mathfrak{A}}\left \lbrack{x\in \ker \mathbb{A}}\right \rbrack \sim \left \lbrack{(1+(1-2w/n)^3)/2}\right \rbrack ^{{\boldsymbol{m}}}.$ Since the total number of such vectors comes to $\binom n{w}$ , we obtain

(2.3) \begin{align} \mathbb{E}_{\mathfrak{A}}|\ker \mathbb{A}|&=\sum _{w=0}^n\binom nw\left ({\frac{1+(1-2w/n)^3}{2}}\right )^{{\boldsymbol{m}}}. \end{align}

Taking logarithms, invoking Stirling’s formula and parametrising $w=z n$ , we simplify (2.3) to

(2.4) \begin{align} \log \,\mathbb{E}_{\mathfrak{A}}|\ker \mathbb{A}|&\sim n\cdot \max _{z\in [0,1]}-z\log z-(1-z)\log (1-z)+\frac{{\boldsymbol{m}}}{n}\log \frac{1+(1-2z)^3}{2}&&\mbox{(cf.\,[21]).} \end{align}

If we substitute $z=1/2$ into (2.4), the expression further simplifies to $(n-{\boldsymbol{m}})\log 2$ . Hence, if the maximum is attained at another value $z\neq 1/2$ , then (2.4) yields $\mathbb{E}_{\mathfrak{A}}|\ker \mathbb{A}|\gg 2^{n-{\boldsymbol{m}}}$ and the second moment method fails.

Figure 3 displays (2.4) for $d=2.5$ and $d=2.7$ . While for $d=2.5$ the function takes its maximum at $z=1/2$ , for $d=2.7$ the maximum is attained at $z\approx 0.085$ . However, the true random 3-XORSAT threshold is $d\approx 2.75$ [Reference Dubois and Mandler21]. Thus, the naive second moment calculation falls short of the real threshold.

Figure 3. The r.h.s. of (2.4) for $d=2.5$ (blue) and $d=2.7$ (red) in the interval $[0,\frac 12]$ .

How so? The expression (2.4) does not determine the ‘likely’ but the expected size of the kernel, a value prone to large deviations effects. Indeed, because the number of vectors in the kernel scales exponentially with $n$ , an exponentially unlikely event that causes an exceptionally large kernel may end up dominating $\mathbb{E}_{\mathfrak{A}}|\ker \mathbb{A}|$ . Precisely such an event manifests itself in the left local maximum in Fig. 3. Moreover, as we approach the satisfiability threshold such large deviations issues are compounded by a diminishing error tolerance. Indeed, while for $d=2.5$ the value at $z=1/2$ just swallows the spurious maximum, this is no longer the case for $d=2.7$ .

For random $k$ -XORSAT Dubois and Mandler managed to identify the precise large deviations effect at work. It stems from fluctuations of a densely connected sub-graph of $\mathbb{G}$ called the 2-core, obtained by iteratively pruning nodes of degree less than two along with their neighbours (if any). Dubois and Mandler pinpointed the 3-XORSAT threshold by applying the second moment method to the minor $\mathbb{A}^{(2)}$ induced by ${\mathbb{G}}^{(2)}$ while conditioning on the 2-core having its typical dimensions.

The technical difficulty is that the rows of $\mathbb{A}^{(2)}$ are no longer independent. Indeed, $\mathbb{A}^{(2)}$ is distributed as a random matrix with a truncated Poisson ${\boldsymbol{d}}^{(2)}\sim \text{Po}_{\geq 2}(d')$ with $d'=d'(d,k)\gt 0$ as the distribution of the variable degrees. Unfortunately, the given-degrees model leads to a fairly complicated moment computation. Instead of the humble one-dimensional problem from (2.4) we now face parameters $(z_i)_{i\geq 2}$ that gauge the fraction of variables of each possible degree $i$ set to one. Additionally, on the constraint side we need to keep track of the number of equations with zero and with two variables set to one. Of course, these variables are tied together through the constraint that the total Hamming weight on the variable side matches that on the constraint side.

With a deal of diligence Dubois and Mandler managed to solve this optimisation problem. However, even just the step on to check degrees $k\gt 3$ turns out to be tricky because now we need to keep track of all the possible ways in which a $k$ -ary parity constraint can be satisfied [Reference Dietzfelbinger, Goerdt, Mitzenmacher, Montanari, Pagh and Rink19, Reference Pittel44]. Yet even these difficulties are eclipsed by those that result from merely advancing to fields of size $q=3$ [Reference Goerdt and Falke23].

Not to mention entirely general degree distributions ${\boldsymbol{d}},{\boldsymbol{k}}$ and general fields $\mathbb{F}_q$ as in Theorem 1.1. The ensuing optimisation problem comes in terms of variables $(z_i)_{i\in{\text{supp}}{\boldsymbol{d}}}$ that range over the space $\mathcal{P}(\mathbb{F}_q)$ of probability distributions on $\mathbb{F}_q$ . Additionally, there is a second set of variables $(\hat z_{\chi _1,\ldots,\chi _\ell })_{\ell \in{\text{supp}}{\boldsymbol{k}},\,\chi _1,\ldots,\chi _\ell \in{\text{supp}}\boldsymbol{\chi }}$ to go with the rows of $\mathbb{A}$ whose non-zero entries are precisely $\chi _1,\ldots,\chi _\ell$ . These variables range over probability distributions on solutions $\sigma \in \mathbb{F}_q^\ell$ to $\chi _1\sigma _1+\cdots +\chi _\ell \sigma _\ell =0$ . In terms of these variables we would need to solve

(2.5) \begin{align} &\max \sum _{\sigma \in \mathbb{F}_q}\mathbb{E}\left \lbrack{({\boldsymbol{d}}-1)z_{{\boldsymbol{d}}}(\sigma )\log z_{{\boldsymbol{d}}}(\sigma )}\right \rbrack \nonumber\\[5pt] &\quad -\frac{d}{k}\mathbb{E}\left \lbrack{\sum _{\substack{\sigma _1,\dots,\sigma _{{\boldsymbol{k}}}\in \mathbb{F}_q\\ {\boldsymbol{\chi }_{1,1}\sigma _1+\cdots +\boldsymbol{\chi }_{1,{\boldsymbol{k}}}\sigma _{{\boldsymbol{k}}}=0}}}\hat z_{\boldsymbol{\chi }_{1,1},\ldots,\boldsymbol{\chi }_{1,{\boldsymbol{k}}}}(\sigma _1,\ldots,\sigma _{{\boldsymbol{k}}})\log \hat z_{\boldsymbol{\chi }_{1,1},\ldots,\boldsymbol{\chi }_{1,{\boldsymbol{k}}}}(\sigma _1,\ldots,\sigma _{{\boldsymbol{k}}})}\right \rbrack \\[5pt] &\quad \mbox{s.t.}\quad \mathbb{E}[{\boldsymbol{d}} z_{{\boldsymbol{d}}}(\tau )]=\mathbb{E}\left \lbrack{\sum _{\substack{\sigma _1,\dots,\sigma _{{\boldsymbol{k}}}\in \mathbb{F}_q\\ {\boldsymbol{\chi }_{1,1}\sigma _1+\cdots +\boldsymbol{\chi }_{1,{\boldsymbol{k}}}\sigma _{{\boldsymbol{k}}}=0}}}{\boldsymbol{k}}{\mathbb{1}}\left \{{\sigma _1=\tau }\right \}\hat z_{\boldsymbol{\chi }_{1,1},\ldots,\boldsymbol{\chi }_{1,{\boldsymbol{k}}}}(\sigma _1,\ldots,\sigma _{{\boldsymbol{k}}})}\right \rbrack \quad \mbox{for all }\tau \in \mathbb{F}_q\nonumber . \end{align}

On an high level, (2.5) is not so different from (2.4): The first summand in (2.5) corresponds to the number of vectors with a specified number of components of each field element, taking into account the different numbers of non-zero entries of the columns. The remaining part corresponds to the probability that any such vector satisfies all equations, taking into account the number of field elements of each type in a random equation. Finally, while the frequencies of the field elements appear decoupled for rows and columns in the first line of (2.5), the second line ensures that only compatible frequencies are considered after all. As in random 3-XORSAT, a simple calculation shows that the value of (2.5) evaluated at the ‘equitable’ solution

(2.6) \begin{align} z_i(\sigma )&=q^{-1}&\hat z_{\chi _1,\ldots,\chi _\ell }(\sigma _1,\ldots,\sigma _\ell )&=q^{1-\ell }&&\mbox{for all }i, \chi _1,\ldots,\chi _\ell \end{align}

hits the value $(1-d/k)\log q$ , which matches the normalised first moment $n^{-1}\log \mathbb{E}_{\mathfrak{A}}[{\boldsymbol{Z}}]$ .

In summary, the second moment method hardly seems like a promising path towards Theorem 1.1. Not only does (2.5) seem unwieldy as even for very special cases of ${\boldsymbol{d}},{\boldsymbol{k}}$ an analytic solution remains elusive [Reference Goerdt and Falke23]. Even worse, just in the case of ‘unabridged’ random $k$ -XORSAT large deviations effects may cause spurious maxima. In effect, even if we could miraculously figure out the precise conditions for (2.5) being attained at the uniform solution, this would hardly determine for what ${\boldsymbol{d}},{\boldsymbol{k}}$ the random matrix $\mathbb{A}$ actually has full row rank w.h.p.

2.2 Quenching and truncating

The large deviations issues ultimately result from our attempt at computing the mean of $|\ker \mathbb{A}|$ , a (potentially) exponential quantity. The mathematical physics prescription is to compute the expectation of its logarithm instead [Reference Mézard40]. In the present algebraic setting this comes down to computing the mean of the nullity $\text{nul}\mathbb{A}=\dim \ker \mathbb{A}$ , or equivalently of the rank ${\text{rk}}\mathbb{A}=n-\text{nul}\mathbb{A}$ . This ‘quenched average’ is always of order $O(n)$ and therefore immune to large deviations effects. In fact, even if on some unfortunate event of exponentially small probability $\exp\!(-\Omega (n))$ the kernel of $\mathbb{A}$ were quite large, the ensuing boost to $\mathbb{E}_{\mathfrak{A}}[\text{nul}\mathbb{A}]$ remains negligible.

Yet computing the quenched average $\mathbb{E}_{\mathfrak{A}}[\text{nul}\mathbb{A}]$ does not suffice to prove Theorem 1.1. Indeed, (1.4) already provides an asymptotic formula for $\mathbb{E}_{\mathfrak{A}}[\text{nul}\mathbb{A}]$ . But as we saw due to the normalisation on the l.h.s. (1.4) merely implies that ${\text{rk}}\mathbb{A}={\boldsymbol{m}}-o(n)$ w.h.p. To actually prove that ${\text{rk}}\mathbb{A}={\boldsymbol{m}}$ w.h.p. we will combine the quenched computation with a truncated moment argument calculation. Specifically, we will harness an enhanced version of (1.4) to prove that under the assumptions of Theorem 1.1 the only combinatorially meaningful solutions to (2.5) asymptotically coincide with the equitable solution (2.6), around which we will subsequently expand (2.5) carefully.

To carry this programme out, let ${\boldsymbol{x}}_{\mathbb{A}}=({\boldsymbol{x}}_{\mathbb{A},i})_{i\in [n]}\in \mathbb{F}_q^n$ be a random vector from the kernel of $\mathbb{A}$ . Consider the event

(2.7) \begin{align} {\mathfrak{O}}&={\left \{{A \;:\; \sum _{\sigma,\tau \in \mathbb{F}_q}\sum _{i,j=1}^n\left |{{\mathbb{P}}\left \lbrack{{\boldsymbol{x}}_{A,i}=\sigma,\ {\boldsymbol{x}}_{A,j}=\tau \mid A}\right \rbrack -q^{-2}}\right |=o(n^2)}\right \}.} \end{align}

Then by Chebyshev’s inequality on $\mathfrak{O}$ w.h.p. we have

\begin{align*} \sum _{i=1}^n{\mathbb{1}}\left \{{{\boldsymbol{d}}_i=\ell,\,{\boldsymbol{x}}_{\mathbb{A},i}=\sigma }\right \}={\mathbb{P}}\left \lbrack{{\boldsymbol{d}}=\ell }\right \rbrack n/q+o(n)&&\mbox{for all }\sigma \in \mathbb{F}_q,\ \ell \in{\text{supp}}{\boldsymbol{d}}. \end{align*}

Hence, on $\mathfrak{O}$ the only combinatorially relevant value of $z_\ell (\sigma )$ from (2.5) is the uniform $1/q$ for every $\ell,\sigma$ , because for every $\ell$ asymptotically almost all kernel vectors set about an equal number of variables of degree $\ell$ to each of the $q$ possible values. Thanks to this observation will prove that w.h.p.

(2.8) \begin{align} \mathbb{E}_{\mathfrak{A}}\left \lbrack{{\boldsymbol{Z}}\cdot {\mathbb{1}}\left \{{\mathbb{A}\in \mathfrak{O}}\right \}}\right \rbrack &\sim \mathbb{E}_{\mathfrak{A}}\left \lbrack{{\boldsymbol{Z}}}\right \rbrack \sim q^{n-{\boldsymbol{m}}}&&\mbox{and} \end{align}

(2.9) \begin{align} \mathbb{E}_{\mathfrak{A}}\left \lbrack{{\boldsymbol{Z}}^2\cdot {\mathbb{1}}\left \{{\mathbb{A}\in \mathfrak{O}}\right \}}\right \rbrack &\sim \mathbb{E}_{\mathfrak{A}}\left \lbrack{{\boldsymbol{Z}}}\right \rbrack ^2, \end{align}

provided that (1.3) is satisfied. Theorem (1.1) will turn out to be an easy consequence of (2.8)–(2.9) and Corollary 1.2 of Theorem 1.1.

Thus, the challenge is to prove (2.8)–(2.9). Specifically, while the second asymptotic equality in (2.8) is easy, the proof of the first is where we require knowledge of the ‘quenched average’ (1.4). In fact, instead of just applying (1.4) as is we will need to perform a ‘quenched’ computation for a slightly enhanced random matrix from scratch. Second, the key challenge towards the proof of (2.9) is to obtain an exact asymptotic equality here, rather than the weaker estimate $\mathbb{E}_{\mathfrak{A}}\left \lbrack{{\boldsymbol{Z}}^2\cdot {\mathbb{1}}\left \{{\mathbb{A}\in \mathfrak{O}}\right \}}\right \rbrack =O(\mathbb{E}_{\mathfrak{A}}\left \lbrack{{\boldsymbol{Z}}}\right \rbrack ^2)$ . This will require a meticulous expansion of the second moment around the uniform solution, which will involve the detailed analysis of the lattices generated by integer vectors that encode conceivable values of $z_i,\hat z_{\chi _1, \ldots, \chi _\ell }$ from (2.5).

2.3 Specified $d_1,\ldots,d_n$ and $k_1,\ldots,k_m$

Given two positive integers $n$ and $m=m(n)$ , consider now two arrays $(d_i^{(n)})_{1\leq i\leq n}$ and $(k_i^{(m)})_{1 \leq i \leq m}$ of non-negative integers such that for all $n$ ,

\begin{equation*}\sum _{i=1}^n d_i^{(n)}=\sum _{j=1}^m k_j^{(m)}.\end{equation*}

We aim to find conditions on the arrays $(d_i^{(n)})_{1\leq i\leq n}$ and $(k_i^{(m)})_{1 \leq i \leq m}$ and the sequence $(m(n))_{n \geq 1}$ which guarantee full row rank for the corresponding matrix in this fixed-degree setting as $n \to \infty$ and are analogous to (1.3). Throughout, we abbreviate $m(n)= m$ . Let ${\boldsymbol{d}}_n$ denote a uniformly chosen element from the sequence $(d_i^{(n)})_{1\leq i\leq n}$ and ${\boldsymbol{k}}_n$ a uniformly chosen element from the sequence $(k_i^{(m)})_{1 \leq i \leq m}$ . Assume that $(d_i^{(n)})_{1\leq i\leq n}$ and $(k_i^{(m)})_{1 \leq i \leq m}$ satisfy the following conditions in terms of the uniformly chosen degrees ${\boldsymbol{d}}_n$ and ${\boldsymbol{k}}_n$ :

1. (P1) There exist (integer-valued) random variables ${\boldsymbol{d}}, {\boldsymbol{k}} \geq 0$ with ${\mathbb{P}}({\boldsymbol{d}} =0)\lt 1$ and ${\mathbb{P}}({\boldsymbol{k}} \geq 3)=1$ such that ${\boldsymbol{d}}_n\stackrel{d}{\longrightarrow } {\boldsymbol{d}}$ and ${\boldsymbol{k}}_n\stackrel{d}{\longrightarrow } {\boldsymbol{k}}$ ;

2. (P2) $\mathbb{E}[{\boldsymbol{d}}], \mathbb{E}[{\boldsymbol{k}}] \lt \infty$ and $\mathbb{E}[{\boldsymbol{d}}_n] \to \mathbb{E}[{\boldsymbol{d}}]$ , $\mathbb{E}[{\boldsymbol{k}}_n] \to \mathbb{E}[{\boldsymbol{k}}]$ as $n \to \infty$ .

3. (P3) $\mathbb{E}[{\boldsymbol{d}}^2], \mathbb{E}[{\boldsymbol{k}}^2] \lt \infty$ and $\mathbb{E}[{\boldsymbol{d}}_n^2] \to \mathbb{E}[{\boldsymbol{d}}^2]$ , $\mathbb{E}[{\boldsymbol{k}}_n^2] \to \mathbb{E}[{\boldsymbol{k}}^2]$ as $n \to \infty$ .

4. (P4) For some $\eta \gt 0$ , $\mathbb{E}[{\boldsymbol{d}}^{2+\eta }] \lt \infty$ and $\mathbb{E}[{\boldsymbol{d}}_n^{2+\eta }] \to \mathbb{E}[{\boldsymbol{d}}^{2+\eta }]$ .

5. (P5) $m \sim \mathbb{E}\left \lbrack{{\boldsymbol{d}}}\right \rbrack n/ \mathbb{E}\left \lbrack{{\boldsymbol{k}}}\right \rbrack$ .

6. (P6) For all $m$ and $j \in [m]$ , $k_j^{(m)}\ge 3$ .

7. (P7) For all $n$ , $\gcd ({\text{supp}}({\boldsymbol{d}}))=\gcd (d_i^{(n)})_{1 \leq i \leq n})$ .

Conditions (P1)-(P3) correspond to standard regularity conditions for the non-bipartite version of the configuration model (see Condition 7.8 in [Reference van der Hofstad25], for example).

Let $D(x)$ and $K(x)$ denote the probability generating functions for ${\boldsymbol{d}}$ and ${\boldsymbol{k}}$ , respectively. We also abbreviate $d=\mathbb{E}\left \lbrack{{\boldsymbol{d}}}\right \rbrack$ and $k=\mathbb{E}\left \lbrack{{\boldsymbol{k}}}\right \rbrack$ as before. We may then define

(2.10) \begin{align} \Phi \;:\; [0,1]&\to{\mathbb{R}},&z&\mapsto D\left (1-K'(z)/k\right )-\frac{d}{k}\left ({1-K(z)-(1-z)K'(z)}\right ). \end{align}

Finally, to make the difference to the i.i.d. degree case apparent, we denote the random matrix constructed by generating a uniformly random simple Tanner graph based on the fixed-degree sequences $(d_i^{(n)})_{1\leq i\leq n}$ and $(k_i^{(m)})_{1 \leq i \leq m}$ by $\underline{\mathbb{A}}$ . Again, the non-zero entries of $\underline{\mathbb{A}}$ are i.i.d. copies of the random variable $\boldsymbol{\chi }$ .

We first prove Proposition 2.1. Then, we prove Theorem 1.1 by showing that w.h.p. $\mathbb{A}\in \mathfrak{O}$ if ${\boldsymbol{m}} \sim \text{Po}(dn/k)$ , $({\boldsymbol{d}}_i)_{i \geq 1}$ and $({\boldsymbol{k}}_j)_{j \geq 1}$ are i.i.d. copies of ${\boldsymbol{d}}$ and ${\boldsymbol{k}}$ . Theorem 1.1 then follows from Proposition 2.1.

In the current case, $\mathfrak{A}$ simply is the $\sigma$ -algebra generated by the numbers ${\boldsymbol{m}}(\chi _1,\ldots,\chi _\ell )$ of equations of degree $\ell \geq 3$ with coefficients $\chi _1,\ldots,\chi _\ell \in \mathbb{F}_q^*$ , since all degrees are deterministic. When $\mathfrak{A}$ is used as a subscript, it serves as a notation that suppresses explicit mentioning of $m$ , $(d_i^{(n)})_{i=1}^n$ and $(k_i^{(m)})_{1 \leq i \leq m}$ . As discussed above, it suffices to prove (2.8) (with ${\boldsymbol{m}}=m$ ) and (2.9) in the more general model $\underline{\mathbb{A}}$ . We observe that (2.8) follows immediately by hypothesis (a) of Proposition 2.1, as w.h.p.,

(2.11) \begin{align} &\mathbb{E}_{\mathfrak{A}}\left \lbrack{{\boldsymbol{Z}}\cdot {\mathbb{1}}\left \{{\underline{\mathbb{A}}\in \mathfrak{O}}\right \}}\right \rbrack \nonumber \\[5pt] &\quad =\sum _{A\in \mathfrak{O}} \sum _{\sigma \in \mathbb{F}^n} \sum _{y\in \mathbb{F}^m}{\mathbb{P}}_{\mathfrak{A}}\left \lbrack{\underline{\mathbb{A}}=A,{\boldsymbol{y}}=y}\right \rbrack {\mathbb{1}}\left \{{A\sigma =y}\right \} = q^{n-m} \sum _{A\in \mathfrak{O}}{\mathbb{P}}_{\mathfrak{A}}\left \lbrack{\underline{\mathbb{A}}=A}\right \rbrack \sim q^{n-m}. \end{align}

Thus, to complete the proof for Proposition 2.1 it suffices to prove (2.9).

2.4 The truncated first moment

We start our discussion by verifying condition (a) of Proposition 2.1 for i.i.d. ${\boldsymbol{d}}_i$ and ${\boldsymbol{k}}_j$ . Hence, let us restrict to the ‘i.i.d. version’ of $\mathbb{A}$ , that is, ${\boldsymbol{d}}_1,\ldots,{\boldsymbol{d}}_n$ and ${\boldsymbol{k}}_1,\ldots, {\boldsymbol{k}}_{{\boldsymbol{m}}}$ are i.i.d. copies of ${\boldsymbol{d}}$ and ${\boldsymbol{k}}$ . Although we know the approximate nullity (1.4) of $\mathbb{A}$ already, this does not suffice to actually prove that $\mathfrak{O}$ is a ‘likely’ event. To this end we need to study a slightly modified matrix instead. Specifically, for an integer $t\geq 0$ obtain $\mathbb{A}_{[t]}$ from $\mathbb{A}$ by adding $t$ more rows that contain precisely three non-zero entries. The positions of these non-zero entries are chosen uniformly, mutually independently and independently of everything else, and the non-zero entries themselves are independent copies of $\boldsymbol{\chi }$ . We require the following lower bound on the rank of $\mathbb{A}_{[t]}$ .

Proposition 2.3. If (1.3) is satisfied then there exists $\delta _0=\delta _0({\boldsymbol{d}},{\boldsymbol{k}})\gt 0$ such that for all $0\lt \delta \lt \delta _0$ we have

(2.12) \begin{align} \limsup _{n\to \infty }\frac 1n\mathbb{E}[\text{nul}\mathbb{A}_{\left \lbrack{\lfloor \delta n\rfloor }\right \rbrack }]&\leq 1-\frac dk-\delta . \end{align}

The proof of Proposition 2.3 relies on the Aizenman-Sims-Starr scheme, a coupling argument inspired by spin glass theory [Reference Aizenman, Sims and Starr5]. The technique was also used in ref. [Reference Coja-Oghlan, Ergür, Gao, Hetterich and Rolvien10] to prove the rank formula (1.4). While we mostly follow that proof strategy and can even reuse some of the intermediate deliberations, a subtle modification is required to accommodate the additional ternary equations. The details can be found in Section 4.

How does Proposition 2.3 facilitate the proof of (2.8)? Assuming (1.3), we obtain from (1.4) that $\text{nul}\mathbb{A}/n\sim 1-d/k$ w.h.p. Hence, (2.12) shows that nearly each one of the additional ternary rows added to $\mathbb{A}_{\left \lbrack{\lfloor \delta n\rfloor }\right \rbrack }$ reduces the nullity. We are going to argue that this is possible only if $\mathbb{A}\in \mathfrak{O}$ w.h.p.

To see this, let us think about the kernel of a general $M\times N$ matrix $A$ over $\mathbb{F}_q$ for a short moment. Draw ${\boldsymbol{x}}_A=({\boldsymbol{x}}_{A,i})_{i\in [N]}\in \ker A$ uniformly at random. For any given coordinate ${\boldsymbol{x}}_{A,i}$ , $i\in [N]$ there are two possible scenarios: either ${\boldsymbol{x}}_{A,i}=0$ deterministically, or ${\boldsymbol{x}}_{A,i}$ is uniformly distributed over $\mathbb{F}_q$ . (This is because if we multiply ${\boldsymbol{x}}_A$ by a scalar $t\in \mathbb{F}_q$ we obtain $t{\boldsymbol{x}}_A\in \ker A$ .) We therefore call coordinate $i$ frozen if $x_{i}=0$ for all $x \in \ker A$ and unfrozen otherwise. Let $\mathfrak{F}(A)$ be the set of frozen coordinates.

If $\mathbb{A}$ had many frozen coordinates then adding an extra random row with three non-zero entries could hardly decrease the nullity w.h.p. For if all three non-zero coordinates fall into the frozen set, then we get the new equation ‘for free’, that is, $\text{nul} \mathbb{A}_{[1]}=\text{nul} \mathbb{A}$ . Thus, Proposition 2.3 implies that $|\mathfrak{F}(\mathbb{A})|=o(n)$ w.h.p. We conclude that ${\boldsymbol{x}}_{\mathbb{A},i}$ is uniformly distributed over $\mathbb{F}_q$ for all but $o(n)$ coordinates $i\in [n]$ . However, this does not yet imply that ${\boldsymbol{x}}_{\mathbb{A},i}$ , ${\boldsymbol{x}}_{\mathbb{A},j}$ are independent for most $i,j$ , as required by $\mathfrak{O}$ . Yet a more careful argument based on the ‘pinning lemma’ from [Reference Coja-Oghlan, Ergür, Gao, Hetterich and Rolvien10] does. The proof of the following statement can be found in Section 5.

2.5 Expansion around the equitable solution

In this part, we consider the more general model $\underline{\mathbb{A}}$ , that is, the model in Proposition 2.1. As outlined earlier, given that we know (2.8), we can establish (2.9) by expanding (2.5) around the uniform distribution (2.6). At first glance, this may not seem entirely immediate because (2.8) only appears to fix the variables $(z_i(\sigma ))_{i,\sigma }$ of (2.5) that correspond to the variable nodes. But thanks to a certain inherent symmetry property the optimal $\hat z_{\chi _1,\ldots,\chi _\ell }$ to go with the check nodes end up being nearly equitable as well. This observation by itself now suffices to show without further ado that

(2.13) \begin{align} \mathbb{E}_{\mathfrak{A}}[{\boldsymbol{Z}}^2\cdot {\mathbb{1}}\{\underline{\mathbb{A}}\in \mathfrak{O}\}]&=O\left ({\mathbb{E}_{\mathfrak{A}}[{\boldsymbol{Z}}\cdot {\mathbb{1}}\{\underline{\mathbb{A}}\in \mathfrak{O}\}]^2}\right ). \end{align}

Yet the estimate (2.13) is not quite precise enough to complete the proof of Proposition 2.1. Indeed, to apply Chebyshev’s inequality we would need asymptotic equality as in (2.9) rather than just an $O(\!\cdot\!)$ -bound; Huang [Reference Huang27] faced the same issue in the case ${\boldsymbol{d}}={\boldsymbol{k}}$ constant and $q$ prime. The proof of this seemingly innocuous improvement actually constitutes one of the main technical obstacles that we need to surmount.

As a first step, using a careful local expansion we will show that the dominant contribution to the second moment actually comes from $(z_\ell )_\ell$ such that

(2.14) \begin{align} \sum _{\ell \geq 0} \sum _{i=1}^n \frac{{\mathbb{1}}{\{d_i=\ell \}}}{n} \sum _{\sigma \in \mathbb{F}_q}|z_\ell (\sigma )-q^{-1}|&=O(n^{-1/2}). \end{align}

But even once we know (2.14) a critical issue remains because we allow general distributions of degrees $d_1, \ldots,$ $d_n$ , $k_1,\ldots,k_m$ and matrix entries $\boldsymbol{\chi }$ . In effect, to estimate the kernel size accurately we need to investigate the conceivable frequencies of field values that can lead to solutions. Specifically, for an integer $k_0\geq 3$ and $\chi _1,\ldots,\chi _{k_0}\in \mathbb{F}_q^*$ let

(2.15) \begin{equation} \mathcal{S}_q(\chi _1,\ldots,\chi _{k_0})=\left \{{\sigma \in \mathbb{F}_q^{k_0}\;:\;\sum _{i=1}^{k_0}\chi _i\sigma _i=0}\right \} \end{equation}

comprise all solutions to a linear equation with coefficients $\chi _1,\ldots,\chi _{k_0}\in \mathbb{F}_q$ . For each $\sigma \in \mathcal{S}_q(\chi _1,\ldots,\chi _{k_0})$ the vector

(2.16) \begin{align} \hat{\sigma }=\left ({\sum _{i=1}^{k_0}{\mathbb{1}}\left \{{\sigma _i=s}\right \}}\right )_{s\in \mathbb{F}_q^*}\in \mathbb{Z}^{\mathbb{F}_q^*} \end{align}

tracks the frequencies with which the various non-zero field elements appear. Depending on the coefficients $\chi _1,\ldots,\chi _{k_0}$ , the frequency vectors $\hat{\sigma }$ may be confined to a proper sub-grid of the integer lattice. For example, in the case $q=k_0=3$ and $\chi _1=\chi _2=\chi _3=1$ they span the sub-lattice spanned by $\binom 11$ and $\binom 03$ . The following proposition characterises the lattice spanned by the frequency vectors for general $k_0$ and $\chi _1,\ldots,\chi _{k_0}$ .

A vital feature of Proposition 2.5 is that the module basis consists of non-negative integer vectors with small $\ell _1$ -norm. In effect, the basis vectors are ‘combinatorially meaningful’ towards our purpose of counting solutions. Perhaps surprisingly, the proof of Proposition 2.5 turns out to be rather delicate, with details depending on whether $q$ is a prime or a prime power, among other things. The details can be found in Section 6.

In addition to the sub-grid constraints imposed by the linear equations themselves, we need to take a divisibility condition into account. Indeed, for any assignment $\sigma \in \mathbb{F}_q^n$ of values to variables, the frequencies of the various field elements $s\in \mathbb{F}_q$ are divisible by the g.c.d. $\mathfrak{d}$ of $d_1,\ldots,d_n$ , that is,

(2.17) \begin{align} \mathfrak{d}\mid \sum _{i=1}^n d_i{\mathbb{1}}\left \{{\sigma _i=s}\right \}&&\mbox{for all }s\in \mathbb{F}_q. \end{align}

To compute the expected kernel size we need to study the intersection of the sub-grid (2.17) with the grid spanned by the frequency vectors $\hat{\sigma }$ for $\sigma \in \mathcal{S}_q(\boldsymbol{\chi }_{1,1},\ldots,\boldsymbol{\chi }_{1,k})$ . Specifically, by way of estimating the number of assignments represented by each grid point and calculating the ensuing satisfiability probability, we obtain the following.

We prove Proposition 2.6 in Section 7. Combining Propositions 2.3–2.6, we now establish the main theorems.

2.6 Discussion and related work

The present proof strategy draws on the prior work [Reference Ayre, Coja-Oghlan, Gao and Müller6, Reference Coja-Oghlan, Ergür, Gao, Hetterich and Rolvien10] on the rank of random matrices. Specifically, toward the proof of Proposition 2.3 we extend the Aizenman-Sims-Starr technique from [Reference Coja-Oghlan, Ergür, Gao, Hetterich and Rolvien10] and to prove Proposition 2.4 we generalise an argument from [Reference Ayre, Coja-Oghlan, Gao and Müller6]. Additionally, the expansion around the centre carried out in the proof of Proposition 2.6 employs some of the techniques developed in the study of satisfiability thresholds, particularly the extensive use of local limit theorems and auxiliary probability spaces [Reference Coja-Oghlan12, Reference Coja-Oghlan13].

The principal new proof ingredient is the asymptotically precise analysis of the second moment by means of the study of the sub-grids of the integer lattice induced by the constraints as sketched in Section 2.5. This issue was absent in the prior literature on variations on random $k$ -XORSAT [Reference Ayre, Coja-Oghlan, Gao and Müller6, Reference Coja-Oghlan, Ergür, Gao, Hetterich and Rolvien10, Reference Cooper, Frieze and Pegden15] and on other random constraint satisfaction problems [Reference Coja-Oghlan12, Reference Coja-Oghlan13]. However, in the study of the random regular matrix from Example 1.5 Huang [Reference Huang27] faced a similar issue in the special case ${\boldsymbol{d}}={\boldsymbol{k}}$ constant and $\boldsymbol{\chi }=1$ deterministically. Proposition 2.5, whose proof is based on a combinatorial investigation of lattices in the general case, constitutes a generalisation of the case Huang studied. A further feature of Proposition 2.5 absent in ref. [Reference Huang27] is the explicit $\ell _1$ -bound on the basis vectors. This bound facilitates the proof of Proposition 2.6, which ultimately carries out the expansion around the equitable solution.

Satisfiability thresholds of random constraint satisfaction problems have been studied extensively in the statistical physics literature via a non-rigorous technique called the ‘cavity method’. The cavity method comes in two installments: the simpler ‘replica symmetric ansatz’ associated with the Belief Propagation message passing scheme, and the more intricate ‘replica symmetry breaking ansatz’. The proof of Theorem 1.1 demonstrates that the former renders the correct prediction as to the satisfiability threshold of random linear equations. By contrast, in quite a few problems, notoriously random $k$ -SAT, replica symmetry breaking occurs [Reference Coja-Oghlan, Müller and Ravelomanana14, Reference Ding and Sly20].

An intriguing question for future work might be to understand the ‘critical’ case of $\Phi$ that attain their global max at $0$ and another point left open by Theorem 1.1. While Example 1.4 shows that it cannot generally be true that $\mathbb{A}$ has full row rank w.h.p., the regular case where ${\boldsymbol{d}}={\boldsymbol{k}}=d$ are fixed to the same constant provides an intriguing example. For this scenario Huang proved that the random $\{0,1\}$ -matrix $\mathbb{B}$ has full rank w.h.p. [Reference Huang27]. The proof, based effectively on a moment computation over finite fields and local limit techniques, also applies to the adjacency matrices of random $d$ -regular graphs.

While the present paper deals with sparse random matrices with a bounded average number of non-zero entries in each row and column, the case of dense random matrices has received a great deal of attention, too. Komlós [Reference Komlós34] first shows that dense square random $\{0,1\}$ -matrices are regular over the rationals w.h.p.; Vu [Reference Vu50] suggested an alternative proof. The computation of the exponential order of the singularity probability subsequently led to a series of intriguing articles [Reference Kahn, Komloś and Szemerédi30, Reference Tao and Vu48, Reference Tikhomirov49]. By contrast, the singularity probability of a dense square matrix over a finite field converges to a value strictly between zero and one [Reference Kovalenko35, Reference Kovalenko, Levitskaya and Savchuk36, Reference Levitskaya38, Reference Levitskaya39].

Apart from the sparse and dense case, the regime of intermediate densities has been studied as well. Balakin [Reference Balakin7] and Blömer, Karp and Welzl [Reference Blömer, Karp and Welzl8] dealt with the rank of such random matrices of intermediate densities over finite fields. In addition, Costello and Vu [Reference Costello16, Reference Costello and Vu17] studied the rational rank of random symmetric matrices of an intermediate density.

Indeed, an interesting open problem appears to be the extension of the present methods to the symmetric case. In particular, it would be interesting to see if the present techniques can be used to add to the line of works on the adjacency matrices of random graphs, which have been approached by means of techniques based on local weak convergence or Littlewood-Offord techniques [Reference Bordenave, Lelarge and Salez9, Reference Ferber, Kwan, Sah and Sawhney22]. Several core ideas of [Reference Coja-Oghlan, Ergür, Gao, Hetterich and Rolvien10] have recently been used to study the asymptotic rank of a special class of symmetric random matrices [Reference van der Hofstad, Müller and Zhu26].

2.7 Organisation

After some preliminaries in Section 3 we begin with the proof of Proposition 2.3 in Section 4. The proof relies on an Aizenman-Sims-Starr coupling argument, some details of which are deferred to Section 8. Section 5 deals with the proof of Proposition 2.4. Subsequently we prove Proposition 2.5 in Section 6, thereby laying the ground for the proof of Proposition 2.6 in Section 7.

4.1 Overview

The first ingredient of the proof of Proposition 2.3 is a coupling argument inspired by the Aizenman-Sims-Starr scheme from mathematical physics [Reference Aizenman, Sims and Starr5], which also constituted the main ingredient of the proof of the approximate rank formula (1.4) from [Reference Coja-Oghlan, Ergür, Gao, Hetterich and Rolvien10]. Indeed, the coupling argument here is quite similar to that from [Reference Coja-Oghlan, Ergür, Gao, Hetterich and Rolvien10], with some extra bells and whistles to accommodate the additional ternary equations. We therefore defer that part of the proof to Section 8. The Aizenman-Sims-Starr argument leaves us with a variational formula for a lower bound on the rank of $\mathbb{A}_{[\lfloor \delta n\rfloor ]}$ . The second proof ingredient is to solve this variational problem. Harnessing the assumption (1.3), we will obtain the explicit expression for the rank provided by Proposition 2.3.

Let us come to the details. As explained in Section 3, we will have an easier time working with the pairing model versions ${\boldsymbol{G}},{\boldsymbol{A}}$ of the Tanner graph and the random matrix. Moreover, to facilitate the coupling argument we will need to poke a few holes, known as ‘cavities’ in physics jargon, into the random matrix. More precisely, we will slightly reduce the number of check nodes and tolerate a small number of variable nodes $x_i$ of degree less than ${\boldsymbol{d}}_i$ . The cavities will provide the flexibility needed to set up the coupling argument. Finally, to be able to assume that the matrices we are dealing with are $(\delta,\ell )$ -free with probability close to one, we also add a random, but bounded number of unary checks $p_1, \ldots, p_{\boldsymbol{\theta }}$ , as described in Proposition 3.4. While this measure does not affect the asymptotic rank, quite crucially, it enables our bound on the rank difference in the coupling argument of Section 8.

Formally, let $\varepsilon, \delta \in (0,1)$ and let $\Theta \geq 0$ be an integer. Ultimately $\Theta$ will depend on $\varepsilon$ but not on $n$ or $\delta$ . We then construct the random matrix ${\boldsymbol{A}}\left \lbrack{n,\varepsilon,\delta,\Theta }\right \rbrack$ as follows. Let

(4.1) \begin{align} {\boldsymbol{m}}_\varepsilon &\sim \text{Po}((1-\varepsilon )dn/k),& {\boldsymbol{m}}_\delta &\sim \text{Po}(\delta n),&\boldsymbol{\theta } &\sim \text{unif}([\Theta ]). \end{align}

The Tanner multi-graph $\textbf{G}\left \lbrack{n,\varepsilon, \delta, \Theta }\right \rbrack$ has variable nodes $x_1, \ldots, x_n$ and check nodes $a_1, \ldots, a_{{\boldsymbol{m}}_{\varepsilon }}, t_1, \ldots, t_{{\boldsymbol{m}}_\delta }, p_1, \ldots, p_{\boldsymbol{\theta }}$ . To connect them draw a random maximum matching $\boldsymbol{\Gamma }\left \lbrack{n,\varepsilon }\right \rbrack$ of the complete bipartite graph with vertex classes

\begin{align*} V_1= \bigcup _{i=1}^{{\boldsymbol{m}}_\varepsilon } \left \{{a_i}\right \} \times [{\boldsymbol{k}}_i] \qquad \text{and} \qquad V_2= \bigcup _{j=1}^{n}\left \{{x_j}\right \} \times [{\boldsymbol{d}}_j]. \end{align*}

For every matching edge $\{(a_i,h),(x_j,\ell )\}\in \boldsymbol{\Gamma }[n,\varepsilon ]$ , $h\in [{\boldsymbol{k}}_i],\ell \in [{\boldsymbol{d}}_j]$ , between a clone of $x_j$ and a clone of $a_i$ we insert an $a_i$ - $x_j$ -edge into $\textbf{G}\left \lbrack{n,\varepsilon, \delta, \Theta }\right \rbrack$ . Moreover, the check nodes $t_1, \ldots, t_{{\boldsymbol{m}}_\delta }$ each independently and uniformly choose three neighbouring variables $\textbf{i}_{i,1}, \textbf{i}_{i,2}, \textbf{i}_{i,3}$ with replacement among $\left \{{x_1, \ldots, x_n}\right \}$ . Further, check node $p_\ell$ for $\ell \in [\boldsymbol{\theta }]$ is adjacent to $x_\ell$ only. Finally, to obtain the random $(\boldsymbol{\theta } +{\boldsymbol{m}}_\varepsilon +{\boldsymbol{m}}_\delta ) \times n$ -matrix ${\boldsymbol{A}}\left \lbrack{n,\varepsilon,\delta,\Theta }\right \rbrack$ from $\textbf{G}\left \lbrack{n,\varepsilon, \delta, \Theta }\right \rbrack$ we let

(4.2) \begin{align} {\boldsymbol{A}}\left \lbrack{n,\varepsilon,\delta,\Theta }\right \rbrack _{p_i,x_h}&={\mathbb{1}}\left \{{i=h}\right \}&&(i\in [\boldsymbol{\theta }],h\in [n]), \end{align}

(4.3) \begin{align} {\boldsymbol{A}}\left \lbrack{n,\varepsilon,\delta,\Theta }\right \rbrack _{a_{i},x_h}&=\boldsymbol{\chi }_{i, h} \sum _{\ell =1}^{{\boldsymbol{k}}_i}\sum _{s=1}^{{\boldsymbol{d}}_h} {\mathbb{1}} \{(x_h, s), (a_{i}, \ell )\}\in \boldsymbol{\Gamma }\left \lbrack{n,\varepsilon }\right \rbrack \}&&(i\in [{\boldsymbol{m}}_\varepsilon ], h\in [n]), \end{align}

(4.4) \begin{align} {\boldsymbol{A}}\left \lbrack{n,\varepsilon,\delta,\Theta }\right \rbrack _{t_i,x_h}&=\boldsymbol{\chi }_{{\boldsymbol{m}}_{\varepsilon }+i, h} \sum _{\ell =1}^3{\mathbb{1}}\{\textbf{i}_{i,\ell } = h\} &&(i\in [{\boldsymbol{m}}_\delta ],h\in [n]). \end{align}

Applying the Aizenman-Sims-Starr scheme to the matrix ${\boldsymbol{A}}[n,\varepsilon,\delta,\Theta ]$ , we obtain the following variational bound.

Proposition 4.1. There exist $\delta _0\gt 0$ , $\Theta _0(\varepsilon )\gt 0$ such that for all $0\lt \delta \lt \delta _0$ and any $\Theta =\Theta (\varepsilon )\geq \Theta _0(\varepsilon )$ we have

(4.5) \begin{align} &\limsup _{\varepsilon \to 0}\limsup _{n \to \infty } \frac{1}{n} \mathbb{E}\left \lbrack{\text{nul}({\boldsymbol{A}}\left \lbrack{n,\varepsilon,\delta,\Theta }\right \rbrack )}\right \rbrack \leq \max _{\alpha, \beta \in [0,1]} \Phi (\alpha ) \nonumber \\[5pt] &\quad + \left ({\exp\!\left ({-3\delta \beta ^2}\right )-1}\right )D(1-K'(\alpha )/k) -\delta + 3\delta \beta ^2 -2\delta \beta ^3. \end{align}

The proof of Proposition 4.1, carried out in Section 8 in detail, resembles that of the rank formula (1.4), except that we have to accommodate the additional ternary checks $t_i$ . Their presence is the reason why the optimisation problem on the r.h.s. comes in terms of two variables $\alpha,\beta$ rather than a single variable as (1.4).

To complete the proof of Proposition 2.3 we need to solve the optimisation problem (4.5). This is the single place where we require that $\Phi (z)$ takes its unique global max at $z=0$ , which ultimately implies that the optimiser of (4.5) is $\alpha =\beta =0$ . This fact in turn implies the following.

Proposition 4.2. For any ${\boldsymbol{d}},{\boldsymbol{k}}$ that satisfy (1.3) there exists $\delta _0\gt 0$ such that for all $0\lt \delta \lt \delta _0$ we have

\begin{align*} \max _{\alpha, \beta \in [0,1]} \Phi (\alpha ) + \left ({\exp\!\left ({-3\delta \beta ^2}\right )-1}\right )D(1-K'(\alpha )/k) -\delta + 3\delta \beta ^2 -2\delta \beta ^3=1-\frac{d}{k}-\delta . \end{align*}

The proof of Proposition 4.2 can be found in Section 4.2. Finally, in Section 4.3 we will see that Proposition 2.3 is an easy consequence of Propositions 4.1 and 4.2.

4.2 Proof of Proposition 4.2

Let

\begin{align*} \tilde{\Phi }_\delta (\alpha, \beta ) &= \Phi (\alpha ) + \left ({\exp\!\left ({-3\delta \beta ^2}\right )-1}\right )D(1-K'(\alpha )/k) -\delta + 3\delta \beta ^2 -2\delta \beta ^3&&(\alpha, \beta \in [0,1]). \end{align*}

Assuming (1.3), we are going to prove that for small enough $\delta$ ,

(4.6) \begin{align} \max _{\alpha, \beta \in [0,1]} \tilde{\Phi }_\delta (\alpha, \beta ) =\tilde{\Phi }_\delta (0,0) = 1-\frac{d}{k}-\delta, \end{align}

whence the assertion is immediate.

The $C^1$ -function $\tilde \Phi _\delta$ attains its maximum either at a boundary point of the compact domain $[0,1]^2$ or at a point where the partial derivatives vanish. Beginning with the former, we consider four cases.

1. Case 1: $\alpha =0$ We have

   (4.7) \begin{align} \tilde{\Phi }_\delta (0, \beta ) = \tilde{\Phi }_\delta (0,0) + 3\delta \beta ^2 - 2\delta \beta ^3 -(1-\exp\!\left ({-3\delta \beta ^2}\right )). \end{align}
   Expanding the exponential function, we see that $3\delta \beta ^2 - 2\delta \beta ^3 -(1-\exp\!\left ({-3\delta \beta ^2}\right )) = - 2\delta \beta ^3 + O_\delta (\delta ^2\beta ^4)$ . Since $- 2\delta \beta ^3 + O_\delta (\delta ^2\beta ^4)$ is non-positive for all $\beta \in [0,1]$ , (4.7) yields $\max _\beta \tilde{\Phi }_\delta (0,\beta )=\tilde \Phi _\delta (0,0)$ for all small enough $\delta \gt 0$ .

2. Case 2: $\beta =0$ The assumption (1.3) ensures that $\Phi$ is maximised in $0$ . Therefore, as $\tilde{\Phi }_\delta (\alpha,0) = \Phi (\alpha )-\delta$ , the maximum on $\left \{{(\alpha,0) \;:\; \alpha \in [0,1]}\right \}$ is attained in $\alpha =0$ .

3. Case 3: $\alpha =1$ We obtain

   \begin{align*}\quad {\tilde{\Phi }_\delta (1,\beta )} &=\Phi (1)+ \left ({\exp\!\left ({-3\delta \beta ^2}\right )-1}\right )D(0) - \delta + 3 \delta \beta ^2 - 2 \delta \beta ^3 =\exp\!\left ({-3\delta \beta ^2}\right )D(0)\\[5pt] &\quad + \delta (3\beta ^2 - 2 \beta ^3-1). \end{align*}
   Again, expanding the exponential, we see that for sufficiently small $\delta$ , $\tilde{\Phi }_\delta (1,\beta ) \leq \tilde{\Phi }_\delta (1,0) = \Phi (1) - \delta$ . Thanks to assumption (1.3), this yields $\max _\beta \tilde{\Phi }_\delta (1,\beta )=\tilde \Phi _\delta (0,0)$ for all small enough $\delta \gt 0$ .

4. Case 4: $\beta =1$ We have

   (4.8) \begin{equation} \tilde{\Phi }_\delta (\alpha, 1) = \Phi (\alpha ) - (1-\exp\!\left ({-3\delta }\right ))D\left ({1-\frac{K'(\alpha )}{k}}\right ). \end{equation}
   Because $D$ and $K'$ are continuous on $[0,1]$ due to the assumption $\mathbb{E}[{\boldsymbol{d}}^2]+\mathbb{E}[{\boldsymbol{k}}^2]\lt \infty$ , for any $\zeta \gt 0$ there exists $\hat{\alpha }\gt 0$ such that $D(1-K'(\alpha )/k) \gt 1-\zeta$ for all $0\lt \alpha \lt \hat{\alpha }$ . Therefore, (4.8) shows that for small enough $\delta \gt 0$ and $0\lt \alpha \lt \hat{\alpha }$ we have $\tilde{\Phi }_\delta (\alpha, 1) \lt \tilde{\Phi }_\delta (\alpha,0)\leq \tilde{\Phi }_\delta (0,0)$ . On the other hand, for $\hat{\alpha }\leq \alpha \leq 1$ the difference $\Phi (\alpha )-\Phi (0)$ is uniformly negative because of our assumption (1.3) that $\Phi$ attains its unique global maximum at $\alpha =0$ . Hence, for $\delta$ small enough and $\hat{\alpha }\le \alpha \leq 1$ we obtain $\tilde{\Phi }_\delta (\alpha, 1) \lt \tilde{\Phi }_\delta (0,0)$ .

Combining Cases 1–4, we obtain

(4.9) \begin{align} \max _{(\alpha,\beta )\in \partial [0,1]^2}\tilde{\Phi }_\delta (\alpha,\beta ) = \tilde{\Phi }_\delta (0,0). \end{align}

Moving on to the interior of $[0,1]^2$ , we calculate the derivatives

\begin{align*} \frac{\partial \tilde \Phi _\delta }{\partial \alpha } &= \Phi '(\alpha ) + \left ({1-\exp\!\left ({-3\delta \beta ^2}\right )}\right ) \frac{K''(\alpha )}{k}D'(1-K'(\alpha )/k) \\[5pt] &= \frac{K''(\alpha )}{k} \left ({d(1-\alpha )-\exp\!\left ({-3\delta \beta ^2}\right )D'(1-K'(\alpha )/k)}\right ),\\[5pt] \frac{\partial \tilde \Phi _\delta }{\partial \beta } &= 6 \delta \beta \left ({1-\beta -\exp\!\left ({-3\delta \beta ^2}\right )D(1-K'(\alpha )/k)}\right ). \end{align*}

Hence, potential maximisers $(\alpha,\beta )$ in the interior of $[0,1]^2$ satisfy

(4.10) \begin{equation} d(1-\alpha )=D'(1-K'(\alpha )/k)\exp\!\left ({-3\delta \beta ^2}\right ) \qquad \text{and} \qquad 1-\beta = \exp\!\left ({-3\delta \beta ^2}\right )D(1-K'(\alpha )/k). \end{equation}

Substituting (4.10) into $\tilde \Phi _\delta$ , we obtain

(4.11) \begin{align} \nonumber \tilde{\Phi }_\delta (\alpha, \beta )&= \Phi (\alpha ) - \delta + \left ({\exp\!\left ({-3\delta \beta ^2}\right )-1}\right )D(1-K'(\alpha )/k) + 3\delta \beta ^2 -2\delta \beta ^3\\[5pt] &=\Phi (\alpha ) - \delta + (1-\beta )(1-\exp\!\left ({3\delta \beta ^2}\right ))+ 3\delta \beta ^2 -2\delta \beta ^3 \nonumber \\[5pt] &\leq \Phi (\alpha ) - \delta - 3\delta \beta ^2 (1-\beta )+ 3\delta \beta ^2 -2\delta \beta ^3 = \Phi (\alpha )-\delta +\delta \beta ^3. \end{align}

To estimate the r.h.s. we consider the cases of small and large $\alpha$ separately. Specifically, by continuity for any $\zeta \gt 0$ there is $0\lt \hat{\alpha }\lt \delta$ such that $D(1-K'(\alpha )/k)\gt 1-\zeta$ for all $0\lt \alpha \lt \hat{\alpha }$ .

1. Case 1: $0\lt \alpha \lt \hat{\alpha }$ Since $D(1-K'(\alpha )/k)\gt 1-\zeta$ , (4.10) implies that for $\beta \gt 0$

   \begin{equation*}1-\beta \gt (1-3\delta \beta ^2) (1-\zeta ) = 1 - \zeta - 3\delta \beta ^2(1-\zeta ).\end{equation*}
   In particular, small $\hat \alpha$ implies that also $\beta$ is small. More precisely, after choosing $\delta,\zeta$ small enough, we may assume that $\beta \lt \hat{\beta }$ for any fixed $\hat{\beta }\gt 0$ . In this case, we may thus restrict to solutions $(\alpha, \beta ) \in (0,1)^2$ to (4.10) where both coordinates are sufficiently small. Also here, we distinguish three cases that all lead to contradictions.

   1. (A) If the solution satisfies $\alpha =\beta$ , consider the function

      \begin{equation*}x \mapsto 1-x-\exp\!\left ({-3\delta x^2}\right ) D(1-K'(x)/k)\end{equation*}
      whose zeros determine the solutions to the right equation in (4.10) under the assumption $\alpha =\beta$ . Its value is zero at $x=0$ and it has derivative
      \begin{equation*}-1+6\delta x \exp\!\left ({-3\delta x^2}\right ) D(1-K'(x)/k) + \exp\!\left ({-3\delta x^2}\right ) D'(1-K'(x)/k)\frac {K''(x)}{k},\end{equation*}
      which is negative in a neighbourhood of $x=0$ . Thus $(\alpha, \alpha )$ cannot be a solution to (4.10) for $\alpha \in (0, \hat{\alpha })$ .

   2. (B) Assume now that $\alpha \lt \beta$ . Then the right equation of (4.10) yields

      \begin{align*} 1-\beta \gt \exp\!\left ({-3\delta \beta ^2}\right )D\left ({1-K'(\beta )/k}\right ) \gt \left ({1-3 \delta \beta ^2}\right ) \left ({1- \frac{d}{k}K'(\beta )}\right ). \end{align*}
      Now since ${\boldsymbol{k}} \geq 3$ , $K'(\beta ) = O_\beta (\beta ^2)$ . But then the above equation yields a contradiction for $\beta$ small enough and thus $(\alpha, \beta ) \in (0,\hat \alpha ) \times (0,\hat \beta )$ with $\alpha \lt \beta$ is no possible solution.

   3. (C) Finally, if $\alpha \gt \beta$ , the left equation of (4.10) yields

      \begin{align*} d\left ({1-\alpha }\right ) \gt \exp\!\left ({-3\delta \alpha ^2}\right )D'\left ({1-K'(\alpha )/k}\right ) \gt d\left ({1-3 \delta \alpha ^2}\right ) \left ({1- \frac{\mathbb{E}\left \lbrack{{\boldsymbol{d}}^2}\right \rbrack }{dk}K'(\alpha )}\right ). \end{align*}
      Now since ${\boldsymbol{k}} \geq 3$ , $K'(\alpha ) = O_\alpha (\alpha ^2)$ . But then the above equation yields a contradiction for $\alpha$ small enough and thus $(\alpha, \beta ) \in (0,\hat \alpha ) \times (0,\hat \beta )$ with $\alpha \gt \beta$ is no possible solution.

   Hence, (4.10) has no solution with $0\lt \alpha \lt \hat{\alpha }$ .

2. Case 2: $\hat{\alpha }\leq \alpha \lt 1$ because $\Phi (\alpha )\lt \Phi (0)$ for all $0\lt \alpha \leq 1$ , (4.11) shows that we can choose $\delta$ small enough so that $\tilde \Phi _\delta (\alpha,\beta )\lt \tilde{\Phi }_\delta (0,0)$ for all $\alpha \geq \hat{\alpha }$ and all $\beta \in [0,1]$ .

Combining both cases and recalling (4.9), we obtain (4.6).

4.3 Proof of Proposition 2.3

Combining Propositions 4.1 and 4.2, we see that

(4.12) \begin{align} \limsup _{n \to \infty }\frac{1}{n} \mathbb{E}\left \lbrack{\text{nul}({\boldsymbol{A}}\left \lbrack{n,\varepsilon,\delta,\Theta }\right \rbrack )}\right \rbrack & \leq 1-\frac{d}{k}-\delta +o_{\varepsilon }(1). \end{align}

The only (small) missing piece is that we still need to extend this result to the original random matrix $\mathbb{A}_{[\lfloor \delta n\rfloor ]}$ based on the simple random factor graph $\mathbb{G}$ . To this end we apply the following lemma.

Lemma 4.3 ([Reference Coja-Oghlan, Ergür, Gao, Hetterich and Rolvien10, Lemma 4.8]). For any fixed $\Theta \gt 0$ there exists a coupling of ${\boldsymbol{A}}$ and ${\boldsymbol{A}}\left \lbrack{n,\varepsilon,0,\Theta }\right \rbrack$ such that

\begin{equation*}\mathbb {E}|\textrm {nul}\boldsymbol{A}-\textrm {nul}\boldsymbol{A}\left \lbrack {n,\varepsilon,0,\Theta }\right \rbrack |=O_{\varepsilon }(\varepsilon n). \end{equation*}

Let ${\boldsymbol{A}}_{[\lfloor \delta n\rfloor ]}$ be the matrix obtained from ${\boldsymbol{A}}$ by adding $\lfloor \delta n\rfloor$ random ternary equations. Combining (4.12) with Corollary 4.3, we obtain

(4.13) \begin{align} \frac{1}{n} \mathbb{E}\left \lbrack{\text{nul}({\boldsymbol{A}}_{[\lfloor \delta n\rfloor ]})}\right \rbrack & \leq 1-\frac{d}{k}-\delta +o(1). \end{align}

Furthermore, since changing a single edge of the Tanner graph ${\boldsymbol{G}}$ or a single entry of ${\boldsymbol{A}}$ can change the rank by at most one, the Azuma–Hoeffding inequality shows that $\text{nul}({\boldsymbol{A}}_{[\lfloor \delta n\rfloor ]})$ is tightly concentrated. Thus, (4.13) implies

(4.14) \begin{align} {\mathbb{P}}\left \lbrack{\frac{1}{n}\text{nul}({\boldsymbol{A}}_{[\lfloor \delta n\rfloor ]}) \leq 1-\frac{d}{k}-\delta +o(1)}\right \rbrack &=1-o(1/n). \end{align}

Finally, combining (4.14) with Proposition 3.7, we conclude that

\begin{equation*}{\mathbb {P}}\left \lbrack {\frac {1}{n}\textrm {nul}(\mathbb {A}_{[\lfloor \delta n\rfloor ]}) \leq 1-\frac {d}{k}-\delta +o(1)}\right \rbrack =1-o(1/n),\end{equation*}

which implies the assertion because $\text{nul}(\mathbb{A}_{[\lfloor \delta n\rfloor ]})\leq n$ deterministically.

6.1 Proof of Lemma 6.1

Because we can just factor out any scalar, it suffices to consider the module

\begin{equation*}\mathfrak {M}=\mathfrak {M}_q\underbrace {(1,\ldots,1)}_{\mbox {$k_0$ times}}.\end{equation*}

Being a submodule of the free $\mathbb{Z}$ -module $\mathbb{Z}^{\mathbb{F}_q^\ast }$ , $\mathfrak{M}$ is free, but it is not entirely self-evident that a basis with the additional properties stated in Lemma 6.1 exists. Indeed, while it is easy enough to come up with $q-1$ linearly independent vectors in $\mathfrak{M}$ that all have $\ell _1$ -norm bounded by $3$ , it is more difficult to show that these vectors generate $\mathfrak{M}$ . In the proof of Lemma 6.1, we sidestep this difficulty by working with two sets of vectors $\mathcal{B}_1$ and $\mathcal{B}_2$ . The first set $\mathcal{B}_1$ is easily seen to generate $\mathfrak{M}$ , while $\mathcal{B}_2$ is a set of linearly independent vectors in $\mathfrak{M}$ with $\ell _1$ -norms bounded by $3$ . To argue that $\mathcal{B}_2$ generates $\mathfrak{M}$ , too, it then suffices to show that the determinant of the change of basis matrix equals one.

To interpret the bases as subsets of $\mathbb{Z}^{q-1}$ rather than $\mathbb{Z}^{\mathbb{F}_q^\ast }$ in the following, we fix some notation for the elements of $\mathbb{F}_q$ . Throughout this section, we let $q=p^\ell$ for a prime $p$ and $\ell \in \mathbb{N}$ . If $\ell =1$ , we regard $\mathbb{F}_q$ as the set $\{0, \ldots, p-1\}$ with $\mod p$ arithmetic. If $\ell \geq 2$ , the field elements can be written as

\begin{align*} \{a_0 + a_1{\mathbb{X}} + \ldots + a_{\ell -1}{\mathbb{X}}^{\ell -1} \;:\; a_j \in \mathbb{F}_p \text{ for } j = 0, \ldots, \ell -1\}, \end{align*}

with $\mod g({\mathbb{X}})$ arithmetic for a prime polynomial $g({\mathbb{X}})\in \mathbb{F}_p[{\mathbb{X}}]$ of degree $\ell$ . Exploiting this representation of the field elements as polynomials, we define the length len $(a_0 + a_1{\mathbb{X}} + \ldots + a_{\ell -1}{\mathbb{X}}^{\ell -1})$ of an element of $\mathbb{F}_q$ to be the number of its non-zero coefficients. Finally, let

(6.1) \begin{align} \mathbb{F}_q^{(\geq 2)} = \left \{{h \in \mathbb{F}_q \;:\; \text{len}(h) \geq 2}\right \} \end{align}

be the set of all elements of $\mathbb{F}_q$ with length at least two. Of course, if $\ell = 1$ , $\mathbb{F}_q^{(\geq 2)}$ is empty.

Recall that we view $\mathfrak{M}$ as a subset of $\mathbb{Z}^{\mathbb{F}_q^\ast }$ that is generated by the vectors

\begin{align*} \left ({\sum _{i=1}^{k_0}{\mathbb{1}}\left \{{\sigma _i=s}\right \}}\right )_{s\in \mathbb{F}_q^*}, \quad \sigma \in{\mathcal{S}_q(1, \ldots, 1)}. \end{align*}

In the above representation, the generators are indexed by $\mathbb{F}_q^*$ rather than by the set $[q-1]$ . But to carry out the determinant calculation, it is immensely useful to represent both $\mathcal{B}_1$ and $\mathcal{B}_2$ as matrices with a convenient structure. Hence, there is ambiguity in the choice of a bijection $f\;:\;\mathbb{F}_q^\ast \to \{1, \ldots, q-1\}$ that maps the non-zero elements of $\mathbb{F}_q$ to coordinates in $\mathbb{Z}^{\mathbb{F}_q^*}$ . To put a clear structure to the matrices in this subsection, we will soon choose $f$ in a particular way. With the above notation, we will from now on fix a bijection $f$ that is monotonically decreasing with respect to the length function on $\mathbb{F}_q^\ast$ : If len $(h_1) \lt$ len $(h_2)$ for $h_1, h_2 \in \mathbb{F}_q^\ast$ , then $f(h_1) \gt f(h_2)$ . More precisely, $f$ maps the $(p-1)^\ell$ elements in $\mathbb{F}_q^\ast$ of maximal length $\ell$ to the interval $[(p-1)^\ell ]$ , the $\ell (p-1)^{\ell -1}$ elements of length $\ell -1$ to the interval $\{(p-1)^\ell +1, \ldots, (p-1)^\ell +\ell (p-1)^{\ell -1}\}$ , and so on. For elements of length one, we further specify that

\begin{align*} f(a{\mathbb{X}}^{i})= q-1-(\ell -i)(p-1) + a \quad \text{for } i \in \{0, \ldots \ell -1\} \text{ and } a \in [p-1]. \end{align*}

For our purposes, there is no need to fully specify the values of $f$ within sets of constant length greater than one, but one could follow the lexicographic order, for example. The benefit of such an ordering will become apparent in the next two subsections.

6.1.1 First basis $\mathcal{B}_1$

The idea behind the first set $\mathcal{B}_1$ is that it consists of vectors whose coordinates can be easily seen to correspond to element statistics of a valid solution while ignoring the $\ell _1$ -restriction formulated in Lemma 6.1. We build $\mathcal{B}_1$ from frequency vectors of solutions of the form

\begin{align*} \left ({a_0 + a_1{\mathbb{X}} + \ldots + a_{\ell -1}{\mathbb{X}}^{\ell -1}}\right ) + \sum _{i=0}^{\ell -1} a_i \cdot ((p-1){\mathbb{X}}^{i}) = 0. \end{align*}

That is, we take any element $a_0 + a_1{\mathbb{X}} + \ldots + a_{\ell -1}{\mathbb{X}}^{\ell -1}$ from $\mathbb{F}_q^\ast$ and cancel it by a linear combination of elements from $\{p-1, (p-1){\mathbb{X}}, \ldots, (p-1){\mathbb{X}}^{\ell -1}\} \subseteq \mathbb{F}_q^\ast$ . Formally, let $e_1, \ldots, e_{q-1}$ denote the canonical basis of $\mathbb{Z}^{q-1}$ . The set of statistics of all frequency vectors of the form described above then reads

\begin{align*} \mathcal{B}_1 = \left \{{e_{f(\sum _{i=0}^{\ell -1} a_i{\mathbb{X}}^{i})} + \sum _{i=0}^{\ell -1} a_i e_{f(-{\mathbb{X}}^{i})} \;:\; \sum _{i=0}^{\ell -1} a_i{\mathbb{X}}^{i} \in \mathbb{F}_q^\ast }\right \}. \end{align*}

A moment of thought shows that $|\mathcal{B}_1| = q-1$ . Indeed, it is helpful to notice that for any $h \in \mathbb{F}_q^\ast \setminus \{-1, \ldots, -{\mathbb{X}}^{\ell -1}\}$ , there is exactly one element with a non-zero position in coordinate $f(h)$ , and this coordinate is $1$ . That is, there is basically exactly one element in $\mathcal{B}_1$ associated with each element of $\mathbb{F}_q^\ast$ . Generally, the elements of $\mathcal{B}_1$ can be ordered to yield a lower triangular matrix $M_q$ . To sketch this matrix, we first consider the case $\ell =1$ . In this case, with our choice of indexing function $f$ , the elements of ${\boldsymbol{B}}_1$ can be ordered to give the matrix displayed in Fig. 4. For the case of fields of prime order, this basis is already implicitly mentioned in [Reference Huang27].

Figure 4. The matrix $M_p$ .

Note that this reduces to $M_2 = (2)$ in the case $p=2$ . In this representation, rows are indexed by the field elements they represent, while columns are indexed by the field elements they are associated with. For $\ell \geq 2$ , we can use the matrix $M_p$ for the compact representation of $M_q$ displayed in Fig. 5.

Figure 5. The matrix $M_q$ for $\ell \geq 2$ .

In the matrix $M_q$ , the upper left block is an identity matrix of the appropriate dimension, the upper right is a zero matrix, the lower left is a matrix that only has non-zero entries in rows that correspond to $-1, \ldots, -{\mathbb{X}}^{\ell -1}$ while the lower right is a block diagonal matrix whose blocks are given by $M_p$ . In particular, $M_p$ is a lower triangular matrix. Because $M_p$ has determinant $p$ the following is immediate.

Claim 6.3. We have $ \det (M_q) = p^\ell = q.$

Let $\mathfrak{B}_1$ denote the $\mathbb{Z}$ -module generated by the elements of $\mathcal{B}_1$ . Then the lower triangular structure of $M_q$ also implies the following.

Claim 6.4. The rank of $\mathcal{B}_1$ is $q-1$ .

The following lemma shows that the module $\mathfrak{M}$ is contained in $\mathfrak{B}_1$ .

Lemma 6.5. The $\mathbb{Z}$ -module $\mathfrak{M}$ is contained in the $\mathbb{Z}$ -module $\mathfrak{B}_1$ .

Proof. We show that each element of $\mathfrak{M}$ can be written as a linear combination of elements of $\mathcal{B}_1$ . To this end it is sufficient to show that every frequency vector of a solution to an equation with exactly $k_0$ non-zero entries and all-one coefficients can be written as a linear combination of the elements of $\mathcal{B}_1$ . Let thus $x \in \mathbb{N}^{q-1}$ be such a frequency vector, that is, $\sum _{i=1}^{q-1} x_i f^{-1}(i) = 0$ in $\mathbb{F}_q$ . Before we state a linear combination of $x$ in terms of $\mathcal{B}_1$ , observe that for each $j \in [q-1] \setminus \{q-1-(\ell -1)(p-1), q-1-(\ell -2)(p-1), \ldots, q-1\}$ , there is exactly one basis vector with a non-zero entry in position $j$ . Moreover, the entry of this basis vector in position $j$ is $1$ . On the other hand, the basis vectors corresponding to the remaining $\ell$ columns $q-1-(\ell -1)(p-1), q-1-(\ell -2)(p-1), \ldots, q-1$ of $M_q$ are actually integer multiples of the standard unit vectors, as

\begin{align*} e_{f((p-1){\mathbb{X}}^{i})} + (p-1) e_{f(-{\mathbb{X}}^{i})} = p e_{f((p-1){\mathbb{X}}^{i})} \end{align*}

for $i=0, \ldots, \ell -1$ . With these observations, the only valid candidate for a linear combination of $x$ in terms of the elements of $\mathcal{B}_1$ is given by

\begin{align*} x &= \sum _{\sum _{i=0}^{\ell -1} a_i{\mathbb{X}}^{i} \in \mathbb{F}_q^\ast \setminus \{-1,\ldots, -{\mathbb{X}}^{\ell -1}\}} x_{f(\sum _{i=0}^{\ell -1} a_i{\mathbb{X}}^{i})}\left ({e_{f(\sum _{i=0}^{\ell -1} a_i{\mathbb{X}}^{i})} + \sum _{j=0}^{\ell -1} a_j e_{f(-{\mathbb{X}}^{j})} }\right )\\[5pt] & \qquad \qquad + \sum _{j=0}^{\ell -1}\frac{x_{f(-{\mathbb{X}}^{j})} - \sum _{\sum _{i=0}^{\ell -1} a_i{\mathbb{X}}^{i} \in \mathbb{F}_q^\ast \setminus \{-1, \ldots, -{\mathbb{X}}^{\ell -1}\}} a_j x_{f(\sum _{i=0}^{\ell -1} a_i{\mathbb{X}}^{i})}}{p} \cdot pe_{f(-{\mathbb{X}}^{j})}. \end{align*}

It remains to argue why the coefficients of the basis vectors $pe_{f(-1)}, \ldots, pe_{f(-{\mathbb{X}}^{\ell -1})}$ in the second sum are integers. At this point, we will use that $x$ is a solution statistic: Because

\begin{align*} \sum _{\sum _{i=0}^{\ell -1} a_i{\mathbb{X}}^{i} \in \mathbb{F}_q^\ast } x_{f(\sum _{i=0}^{\ell -1} a_i{\mathbb{X}}^{i} )} \sum _{j=0}^{\ell -1} a_j{\mathbb{X}}^{j} = 0 \qquad \text{in} \quad \mathbb{F}_q \end{align*}

and the additive group $(\mathbb{F}_q,+)$ is isomorphic to $((\mathbb{F}_p)^\ell, +)$ , all ‘components’ in the above sum must be zero and thus

\begin{align*} \sum _{\sum _{i=0}^{\ell -1} a_i{\mathbb{X}}^{i} \in \mathbb{F}_q^\ast } x_{f(\sum _{i=0}^{\ell -1} a_i{\mathbb{X}}^{i} )} a_j= 0 \qquad \text{in} \quad \mathbb{F}_p \end{align*}

for all $j=0, \ldots, \ell -1$ . However, isolating the contribution from $\{-1, \ldots, -{\mathbb{X}}^{\ell -1}\}$ yields

(6.3) \begin{align} 0 = \sum _{\sum _{i=0}^{\ell -1} a_i{\mathbb{X}}^{i} \in \mathbb{F}_q^\ast } x_{f(\sum _{i=0}^{\ell -1} a_i{\mathbb{X}}^{i} )} a_j = - x_{f(-{\mathbb{X}}^{j})} + \sum _{\sum _{i=0}^{\ell -1} a_i{\mathbb{X}}^{i} \in \mathbb{F}_q^\ast \setminus \{-1, \ldots, -{\mathbb{X}}^{\ell -1}\}} a_j x_{f(\sum _{i=0}^{\ell -1} a_i{\mathbb{X}}^{i} )} \qquad \text{in} \quad \mathbb{F}_p, \end{align}

as the coefficient $a_j$ of ${\mathbb{X}}^{j}$ in $-{\mathbb{X}}^{i}$ is zero unless $i=j$ . Therefore, the right-hand side in (6.3) is divisible by $p$ and the claim follows.

6.1.2 Second basis $\mathcal{B}_2$

In this subsection, we define a candidate set for the vectors $(\mathfrak{b}_1, \ldots, \mathfrak{b}_{q-1})$ in the statement of Lemma 6.1. That is, we define a set $\mathcal{B}_2$ all whose elements have non-negative components and $\ell _1$ -norm at most three. In other words, we are looking for solutions to

(6.4) \begin{align} x_1 + \ldots + x_{k_0} = 0 \end{align}

with at most three different non-zero components.

Here again, our construction basically associates one basis vector to each element of $\mathbb{F}_q^\ast$ . However, due to the $\ell _1$ -restriction, there is less freedom in choosing the remaining non-zero coordinates. Our approach to design a set that satisfies this restriction while retaining a representation in a convenient block lower triangular matrix structure is to distinguish between elements of length one and of length at least two. We will therefore construct $\mathcal{B}_2$ via two sets $\mathcal{B}^{(1)}$ and $\mathcal{B}^{(\geq 2)}$ such that $\mathcal{B}_2$ is given as

(6.5) \begin{equation} \mathcal{B}_2 = \mathcal{B}^{(1)} \cup \mathcal{B}^{(\geq 2)}. \end{equation}

Let us start with an element $h = \sum _{i=0}^{\ell -1}a_i{\mathbb{X}}^{i}$ of length at least two in $\mathbb{F}_q$ . Assume that its leading coefficient is $a_r$ for $r \in [\ell -1]$ . If a variable in (6.4) takes value $h$ , we may cancel its contribution to the equation by subtracting the two elements $a_r{\mathbb{X}}^r$ and $h - a_r{\mathbb{X}}^r$ , both of which are shorter than $h$ :

\begin{align*} \sum _{i=0}^{\ell -1} a_i{\mathbb{X}}^{i} - a_r{\mathbb{X}}^r - \left ({\sum _{i=0}^{\ell -1}a_i{\mathbb{X}}^{i} - a_r{\mathbb{X}}^r}\right )=0. \end{align*}

This solution corresponds to the vector

\begin{align*} e_{f(h)} + e_{f(-a_r{\mathbb{X}}^r)} + e_{f(-h+a_r{\mathbb{X}}^r)}. \end{align*}

This idea for field elements $h \in \mathbb{F}_q^{(\geq 2)}$ of length at least two then yields the $q-1-\ell (p-1)$ integer vectors

\begin{align*} \mathcal{B}^{(\geq 2)} = \left \{{e_{f(h)} + e_{f(-a_r{\mathbb{X}}^r)} + e_{f(-h+a_r{\mathbb{X}}^r)} \;:\; r \in [\ell -1] \text{ and } h = \sum _{i=0}^{r}a_i{\mathbb{X}}^{i} \in \mathbb{F}_q^{(\geq 2)} \text{ with } a_r\not =0}\right \}. \end{align*}

For a field element $h$ of length one, an analogous shortening operation would correspond to the vector

\begin{align*} e_{f(h)} + e_{f(-h)}. \end{align*}

If $p=2$ , this procedure applied to all field elements of length one yields $\ell$ distinct vectors and we are done. However, if $p \gt 2$ , employing this idea for all elements of length one would only lead to $\ell (p-1)/2$ rather than $\ell (p-1)$ additional vectors, as $h$ and $-h$ are distinct and obviously give rise to the same statistic. As a consequence, for $p\gt 2$ , we need to deviate from the above construction and come up with a modified ‘short-solution’ scheme. Let $h = a_r{\mathbb{X}}^r$ be an element of length one. If $a_r \in \{1, \ldots (p-1)/2\}$ , we simply associate the vector $e_{f(h)} + e_{f(-h)}$ to it, as indicated. If on the other hand $a_r \in \{(p+1)/2, \ldots, p-1\}$ , we let $h$ correspond to the vector

\begin{align*} e_{f(h)} + e_{f(-{\mathbb{X}}^r)} + e_{f(-h+{\mathbb{X}}^r)}. \end{align*}

With this, for $p\gt 2$ , the part of $\mathcal{B}_2$ that corresponds to field elements of length one is given by the set

(6.6) \begin{align} \mathcal{B}^{(1)}&= \bigcup _{r=0}^{\ell -1}\left (\left \{{e_{f(a_r{\mathbb{X}}^r)} + e_{f(-a_r{\mathbb{X}}^r)} \;:\; a_r \in [(p-1)/2]}\right \} \right.\nonumber \\[5pt] &\left.\quad \cup \left \{{e_{f(-a_r{\mathbb{X}}^r)} + e_{f(-{\mathbb{X}}^r)} + e_{f(a_r{\mathbb{X}}^r+{\mathbb{X}}^r)} \;:\; a_r \in [(p-1)/2]}\right \}\right ). \end{align}

If $p=2$ , in line with the above discussion, we simply let

(6.7) \begin{align} \mathcal{B}^{(1)}= \bigcup _{r=0}^{\ell -1}\left \{{2e_{f({\mathbb{X}}^r)} }\right \}. \end{align}

Again, a moment of thought shows that in any case, $|\mathcal{B}_2| = |\mathcal{B}_1| = q-1$ . Let $\mathfrak{B}_2$ denote the $\mathbb{Z}$ -module generated by the elements of $\mathcal{B}_2$ . Our choice of $\mathcal{B}_2$ has the advantage that again, its elements may be represented in a block lower triangular matrix. For this representation, it is instructive to consider the case $\ell =1$ first. In this case and with our choice of $f$ , the elements of $\mathcal{B}_2$ can be arranged as the columns of a matrix $A_p$ as in Fig. 6.

Figure 6. The matrix $A_p$ .

Here, as in the construction of $M_p$ , column $i$ corresponds to the unique vector associated to $i \in \mathbb{F}_q$ . In the special case $p=2$ , this matrix reduces to

\begin{align*} A_2 = (2). \end{align*}

For $\ell \geq 2$ , the elements of $\mathcal{B}_2$ may then be visualised in the matrix from Fig. 7.

Figure 7. The matrix $A_q$ for $\ell \geq 2$ .

In $A_q$ , column $i \in [q-1]$ corresponds to the unique vector that is associated with the field element $f^{-1}(i)$ . Moreover, at this point, a moment of appreciation of our indexing choice $f$ is in place: Because $f$ is monotonically decreasing with respect to length, there are no entries above the diagonal in the first $|\mathbb{F}_q^{(\geq 2)}|$ columns, as we only cancel field elements by strictly shorter ones. Moreover, the remaining $\ell (p-1)$ columns are governed by a simple block structure. As a concrete example, (6.8) with $p=7$ reads

\begin{equation*} A_7 = \begin {pmatrix} 1 & 0 & 0 & 0 & 0 & 0\\[5pt] 0 & 1 & 0 & 0 & 0 & 1\\[5pt] 0 & 0 & 1 & 0 & 1 & 0\\[5pt] 0 & 0 & 1 & 2 & 0 & 0\\[5pt] 0 & 1 & 0 & 0 & 1 & 0\\[5pt] 1 & 0 & 0 & 1 & 1 & 2 \end {pmatrix} \end{equation*}

and $A_7$ would be used as a block matrix in any field of order $7^\ell$ as shown in (6.9).

As each element of $\mathcal{B}_2$ corresponds to a solution with at most $3 \leq k_0$ non-zero components, we obtain the following.

Claim 6.6. The $\mathbb{Z}$ -module $\mathfrak{B}_2$ is contained in the $\mathbb{Z}$ -module $\mathfrak{M}$ .

Thus far we know $ \mathfrak{B}_2 \subseteq \mathfrak{M} \subseteq \mathfrak{B}_1.$ Moreover, $\mathcal{B}_2$ has the desired $\ell _1$ -property. On the other hand, in comparison to $\mathcal{B}_1$ , it is less clear that $\mathcal{B}_2$ generates $\mathfrak{M}.$ It thus remains to show that in fact $\mathfrak{B}_2 = \mathfrak{B}_1$ . We will do so by using the following fact, which is an immediate consequence of the adjugate matrix representation of the inverse matrix.

Fact 6.7. If $M$ is a free $\mathbb{Z}$ -module with basis $x_1, \ldots, x_n$ , a set of elements $y_1, \ldots, y_n\in M$ is a basis of $M$ if and only if the change of basis matrix $(c_{ij})$ has determinant $\pm 1$ .

We will apply Fact 6.7 to $M=\mathfrak{B}_1$ with $\{x_1, \ldots, x_n\} = \mathcal{B}_1$ and $\{y_1, \ldots, y_n\} = \mathcal{B}_2$ . Let $C_q \in \mathbb{Z}^{(q-1) \times (q-1)}$ be the matrix whose entries comprise the coefficients when we express the elements of $\mathcal{B}_2$ by $\mathcal{B}_1$ (recall that $\mathfrak{B}_2 \subseteq \mathfrak{B}_1$ ) when we order the elements of $\mathcal{B}_1, \mathcal{B}_2$ as done in the construction of $M_q$ and $A_q$ . Thus $A_q = M_q C_q.$ As

\begin{align*} \det (A_q) = \det (M_q C_q) = \det (M_q) \cdot \det (C_q), \end{align*}

we do not need to compute $C_q$ explicitly to apply Fact 6.7, but instead it suffices to compute $\det (M_q)$ and $\det (A_q)$ . From Claim 6.3, $\det (M_q)$ is already known. Moreover, for $A_q$ , the computation will not be too hard, as $A_q$ is a block lower triangular matrix. Therefore, we are just left to calculate the determinant of the non-trivial diagonal blocks.

Lemma 6.8. For any prime $p$ we have $\det (A_p) = p.$

Proof. The case $p=2$ is immediate. We thus assume that $p\gt 2$ in the following. We transform $A_p$ into a lower triangular matrix by elementary column operations. To this end, let $a_1, \ldots, a_{q-1}$ be the columns of $A_p$ . The first $(p+1)/2$ columns already have the right form, so we do not alter this part of the matrix. For any $j=(p+3)/2, \ldots, p-1$ , subtract column $a_{p+1-j}$ from column $a_j$ . This yields the matrix

Next, we swap column $(p+1)/2$ successively with columns $(p+3)/2, \ldots$ up to $p-1$ , yielding

This changes the determinant by a factor of $(-1)^{(p-3)/2}$ . Finally, in order to erase the entry $2$ in row $(p+1)/2$ and column $p-1$ , we add twice the sum of columns $(p+1)/2, \ldots, p-2$ to column $p-1$ . We thus obtain the matrix

with determinant $(-1)^{(p-3)/2}p$ . Multiplying with $(-1)^{(p-3)/2}$ from the column swaps yields the claim.

Corollary 6.9. For any prime $p$ and $\ell \geq 1$ , we have $ \det (A_q) = q.$

Finally, Claim 6.3 and Corollary 6.9 imply that $\det (C_q) = 1.$ Thus, by Fact 6.7, $\mathcal{B}_2$ is a basis of $\mathfrak{B}_1$ , which implies that $ \mathfrak{B}_1 = \mathfrak{B}_2 = \mathfrak{M}.$ The column vectors $\mathfrak{b}_1,\ldots,\mathfrak{b}_{q-1}$ of $A_q$ therefore enjoy the properties stated in Lemma 6.1.

6.2 Proof of Lemma 6.2

Assume w.l.o.g. that $\chi _1=1$ . Moreover, by assumption, the set $\{\chi _1, \ldots, \chi _{k_0}\}$ contains at least two different elements, and so we may also assume that $\chi _3 \not = 1$ (recall that $k_0 \geq 3$ ).

We define $(\mathfrak{b}_1, \ldots, \mathfrak{b}_{q-1})$ by distinguishing between three cases:

Case 1: $p=2$ and $\chi _2=1$ .

Denote the order of $\chi _3^{-1}$ in $(\mathbb{F}_q^\ast, \cdot )$ by $\mathfrak{o}$ , so that the elements $1, \chi _3^{-1}, \ldots, \chi _3^{-(\mathfrak{o}-1)}$ are pairwise distinct. Since $p=2$ and $\mathfrak{o} \mid q-1$ , $\mathfrak{o}$ is an odd number. Moreover, because $\chi _3^{-1} \not =1$ , $\mathfrak{o} \geq 3$ . We now partition $\mathbb{F}_q^\ast$ into orbits of the action of $(\{1, \chi _3^{-1}, \ldots, \chi _3^{-(\mathfrak{o}-1)} \}, \cdot )$ on $\mathbb{F}_q^\ast$ such that

\begin{equation*}\mathbb {F}_q^\ast =\dot \bigcup _{j=1}^{(q-1)/\mathfrak {o}} \mathfrak {O}_j,\end{equation*}

where each orbit $\mathfrak{O}_j$ contains exactly $\mathfrak{o}$ elements. Suppose that $\mathcal{O}_j=\{g^{(j)}_1, \ldots, g^{(j)}_{\mathfrak{o}}\}$ , where the elements are indexed such that $g_{i+1}^{(j)} = \chi _3^{-1}g_{i}^{(j)}$ .

To each $\mathfrak{O}_j$ , we associate a set of potential basis vectors whose union over different $j$ then yields the full set $(\mathfrak{b}_1, \ldots, \mathfrak{b}_{q-1})$ . More precisely, the set corresponding to $\mathfrak{O}_j$ is defined as

\begin{align*} \mathcal{B}_j = \bigcup _{i=1}^{\mathfrak{o}-1} \left \{{e_{g^{(j)}_i}+ e_{g^{(j)}_{i+1}}}\right \} \cup \left \{{e_{g_{1}^{(j)}} + e_{g_{2}^{(j)}} + e_{g_{2}^{(j)}+g_3^{(j)}}}\right \} . \end{align*}

In this definition, we have used that for $\chi _1 = -\chi _2=1$ and any $h \in \mathbb{F}_q$ ,

\begin{align*} \chi _1 \cdot h + \chi _2 \cdot 0 + \chi _3 \cdot \chi _3^{-1}h = 0 \qquad \text{as well as} \qquad \chi _1 \cdot h + \chi _2 \cdot \chi _3^{-1}h + \chi _3 \cdot (\chi _3^{-1}h+\chi _3^{-2}h) = 0. \end{align*}

Note that the element

\begin{align*} g_2^{(j)}+g_3^{(j)} = (1+ \chi _3^{-1})g_2^{(j)} \end{align*}

is non-zero and distinct from both $g_2^{(j)}$ and $g_3^{(j)}$ . It might be one of $g_{1}^{(j)},g_4^{(j)}, \ldots, g_{\mathfrak{o}}^{(j)}$ .

We next argue that the union of the different $\mathcal{B}_j$ generates $\mathbb{Z}^{\mathbb{F}_q^\ast }$ . By linear transformation and using that $\mathfrak{o}$ is odd, $\mathcal{B}_j$ has the same span as

\begin{align*} \left \{{e_{g^{(j)}_1} + e_{g_2^{(j)}}, e_{g^{(j)}_1} - e_{g_3^{(j)}}, e_{g^{(j)}_1} + e_{g_4^{(j)}}, \ldots, e_{g^{(j)}_1} - e_{g_{\mathfrak{o}}^{(j)}}}\right \} \cup \left \{{e_{g_{1}^{(j)}+g_2^{(j)}}}\right \}. \end{align*}

Now, there are two cases.

1. 1. For all $j \in [(q-1)/\mathfrak{o}]$ , $g_2^{(j)}+g_3^{(j)} \in \{g_{1}^{(j)}, g_4^{(j)}, \ldots, g_{\mathfrak{o}}^{(j)}\}$ . In this case, either $e_{g_2^{(j)}+g_3^{(j)}} =e_{g_{1}^{(j)}}$ , or we can subtract $e_{g_2^{(j)}+g_3^{(j)}}$ from or add it to the element $e_{g^{(j)}_1} \pm e_{g_2^{(j)}+g_3^{(j)}}$ to obtain $e_{g_{1}^{(j)}}$ . After isolating $e_{g_1^{(j)}}$ , a straightforward linear transformation yields a set of $\mathfrak{o}$ distinct unit vectors whose non-zero components are given by $\mathfrak{O}_j$ . Thus, the union over all $\mathcal{B}_j$ constitutes a set of linearly independent elements that generates $\mathbb{Z}^{\mathbb{F}_q^*}$ .

2. 2. For all $j \in [(q-1)/\mathfrak{o}]$ , $g_2^{(j)}+g_3^{(j)} \notin \{g_1^{(j)}, g_4^{(j)}, \ldots, g_{\mathfrak{o}}^{(j)}\}$ . In this case, consider the union $\bigcup _{j=1}^{(q-1)/\mathfrak{o}}\mathcal{B}_j$ , which has the same span as

   \begin{align*} \bigcup _{j=1}^{(q-1)/\mathfrak{o}} \left \{{e_{g^{(j)}_1} + e_{g_2^{(j)}}, e_{g^{(j)}_1} - e_{g_3^{(j)}}, e_{g^{(j)}_1} + e_{g_4^{(j)}}, \ldots, e_{g^{(j)}_1} - e_{g_{\mathfrak{o}}^{(j)}}}\right \} \cup \left \{{e_{g_{1}^{(j)}+g_2^{(j)}}}\right \}. \end{align*}
   Since for each $j$ , the element $g_1^{(j)}+g_2^{(j)}$ must be contained in some $\mathfrak{O}_{j'}$ for $j\not =j'$ , as in case (1), $e_{g_{1}^{(j)}+g_2^{(j)}}$ can be used to isolate $e_{g_1^{(j')}}$ . After isolating $e_{g_1^{(j')}}$ for all $j'$ , these elements can be straightforwardly used to linearly transform the union over all $\mathcal{B}_j$ into the standard basis $(e_{h})_{h \in \mathbb{F}_q^\ast }$ of $\mathbb{Z}^{\mathbb{F}_q^*}$ .