We establish various martingale inequalities in a rearrangement-invariant (RI) Banach function space. If $X$ is an RI space that is not too small, we associate with it RI spaces $\mathcal{H}_p(X)$ $(1\leq p\lt\infty)$ and $K(X)$, and discuss martingale inequalities in these spaces. One of our results is as follows. Let $1\leq p\lt\infty$, let $f=(f_n)$ be an $L_p$-bounded martingale, and let $|f|^p=g+h$ be the Doob decomposition of the submartingale $|f|^p=(|f_n|^p)$ into a martingale $g=(g_n)$ and a predictable non-decreasing process $h=(h_n)$ with $h_0=0$. Then, in the case where $1\ltp\lt\infty$, we obtain the inequalities

$$ \|h_{\infty}^{1/p}\|_X\leq2\|f_{\infty}\|_{\mathcal{H}_p(X)}\quad\text{and}\quad \Big\|\sup_n|g_n|^{1/p}\Big\|_X\leq4\|f_{\infty}\|_{\mathcal{H}_p(X)}, $$

and, in the case where $p=1$, we obtain the inequalities

$$ \|h_{\infty}\|_X\leq\sup_{n\in\mathbb{Z}_{+}}\|f_n\|_{K(X)}\quad\text{and}\quad \sup_{n\in\mathbb{Z}_{+}}\|g_n\|_X\leq2\sup_{n\in\mathbb{Z}_{+}}\|f_n\|_{K(X)}. $$

For some specific choices of $X$, we can give explicit expressions for $\mathcal{H}_p(X)$ and $K(X)$. For example, $\mathcal{H}_1(L_1)=L\log L$, $\mathcal{H}_p(L_{p,\infty})=L_{p,1}$, and so on. Furthermore, if the Boyd indices of $X$ satisfy $0\lt\alpha_{X}\leq\beta_{X}\lt1/p$ (respectively, $0\lt\alpha_{X}$), then $\mathcal{H}_p(X)=X$ (respectively, $K(X)=X$). In any case, $\mathcal{H}_p(X)$ is embedded in $K(X)$, and $K(X)$ is embedded in $X$.

AMS 2000 Mathematics subject classification: Primary 60G42; 60G46. Secondary 46E30