Skip to main content Accessibility help
×
Home

Information:

  • Access
  • Open access

Figures:

Actions:

      • Send article to Kindle

        To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

        Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

        Find out more about the Kindle Personal Document Service.

        Noise bias compensation for tone mapped noisy image using prior knowledge
        Available formats
        ×

        Send article to Dropbox

        To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

        Noise bias compensation for tone mapped noisy image using prior knowledge
        Available formats
        ×

        Send article to Google Drive

        To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

        Noise bias compensation for tone mapped noisy image using prior knowledge
        Available formats
        ×
Export citation

Abstract

A large number of studies have been made on denoising of a digital noisy image. In regression filters, a convolution kernel was determined based on the spatial distance or the photometric distance. In non-local mean (NLM) filters, pixel-wise calculation of the distance was replaced with patch-wise one. Later on, NLM filters have been developed to be adaptive to the local statistics of an image with introduction of the prior knowledge in a Bayesian framework. Unlike those existing approaches, we introduce the prior knowledge, not on the local patch in NLM filters but, on the noise bias (NB) which has not been utilized so far. Although the mean of noise is assumed to be zero before tone mapping (TM), it becomes non-zero value after TM due to the non-linearity of TM. Utilizing this fact, we propose a new denoising method for a tone mapped noisy image. In this method, pixels in the noisy image are classified into several subsets according to the observed pixel value, and the pixel values in each subset are compensated based on the prior knowledge so that NB of the subset becomes close to zero. As a result of experiments, effectiveness of the proposed method is confirmed.

I. INTRODUCTION

Although a large number of studies have been made on denoising, most of them are focused on utilizing correlation between pixels. In regression filters, a convolution kernel was determined based on the spatial distance between pixels [1,2]. Those were extended to bilateral filters introducing the photometric distance [3,4]. Recently, interests in non-local mean (NLM) filters have been growing [510]. This class of filters replaces pixel-wise calculation of the distance with patch-wise one. Reports on the NLM filter have been actively studied, such as improvement of denoising performance [1113], processing speed [1416], combination of NLM filter, and another method [17,18].

Later on, NLM filters have been developed to be adaptive to the local statistics of an image with the introduction of the prior knowledge in a Bayesian framework [1925]. Lebrun et al. proposed the non-local Bayes algorithm in which the patch is modeled as a Gaussian distribution and its parameters are computed from a local neighborhood [19]. This kind of technique was referred to as the hierarchical Bayesian modeling [20] and applied to the image restoration [21,22], the image un-mixing problem [23] and the image de-convolution [24]. Recently, it was extended to be stable against the missimg problem [25]. All of them share a common Bayesian framework based on the prior knowledge on parameters of the Gaussian distribution for each local patch.

Unlike those Bayesian approaches, we utilize the prior knowledge, not on the local patch in NLM but, on the noise bias (NB) which is newly introduced in this paper. Most of the literatures usually assume the noise to be i.i.d. additive white noise. Especially, the zero-mean assumption has been widely imposed on the filter design [9,10]. However, in tone mapping (TM) processing [2628], such as brightness correction, contrast adjustment for dark images, or RAW images, the output noise has a non-zero average (= NB). This NB is due to the non-linearity of TM such as the power function, the logarithmic function, and the Hill function, etc. However, little attention has been given to NB.

This paper tries to recover the ideal output image from the observed output image by compensating NB. The NB is a different notion from the ensemble average of the noise over “all” pixels. In this paper, pixels in the noisy image are classified into several subsets according to the observed pixel value, and compensates the pixel value in each subset with a preliminarily determined compensation value. This procedure is the NB compensation (NBC). NB in this paper is the mean of the noise in the subset corresponding to the observation pixel value and it is compensated. A primitive idea was reported in [29]. Extending the idea, a method of determining the compensation value from all pixel values in an input image based on the Bayesian inference theory was reported without enough experimental results [30]. In addition, it is assumed that all histogram information of an input image is included in the overhead information.

In this paper, we propose a new method based on compensation value calculated from reduced information of the histogram of an input image and the noise before TM. In the proposed method, it is assumed that the histogram of the pixel values in an input image is included in the overhead information which is reduced much more than [30].

This paper is organized as follows. Section II describes the problems dealt with in this paper. Section III describes the proposed method. Section IV shows experimental results using night scene images and confirms effectiveness of the proposed method. Finally, the paper is concluded in Section V.

II. PROBLEM SETTING

Figure 1 illustrates a situation this paper assumes. An image is assumed to have additive noise. The noisy image is tone mapped (brightness corrected). As a result, NB becomes non-zero value. This paper compensates the NB of the noisy image after TM. This paper regards a night scene image and a tone mapped noisy image as an input image and an observed output image, respectively, and assumes noise as Gaussian noise. Section A describes effect of TM on noise, and Section B describes NB after TM.

Fig. 1. A situation this paper assumes.

A) Effect of tone mapping on noise

In this paper, we consider the case where a noisy image is tone mapped. Figure 2 illustrates examples. The input image signal is expressed as

(1)$$x_0({\bf{n}}) = x_0(n_1, n_2), \quad x_0 \in [0, X_{MAX}] \subseteq {\open Z},$$

where $x_0({\bf{n}})$ denotes a pixel at location ${\bf{n}}=[n_1,\, n_2]$ and $X_{MAX}=255$. We shall omit the coordinate (${\bf{n}}$) when we are looking at a particular pixel and the position is not important. A pixel value $x_0$ is tone mapped with a function f as

(2)$$y_0 = R[ f(x_0) ], \quad y_0 \in [0, Y_{MAX}] \subseteq {\open Z},$$

where a pixel value $y_0$ is the ideal tone mapped value and $Y_{MAX}=255$. $R[\ ]$ denotes rounding to the nearest integer and is defined as

(3)$$R[x] = \lfloor x + 2^{-1} \rfloor.$$

Fig. 2. Example images before and after TM; (a) and (d) are images before TM; (b) and (e) are images after TM with $\gamma =3$. (a) Input image, (b) ideal output image, (c) input noise $(\sigma = 8)$, (d) noisy image, (e) observed output image, (f) observed output noise.

As the simplest example, γ correction is used as the TM function in this paper. TM function f is formulated as

(4)$$f(x) = \left\{\matrix{0 \hfill & {\rm for} \hfill & x \lt 0 \hfill \cr Y_{MAX} \cdot (X_{MAX}^{-1} \cdot x)^{1/\gamma} \hfill & {\rm for} \hfill & x \in [0, X_{MAX}] \cr Y_{MAX} \hfill & {\rm for} \hfill & x \gt X_{MAX},\hfill}\right.$$

where γ is a parameter. For a given noise $\varepsilon _1({\bf{n}})$ is expressed as

(5)$$\varepsilon_1({\bf{n}}) = \varepsilon_1(n_1, n_2),$$

a pixel value $x_1$ in the noisy image is expressed as

(6)$$x_1 = x_0 + \varepsilon_1,$$

where $x_1$ is clipped to the range of $[0,\, X_{MAX}]$. In Fig. 2, the probability mass function (PMF) of the noise $\varepsilon _1$ on $x_0$ is given as the Gaussian function

(7)$$P(\varepsilon_1 | x_0) = {1\over \sqrt{2\pi}\sigma} \, {exp} \left( - {\varepsilon_1^2\over 2 \sigma^2} \right),$$

where $\sigma ^2$ denotes the variance and the mean of the noise is zero value. A pixel value $y_1$ which is a tone mapped value of $x_1$ is expressed as

(8)$$\eqalign{y_1 & = f(x_1) \quad \cr & = f(x_0 + \varepsilon_1) \quad \cr & = y_0 + \delta_1,}$$

where $\delta _1$ denotes an observed output noise. Figures 3(a) and 3(b) illustrate the flow of TM for an input image and a noisy image, respectively.

Fig. 3. (a) Flow of TM for an input image. $y_0$ is the ideal tone mapped pixel value. (b) Flow of TM for a noisy image. The mean of output noise $\delta _1$ included in an image after TM hasa non-zero value. (c) Flow of NBC.

B) Noise bias after tone mapping

In an image processing such as TM, an output noise which is included in an image after TM hasa non-zero mean. We investigate the PMF of pixel values before and after TM. Figure 4(a) illustrates the conditional-PMF $P(x_1|x_0)$ at $x_0=10$. It seems that the observed $P(x_1|x_0)$ (= blue dots) and the theoretical $P(x_1|x_0)$ (= green curve) are almost the same. The conditional mean $E[x_1|x_0]$ approximates 10 (${=}x_0$). $P(x_1|x_0)$ is formulated as

(9)$$P(x_1|x_0) =\left\{\matrix{\sum\nolimits_{t=-\infty}^{0} g(t|x_0) \hfill & {\rm for}\quad x_1 = 0 \hfill \cr g(x_1|x_0) \hfill & {\rm for} \quad x_1 \in (0, X_{MAX})\hfill \cr \sum\nolimits_{t=X_{MAX}}^{\infty} g(t|x_0) \hfill & {\rm for} \quad x_1 = X_{MAX}\hfill \cr 0 \hfill & {otherwise},\hfill}\right.$$

where

(10)$$g(x_1|x_0) = {1\over \sqrt{2\pi} \sigma} \, exp \left( - {(x_1-x_0)^2\over 2 \sigma^2} \right).$$

Figures 4(b) and 4(c) illustrate the conditional-PMF $P(y_1|x_0)$ and $P(\delta _1|x_0)$, respectively. In Fig. 4(b), $P(y_1|x_0)$ is asymmetric with respect to the ideal tone mapped value $y_0$, and a bias is generated. In Fig. 4(c), although the mean of the noise before TM isa zero value, the mean of an output noise $\delta _1$ isa non-zero value (= NB). The next section introduces a new method to compensate NB. This is the fact we are focusing on in this paper.

Fig. 4. (a) $P(x_1|x_0=10)$. $x_0$ and $x_1$ are pixel values in an input image and a noisy image, respectively. The mean of an input noise $\varepsilon _1$ is a zero value. (b) $P(y_1|x_0=10)$. $y_1$ is pixel values in an image after TM. (c) $P(\delta _1|x_0=10)$. $\delta _1$ is noise in an image after TM. The mean of an output noise $\delta _1$ isa non-zero value.

III. PROPOSED METHOD

In Section II, it was shown that the mean of output noise after TM has NB. This paper tries to recover the ideal output image from the observed output image by compensating NB. In this section, we propose NBC which is a new method based on compensation value calculated from prior knowledge.

A) NB compensation

In order to recover the ideal output image from the observed image, a calculated value of NB (compensation value) is subtracted from an observed pixel value. NBC is defined as

(11)$$\eqalign{y_2 &= y_1 - h(y_1) \cr &= y_0 + \delta_2,}$$

where $y_2$ is a pixel value after NBC, $h(y_1)$ is a compensation function giving the compensation value for the observed pixel value $y_1$ and $\delta _2$ denotes the error with respect to the ideal tone mapped value. Figure 3(c) illustrates the flow of NBC.

Note that unlike the Bayesian MAP estimation which maximizes the posterior probability density function [3134], our method calculates the compensation value from a subset according to the observed pixel value as indicated in equation (11).

B) Subset according to the observed pixel value

Figure 5(a) illustrates the conditional-PMF $P(x_0|y_1)$ at $y_1=100$. Let N be the set of all pixels in an image and ${M}_{y_1}$ the set of pixels that derive the observed pixel value $y_1$. Note that ${M}_{y_1}$ is a subset of N;

(12)$$\left\{\eqalign{& {N} = \left\{{\bf{n}} \in {image} \right\}, \cr & {M}_{\eta} = \left\{{\bf{m}} | y_1({\bf{m}}) = \eta \right\} \subseteq {N}.} \right.$$

For a pixel value in Fig. 5(a) expressed as $x_0({\bf{m}})$, Fig. 5(b) illustrates the conditional-PMF of an output noise.

Fig. 5. (a) $P(x_0|y_1=100)$. The pixel value $x_0$ is the subset according to the observed pixel value $y_1=100$. (b) $P(\delta _1|y_1=100)$. (c) Relationship between $\delta _1$ and $x_0$. The mapping from $\delta _1$ to $x_0$ is a bijective.

C) Compensation function

In this paper, the compensation value for the observed pixel value $y_1$ is defined as the conditional mean of the observed output noise $\delta _1({\bf{m}})$. Therefore, the compensation function is defined as

(13)$$\eqalign{& h(y_1)=E[\delta_1|y_1] \cr & \quad \quad = {1\over |{M}_{y_1}| } \sum_{{\bf{m}} \in {M}_{y_1}} \delta_1({\bf{m}}).}$$

Equation (13) is equivalently expressed as

(14)$$\eqalign{h(y_1) &= \sum_{ {\bf{m}} \in {M}_{y_1} } P \left( \delta_1({\bf{m}}) \right) \cdot \delta_1({\bf{m}}) \cr & = \sum_{ {\bf{m}} \in {M}_{y_1} } P \left( \delta_1({\bf{m}}) \right) \cdot \left( y_1 - y_0\right) \cr & = \sum_{ {\bf{m}} \in {M}_{y_1} } P \left( \delta_1({\bf{m}}) | y_1 \right) \cdot \left( y_1 - y_0\right)}$$

Note that $P ( \delta _1({\bf{q}}) | y_1 ) = 0$ where ${\bf{q}} \in \overline {{M}_{y_1}} \subseteq {\rm N}$. Therefore, equation (14) is equivalently expressed as

(15)$$h(y_1) = \sum_{{\bf{n}} \in {N}} P \left( \delta_1({\bf{n}}) | y_1 \right) \cdot \left( y_1 - y_0\right).$$

Here, the mapping from $\delta _1$ to $x_0$ is a bijective. Because, using equation (8), the observed output noise $\delta _1({\bf{m}})$ is expressed as

(16)$$\delta_1({\bf{m}}) = y_1 - f( x_0({\bf{m}}) ),$$

equivalently expressed as

(17)$$x_0({\bf{m}}) = f^{-1}( y_1 - \delta_1({\bf{m}}) ).$$

Figure 5(c) illustrates the relationship between $\delta _1({\bf{m}})$ and $x_0({\bf{m}})$. Since the relationship between $\delta _1({\bf{m}})$ and $x_0({\bf{m}})$ is the bijective, the following equations hold.

(18)$$\eqalign{P \left( \delta_1({\bf{m}}) \right) &= P \left( x_0({\bf{m}}) \right)\quad \cr P \left( \delta_1({\bf{n}}) | y_1 \right) &= P \left( x_0({\bf{n}}) | y_1 \right).}$$

Substituting equation (18) into equation (15),

(19)$$\eqalign{h(y_1) &= \sum_{ {\bf{n}} \in {N} } P \left( x_0({\bf{n}}) | {y_1} \right) \cdot \left( y_1 - y_0\right) \cr \quad &= \sum_{ x_0 } P \left( x_0 | y_1 \right) \cdot \left( y_1 - y_0\right).}$$

According to the Bayes' theorem and the addition theorem,

(20)$$P(x_0|y_1) = {P(x_0,y_1) \over P(y_1)},$$

and

(21)$$P(y_1)=\sum_{x_0} P(x_0,y_1). $$

hold, respectively. Substituting equations (20) and (21) into equation (19),

(22)$$\eqalign{& h(y_1) = \sum_{ x_0 } { P(x_0,y_1) \over P(y_1) } \cdot \left\{ y_1 - f(x_0) \right\} \cr & \quad \quad = {\sum_{ x_0 } P(x_0,y_1) \cdot \left\{y_1 - f(x_0) \right\}\over { P(y_1) }} \cr & \quad \quad= { \sum_{ x_0 } P(x_0,y_1) \cdot \left\{y_1 - f(x_0) \right\}\over { \sum_{x_0} P(x_0,y_1)}}.}$$

The joint-PMF $P(x_0,\,y_1)$ is the “prior knowledge” which can be obtained using all pixel values $x_0$ in an image. Figure 6 illustrates the TM function and the prior knowledge.

Fig. 6. (a) TM function ($\gamma =3$). (b) $P(x_0,\,y_1)$. (c) $P(x_0,\,x_1)$. (d) $\hat {P}(x_0,\,y_1)$. $P(x_0,\,y_1)$ and $\hat {P}(x_0,\,y_1)$ is the prior knowledge. Note that the log-scaled joint-PMF is illustrated.

Note that it is assumed that all pixel values in an input image are included in the overhead information. If the histogram of pixel values in an input image is included in the overhead information instead of all pixel values, it becomes possible that the overhead is reduced. In the next section, we introduce the modeling of prior knowledge $P(x_0,\,y_1)$ from the histogram of pixel values in an input image and that of the noise before TM.

D) Modeling of PMF

In this paper, we propose a method of determining the compensation value from the histogram of pixel values in an image and that of the noise before TM based on the Bayesian inference theory. Let modeled prior knowledge be $\hat {P}(x_0,\,y_1)$. In the sequel, we derive a reasonable model $\hat {P}(x_0,\,y_1)$ assuming only the knowledge of $P(x_0)$ (= the histogram of pixel values in an input image before TM) and $g(x_1)$ (= the histogram of the noise before TM) as the overhead information. The compensation function using $\hat {P}(x_0,\,y_1)$ is expressed as

(23)$$\hat{h}(y_1) = { \sum_{ x_0 } \hat{P}(x_0,y_1) \cdot \left\{ y_1 - f(x_0) \right\} \over \sum_{ x_0 } \hat{P}(x_0,y_1) }.$$

The prior knowledge $P(x_0,\,y_1)$ is obtained by mapping the joint-PMF $P(x_0,\,x_1)$ shown in Fig. 6(c) according to the gradient of the TM function. According to the Bayes' theorem,

(24)$$P(x_0,x_1) = P(x_1|x_0) P(x_0),$$

holds. In this modeled case, it is assumed that the prior probability $P(x_0)$ is included in the overhead information. From (9), the posterior probability $P(x_1|x_0)$ is modeled as

(25)$$\hat{P}(x_1|x_0) = \left\{\matrix{{\sum\nolimits_{t=x_0-3\sigma}^{0}} g(t|x_0) \hfill & {\rm for} \quad x_1=0 \hfill \cr {}\hfill &\quad\quad \wedge x_0-3\sigma \leq 0 \hfill \cr g(x_1|x_0) \hfill & {\rm for} \quad x_1 \in (0, X_{MAX})\hfill \cr \sum\nolimits_{t=X_{MAX}}^{x_0+3\sigma} g(t|x_0) \hfill & {\rm for} \quad x_1=X_{MAX}\hfill \cr {}\hfill & \wedge X_{MAX} \leq x_0 + 3\sigma \hfill\cr 0 \hfill & {otherwise},\hfill}\right.$$

where $g(x_1)$ is indicated by (10). In the Gaussian distribution, the 3σ interval is a confidence interval of about 99.7%. Note that σ is given by users. From (24) and (25), the modeled prior knowledge is expressed as

(26)$$\hat{P}(x_0,x_1) = \hat{P}(x_1|x_0) P(x_0).$$

The mapping from $\hat {P}(x_0,\,x_1)$ to $\hat {P}(x_0,\,y_1)$ is calculated as

(27)$$\hat{P}(x_0,y_1\in {W}_i) = {1\over | {W}_i | }\sum_{x_1 \in {V}_i } \hat{P}(x_0,x_1),$$

where

(28)$$\left\{\eqalign{& z(x) = R[f^{-1}(x)],\cr & {U} = \{ z(x) | x \in [0, X_{MAX}] \} \cup \{ X_{MAX} + 1\},\cr & {V}_i = \{ x | {U}(i) \leq x \lt {U}(i+1)\},\cr & {W}_i = \{y | {U}(i) \leq z(y) \lt {U}(i+1) \}.}\right.$$

Note that ${U}(i)$ indicates the i-th smallest element in the set U. For example, when $\gamma =3$, ${U}=\{0,\, 1,\, \ldots ,\, 252,\, 255,\, 256 \}$. In the case of ${U}(i)=0$, ${V}_i=\{0\}$, and ${W}_i=\{0,\, 1,\, \ldots ,\, 31\}$, thus $\hat {P}(x_0,\,y_1=0)=\cdots =\hat {P}(x_0,\,y_1=31)=\hat {P}(x_0, x_1=0)/32$. On the other hand, in the case of ${U}(i)=252$, ${V}_i=\{252,\, 253,\, 254\}$, and ${W}_i=\{254\}$, thus $\hat {P}(x_0,\,y_1=254)=\sum _{x_1\in \{252, 253, 254\}}\hat {P}(x_0,\,x_1)$. Figure 6(d) illustrates the modeled prior knowledge $\hat {P}(x_0,\,y_1)$.

Figure 7 illustrates the compensation values $h(y_1)$ and $\hat {h}(y_1)$ of the measured and modeled cases calculated by (22) and (23), respectively. The size of input image shown in Fig. 2(a) is $471 \times 640$ (pixels), and 8 bit depth (256 tones) grayscale. In the measured case, the data size to be included in the overhead information is about 2.4 million bits. On the other hand, in the modeled case, that is about 16 thousand bits. Note that, it is assumed that histogram information expresses each tone by the Double type (64 bits). The overhead information in the modeled case is greatly less than the measured case. Thus, the modeled case makes it possible to greatly reduce the data size to be included in the overhead information while maintaining the measured case quality.

Fig. 7. Noise bias compensation values.

IV. EXPERIMENTAL RESULTS

NBC in this paper classifies pixels in the noisy image into several subsets according to the observed pixel value, and compensates the pixel value in each subset with a preliminarily determined compensation value. Based on the Bayesian inference theory, the compensation value is determined from the histogram of pixel values in an image and that of the noise before TM. In NBC, for each image, the compensation value corresponding to that image is automatically calculated. This section confirms effectiveness of the proposed method experimentally.

A) Effect of NBC

Figure 8 illustrates the NB before and after TM for the input image shown in Fig. 2(a). For pixel values with small values, the noise bias is greatly reduced. For small pixel values, the noise bias is greatly reduced. This means that the NBC has a large effect on compensation of small pixel values. Table 1 summarizes the average and variance of NB. After NBC, the variance is greatly reduced, and the average is approaching zero value.

Fig. 8. NB before and after NBC for the input image shown in Fig. 2(a).

Table 1. Average and variance of all NB shown in Fig. 8.

B) Quality of compensated images

The image quality before and after NBC is evaluated with the peak signal to noise ratio (PSNR) defined as

(29)$$PSNR = 10 \log_{10} { Y_{MAX}^2 \over{Var}[\delta({\bf{n}})]}.$$

Figure 9 illustrates comparison of PSNR before and after NBC for the input image shown in Fig. 2(a). Figures 9(a) and 9(b) investigate the effect of γ in the TM function in (3) and that of σ in the PMF of noise in (6), respectively. It is observed that the image quality after NBC is improved. In Figs 9(a) and 9(b), the modeled NBC is only 0.0053 (dB) and 0.0197 (dB) lower than the measured on average respectively, and there is no significant difference.

Fig. 9. PSNR of the compensated image. (a) Effect of γ ($\sigma = 8$), (b) effect of σ ($\gamma = 3$).

C) Combination with non-local mean filter

This section investigates combination of NBC and NLM filter. In the NLM filter used in the experiment, the sizes of the search window and the similarity window were set to $3\times 3$ and $2\times 2$, respectively. The CVC-14 dataset [35] that has 4072 night scene images gathered using visible cameras were used as test images. In addition, 16 astronomical images randomly selected from the NASA Image and Video Library [36] were used as well. Figure 10 illustrates experimental results of comparison of the average PSNR of all night scene images in the CVC-14 dataset. The average of NBC is 1.68 (dB) lower than that of NLM. However, in about $20\%$ of all night scene images, NBC is superior to NLM. The average of NBC+NLM is 0.94 (dB) higher than that of NLM. In the best results, NBC+NLM is 3.37 (db) higher than NLM. In about $80\%$ of all night scene images, NBC+NLM is superior to NLM. Figure 11 illustrates experimental results of comparison of the average PSNR of astronomical images. The average of NBC is 7.11 (dB) higher than that of NLM. The average of NBC+NLM is 7.43 (dB) higher than that of NLM. Denoising performance is improved by combination. This means that NBC can coexist with approaches focusing on the correlation between pixels like NLM filter. In addition, NBC is effective as preprocessing such as NLM filter.

Fig. 10. The average PSNR of the compensated and filtered images for the CVC-14 dataset. $\gamma =3$, $\sigma =8$. Note that “observed”, “NLM”, “NBC”, and“NBC+NLM” indicate the observed image, the image after NLM filter, the image after NBC (the modeled case), and the combination of NBC (the modeled case) and NLM filter, respectively.

Fig. 11. The average PSNR of the compensated and filtered images for astronomical images. $\gamma =3$, $\sigma =8$.

Result images after NBC, NLM filter, and NBC+NLM are illustrated in Figs 12 and 13. Compared with NLM, noise was reduced in NBC and NBC+NLM. NBC has a large effect on compensation of small pixel values. Therefore, in the TM of a dark image like night scene images, NBC is a great effect on denoising.

Fig. 12. Results of NBC, NLM, and NBC+NLM for the image shown in Fig. 2(a). PSNR of observed output is 28.13 (dB). (a) NBC (29.99 dB), (b) NLM (30.13 dB), (c) NBC+NLM (32.45 dB).

Fig. 13. Results of NBC, NLM, and NBC+NLM for an astronomical image. (a) Input image, (b) ideal output image, (c) input noise $(\sigma = 8)$, (d) noisy image, (e) observed output image (16.15 dB), (f) NBC (21.52 dB), (g) NLM (16.15 dB), (h) NBC+NLM (21.65 dB).

Figure 14 illustrates details of the bright area. There is no difference in all images. Figure 15 illustrates details of the dark area. NLM is similar to the observed image, and these are noisy. On the other hand, NBC and NBC+NLM are not noisy, and improvement of image quality is confirmed. TM function in (3), when $\gamma >1$, the gradient is steep as the pixel value is smaller as shown in Fig. 6(a). Moreover, when the pixel value is small, the absolute value of NB is large (= large bias), as shown in Fig. 7. Therefore, significant effects can be obtained in the dark area. On the other hand, when the pixel value is large, NB is close to zero value (= no bias). Therefore, no significant effects can be obtained in the bright area.

Fig. 14. Detailed results of the bright place. (a) Reference (ideal output image), (b) observed, (c) NBC, (d) NLM, (e) NBC+NLM.

Fig. 15. Detailed results of the dark place. (f)–(i) Output noise. (a) Reference (ideal output image), (b) observed, (c) NBC, (d) NLM, (e) NBC+NLM, (f)observed, (g) NBC, (h) NLM, (i) NBC+NLM.

V. CONCLUSIONS

In this paper, NBC method for tone mapped noisy image was proposed. The compensation value is calculated using the prior knowledge. The effectiveness of the proposed method over the existing method was experimentally confirmed for several tone mapped noisy LDR images. It was experimentally confirmed that the combination of NBC and approaches focusing on the correlation between pixels like NLM filter improved the denoising performance more than when only either one was used. The advantage of the denoising performance was confirmed by using NBC as preprocessing for NLM filter. NBC is not an approach focusing on the correlation between pixels, and NBC can coexist with approaches focusing on the correlation between pixels like NLM filter. Noise with other probability distributions, such as shot noise with the Poisson distribution, and single domain or global information-based should be investigated in the future. In addition, we should analyze the effectiveness of combinations with various filters other than NLM in future work.

FINANCIAL SUPPORT

This work was supported by JSPS KAKENHI Grant Number 16K13715.

REFERENCES

1Wand, M.P.; Jones, M.C.: Kernel Smoothing. Monographs on Statistics and Applied Probability, Chapman & Hall, London, 1995.
2Gonzalez, R.; Woods, J.: Digital Image Processing, 3rd ed., Prentice-Hall, Englewood Cliffs, NJ, 2008.
3Zhang, M.; Gunturk, B.K.: Multiresolution bilateral filtering for image denoising. IEEE Trans. Image. Process., 17 (12) (2008), 23242333.
4Honghong, P.; Rao, R.; Dianat, S.A.: Multispectral image denoising with optimized vector bilateral filter. IEEE Trans. Image Process., 23 (1) (2014), 264273.
5Buades, A.; Coll, B.; Morel, J.: A non-local algorithm for image denoising. IEEE Computer Vision and Pattern Recognition (CVPR), June (2005), 60–65.
6Awate, S.P.; Whitaker, R.T.: Unsupervised, information-theoretic, adaptive image filtering for image restoration. IEEE Trans. Pattern Anal., Mach. Intell., 28 (3) (2006), 364376.
7Kervrann, C.; Boulanger, J.: Optimal spatial adaptation for patch-based image denoising. IEEE Trans. Image Process., 15 (10) (2006), 28662878.
8Chatterjee, P.; Milanfar, P.: Patch-based near-optimal denoising. IEEE Trans. Image Process., 21 (4) (2011), 16351649.
9Dabov, K.; Foi, A.; Katkovnik, V.; Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process., 16 (8) (2007), 20802095.
10Milanfar, P.: A tour of modern image filtering: new insights and methods, both practical and theoretical. IEEE Signal. Process. Mag., 30 (1) (2013), 106128.
11Liu, G.; Zhong, H.; Jiao, L.: Comparing noisy patches for image denoising: a double noise similarity model. IEEE Trans. Image. Process., 24 (3) (2015), 862872.
12Xu, G.; Jiang, S.: Nonlocal means denoising using a content-based searching region. Int. Congress on Image and Signal Processing (CISP), October 2015, 293297.
13Ghosh, S.; Mandal, A.K.; Chaudhury, K.N.: Pruned non-local means. IET. Image. Process., 11 (5) (2017), 317323.
14Chan, S.H.; Zickler, T.; Lu, Y.M.: Monte Carlo non-local means: random sampling for large-scale image filtering. IEEE Trans. Image. Process., 23 (8) (2014), 37113725.
15Xue, B.; Huang, Y.; Yang, J.; Shi, L.; Zhan, Y.; Cao, X.: Fast nonlocal remote sensing image denoising using cosine integral images. IEEE Geosci. Remote. Sens. Lett., 10 (6) (2013), 13091313.
16Huang, F. et al. : A parallel nonlocal means algorithm for remote sensing image denoising on an intel Xeon Phi platform. IEEE Access., 5 (2017), 85598567.
17Mohan, M.R.M.; Sheeba, V.S.: A Novel Method of Medical Image Denoising Using Bilateral and NLm Filtering. Int. Conf. on Advances in Computing and Communications (ICACC), pp.186191, August 2013.
18Kim, M.; Park, D.; Han, D.K.; Hanseok, Ko: A novel approach for denoising and enhancement of extremely low-light video. IEEE Trans. Consum. Electron., 61 (1) (2015), 7280.
19Lebrun, M.; Buades, A.; Morel, J.: A nonlocal bayesian image denoising algorithm. SIAM. J. Imaging. Sci., 6 (3) (2013), 16651688.
20Gelman, A.; Carlin, J.B.; Stern, H.S.; Rubin, D.B.: Bayesian data analysis, vol. 2, Chapman & Hall/CRC, Boca Raton FL, USA, 2014.
21Molina, R.: On the hierarchical bayesian approach to image restoration applications to astronomical images. IEEE Trans. Pattern. Anal. Mach. Intell., 16 (11) (1994), 11221128.
22Molina, R.; Katsaggelos, A.K.; Mateos, J.: Bayesian and regularization methods for hyperparameter estimation in image restoration. IEEE Trans. Image. Process., 8 (2) (1999), 231246.
23Dobigeon, N.; Tourneret, J.-Y.; Chang, C.-I.: Semi-supervised linear spectral unmixing using a hierarchical bayesian model for hyperspectral imagery. IEEE Trans. Signal. Process., 56 (7) (2008), 26842695.
24Orieux, F.; Giovannelli, J.; Rodet, T.: Bayesian estimation of regularization and point spread function parameters for Wiener-Hunt deconvolution. J. Opt. Soc. Am. A, 27 (7) (2010), 15931607.
25Aguerrebere, C.; Almansa, A.; Delon, J.; Gousseau, Y.; Muse, P.: A Bayesian hyperprior approach for joint image denoising and interpolation with an application to HDR imaging. IEEE Trans. Comput. Imaging., 3 (4) (2017), 633646.
26Reinhard, E.; Stark, M.; Shirley, P.; Ferwerda, J.: Photographic tone reproduction for digital images, in journal ACM transactions on graphics - Proc. SIGGRAPH, 21 (3) (2002), 267276.
27Mai, Z.; Mansour, H.; Mantiuk, R.; Nasiopoulos, P.; Ward, R.; Heidrich, W.: Optimizing a tone curve for backward-compatible high dynamic range image and video compression. IEEE Trans. Image. Process., 20 (6) (2011), 15581571.
28Koz, A.; Dufaux, F.: Optimized Tone Mapping with LDR Image Quality Constraint for Backward-Compatible High Dynamic Range Image and Video Coding. IEEE Int. Conf. Image Processing (ICIP), September 2013, 17621766.
29Iwahashi, M.; Kiya, H.: Noise Bias Compensation of Tone Mapped Noisy Image. IEEE Int. Conf. Image Processing (ICIP), no. TEC-P6.3, October 2014.
30Iwahashi, M.; Hamzah, F.A.B.; Yoshida, T.; Hitoshi, Kiya: Noise bias compensation based on Bayesian inference for tone mapped noisy image. Asia-Pacific Signal and Information Processing Association Annual Summit and Conf. (APSIPA), December 2015.
31Liu, C.; Freeman, W.T.; Szeliski, R.; Kang, S.B.: Noise estimation from a single image. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 901-908, 2006.
32Hou, J.; Xiong, C.: An Efficient Locally Adaptive Wavelet Denoising Method Based on Bayesian MAP Estimation. 2006 Int. Conf. on Communications, Circuits and Systems, June 2006.
33Dan, Z.; Chen, X.; Gan, H.; Gao, C.: Locally Adaptive Shearlet Denoising Based on Bayesian MAP Estimate. 2011 Sixth Int. Conf. on Image and Graphics, August 2011.
34Sadreazami, H.; Ahmad, M.O.; Swamy, M.N.S.: Color image denoising using multivariate cauchy PDF in the contourlet domain. 2016 IEEE Canadian Conf. on Electrical and Computer Engineering (CCECE), May 2016.
35González, A.; Fang, Z.; Socarras, Y.; Serrat, J.; Vázquez, D.; López, A.M.: Pedestrian Detection at Day/Night Time with Visible and FIR Cameras: A Comparison. Sens., 16 (6) (2016), 111.
36NASA Image and Video Library, https://images.nasa.gov/.

Sayaka Minewaki received her B.Eng. and M.Eng. degrees in Engineering from Kyushu Institute of Technology in 2001 and 2003, respectively. In 2006, she finished a Ph.D. program without dissertation at the Department of Artificial Intelligence, Kyushu Institute of Technology. In 2006, she joined Department of Computer Science and Engineering, National Institute of Technology, Yuge College, where she served concurrently as a Lecturer. From 2016 to 2018, she joined Nagaoka University of Technology, where she is currently an Assistant Professor of the Department of Electrical, Electronics and Information Engineering. From 2018, she has been with the Department of Computer Science and Engineering, National Institute of Technology, Yuge College. Her research interests are in the fields of digital signal processing, image compression, and natural language processing.

Taichi Yoshida received B.Eng., M.Eng, and Ph.D. degrees in Engineering from Keio University, Yokohama, Japan, in 2006, 2008, and 2013. In 2014, he joined Nagaoka University of Technology, where he is currently an Assistant Professor in the Department of Electrical, Electronics and Information Engineering, Faculty of Technology. His research interests are in the field of filter bank design, image coding, and image processing.

Yoshinori Takei received the B.Sc. and M.Sc. degrees in mathematics in 1990 and 1992, respectively, from Tokyo Institute of Technology and the Dr. Eng. degree in information processing in 2000, from Tokyo Institute of Technology, Yokohama, Japan. From 1992 to 1995, he was with Kawasaki Steel System R&D Inc., Chiba, Japan. From 1999 to 2000, he was an Assistant Professor with the Department of Electrical and Electronic Engineering, Tokyo Institute of Technology. From 2000 to 2017, he was with the Department of Electrical Engineering, Nagaoka University of Technology, Niigata, Japan, where he was finally an Associate Professor. From 2017, he has been withthe Department of Electrical and Computer Engineering, National Institute of Technology, Akita College, where he is an Associate Professor. His current research interests include computational complexity theory, public-key cryptography, randomized algorithms, and digital signal processing. Especially, he is now interested in the application of non-commutative harmonic analysis to data-mining. Dr. Takei is a member of LA, SIAM, ACM, AMS, and IEICE.

Masahiro Iwahashi received his B.Eng, M.Eng., and D.Eng. degrees in electrical engineering from Tokyo Metropolitan University in 1988, 1990, and 1996, respectively. In 1990, he joined Nippon Steel Co. Ltd.. From 1991 to 1992, he was dispatched to Graphics Communication Technology Co. Ltd.. In 1993, he joined Nagaoka University of Technology, where he is currently a professor of Department of Electrical Engineering, Faculty of Technology. From 1995 to 2001, he served concurrently as a lecturer of Nagaoka Technical College. From 1998 to 1999, he dispatched to Thammasat University in Bangkok, Thailand as a JICA expert. His research interests are in the area of digital signal processing, multi-rate systems, image compression. From 2007 to 2011, he served as an editorial committee member of the transaction on fundamentals. He is serving as a reviewer of IEEE, IEICE, and APSIPA. He is currently a senior member of the IEEE and IEICE.

Hitoshi Kiya received his B.E and M.E. degrees from Nagaoka University of Technology, in 1980 and 1982 respectively, and his Dr. Eng. degree from Tokyo Metropolitan University in 1987. In 1982, he joined Tokyo Metropolitan University, where he became Full Professor in 2000. He is a Fellow of IEEE, IEICE, and ITE. He currently serves as President of APSIPA (2019–2020), and served as Inaugural Vice President (Technical Activities) of APSIPA from 2009 to 2013, Regional Directorat-Large for Region 10 of the IEEE Signal Processing Society from 2016 to 2017. He was Editorial Board Member of eight journals, including IEEE Transactions on Signal Processing, Image Processing, and Information Forensics and Security, Chair of two technical committees and Member of nine technical committees including APSIPA Image, Video, and Multimedia TC, and IEEE Information Forensics and Security TC. Kiya was a recipient of numerous awards, including six best paper awards.