current position:Home>Computational Photography - Image Denoising (2)

Computational Photography - Image Denoising (2)

2022-09-12 03:26:31Turned_MZ

         This chapter will be organized图像去噪相关的内容,主要包括:噪声类型,评估方法,传统去噪方法,Deep Learning Denoising Methods,常用数据集,This chapter mainly discusses the last two parts.The previous content can refer to:计算摄影——图像去噪(一)_Turned_MZ的博客-CSDN博客

Commonly used denoising datasets

        At present, the main methods for establishing denoising datasets are as follows:3种:

  1. Obtain high-quality images from existing image datasets,然后做图像处理(线性变化、亮度调整等)And add artificially synthesized noise according to the noise model,生成噪声图像.However, this method is artificially synthesized due to noise.,Different from the noise in the real scene,Therefore, the effect is not good in the real scene.
  2. for the same image,Shoot low-sensitivity images as true values,High-sensitivity images as noise images,And adjust the exposure parameters to real two image brightness.This class of methods directly uses the low-sensitivity image as the ground truth,Inevitably there will be residual noise,And there may also be brightness differences and misalignment issues with noisy images.
  3. Shooting multiple images of the same scene in succession,Then a ground truth is synthesized by image registration, etc.,This way requires a lot of images to be taken,工作量比较大,But the final true value is of high quality.

        Common datasets and corresponding papers:

数据集GTnumber of scenesPicture logarithm主要领域说明相关论文
RENIOR低ISO120240低光

拍摄了120dark scene,Contains indoor and outdoor scenes.About each scene4张图像,包含2One noisy image and two low-noise images.

RENOIR - A Dataset for Real Low-Light Image Noise Reduction
Nam-CC15均值1117物品包含11个场景,and mostly similar objects and textures.针对这11A total of scenes were shot500张JPEG图像.A Holistic Approach to Cross-Channel Image Noise Modeling and its Application to Image Denoising
Nam-CC60均值1160物品--
DND低ISO5050室内外拍摄50个场景,Includes indoor and outdoor scenes.Benchmarking Denoising Algorithms with Real Photographs
PolyU均值4040室内外拍摄了40个场景,Including normal light and dark scene scene,Outdoor normal lighting scene.Continuous shooting for each scene500次Real-world Noisy Image Denoising: A New Benchmark
SIDD均值200400多领域用5个相机(Google Pixel、iPhone 7、Samsung Galaxy S6 Edge、Motorola Nexus 6、LG G4)Shot under four camera parameters10个场景,200scene instance,Each scene was shot continuously150张图像.其中160scene instances as a training set,40scene instances as a test set(the benchmark).A High-Quality Denoising Dataset for Smartphone Cameras
High-ISO均值28110高ISOhttp://cwc.ucsd.edu/sites/cwc.
NIND低ISO101606多级ISO
IOCI均值-200Various camerashttps://arxiv.org/pdf/2011.0346

Deep Learning Denoising Methods

Estimating Noise Residuals

Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising

        in the field of information processing,Learning the amount of change in the signal is often simpler than learning the original signal,This idea is used in very efficient residual networks.DnCNNThe model draws on this idea,It does not directly output the denoised image,But the prediction residual image,The observation noise image and the difference between potential noiseless image.

        CNN在VGG的基础上进行修改,网络结构是(卷积、BN、ReLU)级联的结构,Inside the model is not likeResNetThere is also a long-hop connection,Instead use residual learning at the output of the network.结构如下:

该网络的特点主要有:

  1. 网络分为三部分,第一部分为Conv+Relu(一层),第二部分为Conv+BN+Relu(若干层),第三部分为Conv(一层),网络层数为17或者20层.
  2. 网络学习的是图像残差,也就是带噪图像和无噪图像差值,损失函数采用的MSE.
  3. 论文中强调了batch normalization的作用

noise estimation model

Toward Convolutional Blind Denoising of Real Photographs        

CBDNetThe model is a real image non-blind denoising model,对于RAW格式的图像,Its noise model is as follows:

image with real noise=图像信号L+real noise signaln(L).CBDNetModel using a noise estimation subnet estimate the noise level,Then together with the original input image input open-label denoising based on jump layer connection network,其结构如下:

1、The noise estimation sub-network converts the noise observation image to the estimated noise level image,Then enter it with the original image,Use the non-blind denoising sub-network to get the final denoising result,除此之外,Noise estimation operator network allows the user to estimate the noise level image input open-label alignment adjustment before denoising subnet,It proposes a simple strategy,即y = ay,i.e. linear scaling,This gives the model an interactive denoising capability.

盲去噪是指在去噪过程中,用于去噪的基础是从有噪声的样本本身学习来的.换句话说,无论我们构建什么样的深度学习体系结构,都应该学习图像中的噪声分布并去噪.所以和往常一样,这都取决于我们提供给深度学习模型的数据类型.The blind denoising, on the other hand.

2、Noise estimation network loss function:Asymmetric loss function( asymmetric loss)+Total Variation Loss Function( total variation)

        2.1、Asymmetric loss function,Calculate the noise estimate map and noiseground truth的平方差:

其中, is the penalty value,When the noise estimation figure value less than noiseground truth值的时候,high penalty,When the noise estimate map value is greater than the noiseground truth值的时候,low penalty.(非对称)The purpose is to avoid underestimating the noise value.

        2.2、Total Variation Loss Function,The purpose is to limit the smoothness of the noise estimate map(Gradient size to reflect the degree of a smooth):其中,Calculate the horizontal gradient of the noise estimate map(Noise Estimation Map Vertical Gradient)

        2.3、Non-blind noise reduction sub-network loss function:Calculate the pixel-level mean squared error between the output image and the input imageMSE:

The total loss function of the network is the sum of the three parts multiplied by the weights:

3、该算法学习的是更接近于真实噪声的高斯泊松噪声,而前面两篇论文都是学习高斯噪声;并且结合使用合成和真实噪声数据来训练模型,提高模型泛化能力,可以更好地对真实场景进行降噪;

 RIDNet

Real Image Denoising with Feature Attention

RIDNet中,The authors propose a new repair module,Learn features and further enhance the functionality of the network. The author team focuses on features by focusing on dependencies between channels,to rescale channel-level features. 还使用LSC,SSC和SCto bypass low frequency information,So the network can focus on study residual.网络结构如下:

该网络的特点主要有:

1、这个网路的结构设计得相对复杂,主要包括三部分:特征提取、4个EMA组成的残差模型、重建.其中特征提取和重建模块都是卷积层+ReLU层.EMA的结构如上图中下半部分框图所示:
(1)首先是两个空洞卷积分支,用来增加感受野,然后进行拼接并进行卷积融合
(2)然后是两个类似残差学习的结构,用于进行特征的提取
(3)最后是注意力机制,主要由一系列1x1的卷积核构成,结构如下图所示:

2、该网络的损失函数为L1损失函数: 

PMRID

Practical Deep Raw Image Denoising on Mobile Devices 

这篇论文是2020 CVPR上旷视提出来的一篇非常elegant的算法,该算法的特点网络结构比较小,通过一个k-sigma变换来解决小网络在不同增益噪声下的鲁棒性问题,网络结构如下图所示:

        该论文提出了k-sigma变换,使用标定的k和sigma按照k-sigma公式变换后,原始的噪声分布就只和没有噪声的数据x ∗ 有关,因此就可以避免不同增益下噪声不同带来的负担.Its specific principle can refer to:Night scene noise reduction algorithm in your phone-Raw域k-Sigma Transform - 知乎

 基于GAN的模型

        It is very difficult to obtain pairs of noisy and noise-free images,Therefore, some researchers have used generative adversarial networks(GAN)to generate paired images for training the model,首先,训练GAN网络,Through the noise of the image to study distribution and generate noise samples,Use this to simulate a noisy image in a real scene,解决HRImage lacks correspondenceLR的问题;其次,训练CNN网络,Use the noise blocks sampled in the previous step to construct a paired training dataset,该数据集用于训练CNNto denoise a given image.网络结构如下:

Estimation from noisy images

https://arxiv.org/pdf/1803.04189.pdf

       Since noise images and noise-free images are difficult to obtain,So is it possible to train a good noise model using only noisy images??Therefore, the researchers proposedNoise2Noise模型,其原理其实很简单:

        本来我们做图像降噪,需要输入的噪音图像 x,和 “干净样本” y.例如,x is a path traced rendering image rendered with a few beams,y It is a picture after long-term rendering.那么如果用 y 作为训练目标,生成 y 是个非常费时费力的过程.But in fact, if you think about it,可以用另一次快速渲染生成的另一个噪音图像(它相当于 y + 另一个不同的噪音)作为训练目标(所以叫Noise2Noise).只要训练样本够多,最终也相当于用 y 作为训练目标.原因是简单的统计学原理.

         

 

 参考链接

Image Denoising Dataset - 知乎

图像降噪算法——DnCNN / FFDNet / CBDNet / RIDNet / PMRID / SID_Leo-Peng的博客-CSDN博客_dncnn

深​​​​​​度学习——CBD-Net_浮生若梦,For the joy of Geometry's blog-CSDN博客_cbdnetHow to estimate the noise level map

Blind image denoising|GAN|GCBD - 知乎

image denoising Noise2Noise 和 Noise2Void_Suyue listens to Feng's blog-CSDN博客_noise2noise

书籍:《深度学习之摄影图像处理》

copyright notice
author[Turned_MZ],Please bring the original link to reprint, thank you.
https://en.hqmana.com/2022/255/202209120319066317.html

Random recommended