Feb.27~image super-resolution reconstruction， paper reading

Multi-frame image super-resolution reconstruction via low-rank fusion combined with sparse coding

这篇文章看起来和2月初读的一篇关于图像去噪是自相矛盾的，这篇文章可以方法成立的条件是，HR和LR图片用字典表示时候的协方差系数矩阵是类似的，也就是那个 $\alpha$α 在两种情况下是一样的。在求解中也应用到了从三个方向特征提取的技巧，还有K-SVD方法。

（思考：那么在这里的特征提取是否可以接入神经网络，模型自己训练，自己提取特征呢？）这里面提取特征是在第二部，是为了提取出特征之后这些特征向量组成训练集，再用K-SVD方法提取出字典。

之前的文章中，两种情况下的$\alpha$α 是有偏差的，之后在模型优化中，专门加入了一项，表示两个协方差的距离，接着进行一系列的估计，迭代，然后求解。

According to the number of input LR images, image SR can be divided into single -frame and multi - frame.

Single-frame SR refers to restore a HR image from a LR image, and multi-frame SR means to recover a HR image from LR image sequence.

2. Low rank fusion and sparse coding

2.1 Low rank fusion

A data matrix $M\in \Bbb{R}^{n\times m}$M∈Rn×m contains structural information as well as noise. Then we can decompose M as
M = L + S
where L is low-rank matrix(contains the internal structure information which are linearly related.)
S is sparse (noise is sparse)
RPCA(robust principle component analysis) method model used to optimize above problem:
$\underset{L,S}{min}~rank(L)+\lambda||S||_0, s.t. M = L +S~~~(1)$L,Smin​ rank(L)+λ∣∣S∣∣0​,s.t.M=L+S   (1)
$\lambda$λ is a balance parameter.
(1) is a non-convex problem and it can be replaced by (2)
$\underset{L,S}{min}~||L||_*+\lambda||S||_1, s.t. M = L +S~~~(2)$L,Smin​ ∣∣L∣∣∗​+λ∣∣S∣∣1​,s.t.M=L+S   (2)
Augmented Lagrange Multiplier (ALM) algorithm is usually used to solve (2)

Supposing there are N frame registered images, then they are converted into a column vector and stored in a matrix by column. Now the matrix M={$M_1,M_2,⋯M_N$M1​,M2​,⋯MN​} has low-rank characteristics.
Next we decompose the matrix by low-rank and sparse decomposition.
Finally the decomposed low-rank part L = {$L_1, L_2, ⋯L_N$L1​,L2​,⋯LN​} is average fused to get the final fusion LR image L∗ =($L_1 +L_2 +⋯+L_N$L1​+L2​+⋯+LN​)/N,

2.2 Sparse coding

After down sampling S and fuzzy B, HR image X is degenerated into LR image Y:
$Y = SBX ~~~~~~~~~~(3)$Y=SBX          (3)
Where SB = L, Y = LX.
If y is an image patch taken from Y ,x is an image patch taken from X which is in the same location with y.
The sparse representation model can be represented as following:
$min ||\alpha||_1, s.t. x= D_h\alpha~~~~~~(4)$min∣∣α∣∣1​,s.t.x=Dh​α      (4)
$D_h\in\Bbb{R}^{n\times K}$Dh​∈Rn×K(K>n) is a HR over-complete dictionary.
According to (3)(4), so y can be expressed as following:
$y = LD_h\alpha = D_l \alpha~~~~~~~~~~~~~~(5)$y=LDh​α=Dl​α              (5)
Where $D_l = LD_h, D_l\in \Bbb{R}^{m\times K}$Dl​=LDh​,Dl​∈Rm×K(K>m) is a LR over-complete dictionary.
So it can be clarified that HR and LR image patches have the same sparse representation coefficient. ($\alpha_x = \alpha_y$αx​=αy​)
Based on a pair of HR and LR dictionaries {$D_h, D_l$Dh​,Dl​}, we are able to rebuild the correspond- ing HR image patch as long as we acquire sparse representation coefficient of the LR image patch.

3. Multi-frame image SR via low-rank fusion combines with sparse coding

3.1 method

The Multi-frame SR using low-rank fusion combines with sparse coding includes three steps: image registration and low-rank fusion, {$D_h, D_l$Dh​,Dl​} dictionary training and SR reconstruction, as shown in Fig. 3.
In image registration and low-rank fusion phase, SURF and RANSAC algorithm are used to image registration. Then registered images are decomposed into the low-rank images and the sparse images, and the low-rank images are fused into a LR image.
In {$D_h, D_l$Dh​,Dl​} dictionary training phase, patch haar wavelet transform is used to get image patches characteristic vectors, and characteristic vectors constitute the joint training set. K-SVD algorithm is applied to train joint training set to obtain the {$D_h, D_l$Dh​,Dl​}.
In SR reconstruction phase, after computing sparse representation coefficient $\alpha$α of LR patch, the HR patch can be obtained from the coefficient $\alpha$α multiplied by the HR dictionary $D_h$Dh​.

The END of Method of this paper.