MFSRNet: spatial-angular correlation retaining for light field super-resolution

Sizhe Wang, Hao Sheng, Da Yang, Zhenglong Cui, Ruixuan Cong, Wei Ke

研究成果: Article同行評審

1 引文 斯高帕斯(Scopus)


Light field (LF) images acquired by hand-held devices suffer from a trade-off between spatial and angular resolutions. To solve this problem, super-resolution (SR) in the spatial and angular domains is studied separately in previous works. However, spatial-angular correlation can not be reconstructed effectively by the separate SR methods. In this paper, a multi-scale feature-assisted synchronous SR network (MFSRNet) is presented to retain spatial-angular correlation for spatial and angular super-resolution, which consists of four modules: multi-scale feature extraction (MFE), view relation reconstruction (VRR), SR information acquisition (SIA) and up-sampling. The MFE module is used to acquire multi-scale angular SR features from low-resolution LF. In the VRR module, these multi-scale features are concatenated with two original adjacent low-resolution view images to reconstruct the angular relation among original and new views. Then, a continuous fusion mechanism is proposed in the SIA module to obtain spatial SR information from four surrounding views and reconstruct the spatial-angular correlation in LF. Finally, super-resolved LF is generated by allocating the sub-pixel information in the up-sampling module. Furthermore, a combined loss is proposed to provide constraints on both angular feature extraction and spatial and angular synchronous SR, and train MFSRNet in an end-to-end fashion. On synthetic and real-world datasets, experimental results show that our algorithm outperforms other state-of-the-art methods in both visual and numerical evaluations. Especially, our method brings significant improvements for sparse LFs from the dataset STFgantry using MFSRNet. Our method improves PSNR/SSIM while preserving the inherent epipolar property in LF.

頁(從 - 到)20327-20345
期刊Applied Intelligence
出版狀態Published - 9月 2023


深入研究「MFSRNet: spatial-angular correlation retaining for light field super-resolution」主題。共同形成了獨特的指紋。