Abstract
A novel light field super-resolution algorithm to improve the spatial and angular resolutions of light field images is proposed in this work. We develop spatial and angular super-resolution (SR) networks, which can faithfully interpolate images in the spatial and angular domains regardless of the angular coordinates. For each input image, we feed adjacent images into the SR networks to extract multi-view features using a trainable disparity estimator. We concatenate the multi-view features and remix them through the proposed adaptive feature remixing (AFR) module, which performs channel-wise pooling. Finally, the remixed feature is used to augment the spatial or angular resolution. Experimental results demonstrate that the proposed algorithm outperforms the state-of-the-art algorithms on various light field datasets. The source codes and pre-trained models are available at https://github.com/keunsoo-ko/ LFSR-AFR
Original language | English |
---|---|
Article number | 9394760 |
Pages (from-to) | 4114-4128 |
Number of pages | 15 |
Journal | IEEE Transactions on Image Processing |
Volume | 30 |
DOIs | |
Publication status | Published - 2021 |
Bibliographical note
Funding Information:Manuscript received July 31, 2020; revised February 8, 2021; accepted March 19, 2021. Date of publication April 2, 2021; date of current version April 9, 2021. This work was supported by the National Research Foundation of Korea (NRF) funded by the Korean Government (MSIT) under Grant NRF-2018R1A2B3003896 and Grant NRF-2019R1F1A1062907. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Yun He. (Corresponding author: Chang-Su Kim.) Keunsoo Ko and Chang-Su Kim are with the School of Electrical Engineering, Korea University, Seoul 02841, South Korea (e-mail: [email protected]; [email protected]).
Publisher Copyright:
© 1992-2012 IEEE.
Keywords
- Light field
- convolutional neural network (CNN)
- feature remixing
- super-resolution
ASJC Scopus subject areas
- Software
- Computer Graphics and Computer-Aided Design