Direct Inter-Intra View Association for Light Field Super-Resolution

Light field (LF) cameras record both intensity and directions of light rays in a scene with a single exposure. However, due to the inevitable trade-off between spatial and angular dimensions, the spatial resolution of LF images is limited which makes LF super-resolution (LFSR) a research hotspot. The key of LFSR is the complementation across views and the extraction of high-frequency information inside each view. Due to the high-dimensinality of LF data, previous methods usually model these two processes separately, which results in insufficient inter-view information fusion. In this paper, LF Transformer is proposed for comprehensive perception of 4D LF data. Necessary inter-intra view correlations can be directly established inside each LF Transformer block. Therefore it can handle complex disparity variations of LF. Then based on LF Transformers, 4DTNet is designed which comprehensively performs inter-intra view high-frequency information extraction. Extensive experiments on public datasets demonstrate that 4DTNet outperforms the current state-of-the-art methods both numerically and visually.