Abstract To achieve accurate depth estimation by using light field data effectively, the light field depth estimation is reviewed in this paper. Firstly, the basic theory of light field is expounded, and the light field depth estimation methods are classified into three categories, methods based on epipolar plane image, multiview and refocusing. Next, the effects of illuminant variations on the performance of depth estimation are compared on synthetic datasets. Besides, a more comprehensive and challenging light field dataset is constructed, and the effect of complex scenes on the performance of depth estimation is qualitatively and quantitatively analyzed on the light field benchmark dataset and LytroDataset. Furthermore, the development of this field is pointed out.
About author:: GAO Jun, born in 1963, Ph.D., professor. His research interests include intelligent information processing and pattern recognition.WANG Lijuan, born in 1990, master student. Her research interests include machine vision and light field technologyZHANG Xudong, born in 1966, Ph.D., professor. His research interests include intelligent information processing and machine vision.ZHANG Jun(Corresponding author), born in 1984, Ph.D., lecturer. Her research interests include computer vision, pattern recognition and cognitive science.)
[1] YOUNG M J, LANDY M S, MALONEY L T. A Perturbation Analysis of Depth Perception from Combinations of Texture and Motion Cues. Vision Research, 1993, 33(18): 2685-2696. [2] MAENO K, NAGAHARA H, SHIMADA A, et al. Light Field Distortion Feature for Transparent Object Recognition // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2013: 2786-2793. [3] ZHANG X D, WANG Y, ZHANG J, et al. Light Field Saliency vs. 2D Saliency: A Comparative Study. Neurocomputing, 2015, 166: 389-396. [4] LI N Y, YE J W, JI Y, et al. Saliency Detection on Light Field // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2014: 2806-2813. [5] LI N Y, SUN B L, YU J Y. A Weighted Sparse Coding Framework for Saliency Detection // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015: 5216- 5233. [6] ZHANG J, WANG M, GAO J, et al. Saliency Detection with a Deeper Investigation of Light Field // Proc of the 24th International Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2015: 2212-2218. [7] 张旭东,李梦娜,张 骏,等.边缘结构保持的加权BDTV全光场图像超分辨率重建.中国图象图形学报, 2015, 20(6): 733-739. (ZHANG X D, LI M N, ZHANG J, et al. Edge Preserved Light Field Image Super-Resolution Based on Weighted BDTV Model. Journal of Image and Graphics, 2015, 20(6): 733-739.) [8] YOON Y G, JEON H G, YOO D G, et al. Learning a Deep Convolutional Network for Light-Field Image Super-Resolution // Proc of the IEEE International Conference on Computer Vision Workshop. New York, USA: IEEE, 2015: 57-65. [9] BOYACI H, DOERSCHNER K, SNYDER J L, et al. Surface Color Perception in Three-Dimensional Scenes. Visual Neuroscience, 2006, 23(3/4): 311-321. [10] 徐树奎,张 军,涂 丹,等.继承、颠覆与超越——计算摄影.计算机研究与发展, 2012, 49(1): 128-143. (XU S K, ZHANG J, TU D, et al. Inheritance, Subversion and Transcendence-Computational Photograph. Journal of Computer Research and Development, 2012, 49(1): 128-143.) [11] 周志良.光场成像技术研究.博士学位论文.合肥:中国科学技术大学, 2012. (ZHOU Z L. Research on Light Field Imaging Technology. Ph.D Dissertation. Hefei, China: University of Science and Technology of China, 2012.) [12] GOLDLUECKE B. Globally Consistent Depth Labeling of 4D Light Fields // Proc of the IEEE International Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE, 2012: 41-48. [13] WANNER S, GOLDLUECKE B. Variational Light Field Analysis for Disparity Estimation and Super-Resolution. IEEE Trans on Pa-ttern Analysis and Machine Intelligence, 2013, 36(3): 606-619. [14] STOLC S, HUBER-MRK R, HOLLNDER B, et al. Depth and All-in-Focus Images Obtained by Multi-line-Scan Light-Field Approach. Image Processing: Machine Vision Applications VII, 2014. DOI: 10.1117/12.2042475. [15] KIM C, ZIMMER H, PRITCH Y, et al. Scene Reconstruction from High Spatio-Angular Resolution Light Fields. ACM Trans on Graphics, 2013, 32(4). DOI: 10.1145/2461912.2461926. [16] ULIYAR M, PUTRAYA G, BASAVARAJA S V. Fast EPI Based Depth for Plenoptic Cameras // Proc of the 20th IEEE International Conference on Image Processing. New York, USA: IEEE, 2013: 1-4. [17] LKE J P, ROSA F, MARICHAL-HERNNPEZ J G, et al. Depth from Light Fields Analyzing 4D Local Structure. Journal of Display Technology, 2015, 11(11): 900-907. [18] LI J Q, LI Z N. Continuous Depth Map Reconstruction from Light Fields. IEEE Trans on Image Processing, 2015, 24(11): 3257-3265. [19] CHEN C, LIN H T, YU Z, et al. Light Field Stereo Matching Using Bilateral Statistics of Surface Cameras // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE, 2014: 1518-1525. [20] TOSIC I, BERKNER K. Light Field Scale-Depth Space Transform for Dense Depth Estimation // Proc of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Washington, USA: IEEE, 2014: 441-448. [21] BOWEN A, MULLINS A, RAJPOOT N, et al. Photo-Consistency and Multiresolution Methods for Light Field Disparity Estimation // Proc of the IEE International Conference on Visual Information Engineering. Herts, UK: IET, 2005: 401-408. [22] BISHOP T E, FAVARO P. Full-Resolution Depth Map Estimation from an Aliased Plenoptic Light Field // Proc of the 10th Asian Conference on Computer Vision. Berlin, Germany: Springer, 2010: 186-200. [23] SABATER N, SEIFI M, DRAZIC V, et al. Accurate Disparity Estimation for Plenoptic Images // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2015: 548-560. [24] YU Z, YU J Y, LUMSDAINE A, et al. Plenoptic Depth Map in the Case of Occlusions. Proc of SPIE, 2013, 8667. DOI: 10.1117/12.2005847. [25] YU Z, GUO X Q, LING H B, et al. Line Assisted Light Field Triangulation and Stereo Matching // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2013: 2792-2799. [26] JEON H G, PARK J, CHOE G, et al. Accurate Depth Map Estimation from a Lenslet Light Field Camera // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015: 1547-1555. [27] CALDERON F C, PARRA C A, NINO C L. Depth Map Estimation in Light Fields Using an Stereo-Like Taxonomy // Proc of the Symposium on Image, Signal Processing and Artificial Vision. New York, USA: IEEE, 2014: 1-5. [28] BISHOP T E, FAVARO P. The Light Field Camera: Extended Depth of Field, Aliasing, and Superresolution. IEEE Trans on Pa-ttern Analysis and Machine Intelligence, 2012, 34(5): 972-986. [29] WANG T C, EFROS A A, RAMAMOORTHI R. Occlusion-Aware Depth Estimation Using Light-Field Cameras // Proc of the IEEE International Conference on Computer Vision. New York, USA: IEEE, 2015: 3487-3495. [30] HEBER S, POCK T. Shape from Light Field Meets Robust PCA // Proc of the 13th European Conference on Computer Vision. Berlin, Germany: Springer, 2014, VI: 751-767. [31] HEBER S, RANFTL R, POCK T. Variational Shape from Light Field // Proc of the 9th International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition. Berlin, Germany: Springer, 2013: 66-79. [32] MOUSNIER A, VURAL E, GUILLEMOT C. Partial Light Field Tomographic Reconstruction from a Fixed-Camera Focal Stack [J/OL].[2016-01-15].http//arxiv.org/pdf/1503.01903v1.pdf. [33] TAO M W, HADAP S, MALIK J, et al. Depth from Combining Defocus and Correspondence Using Light-Field Cameras // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2013: 673-680. [34] TAO M W, SRINIVASAN P P, MALIK J, et al. Depth from Shading, Defocus, and Correspondence Using Light-Field Angular Coherence // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE, 2015: 1940-1948. [35] LIN H T, CHEN C, KANG S B, et al. Depth Recovery from Light Field Using Focal Stack Symmetry // Proc of the IEEE Internatio-nal Conference on Computer Vision. New York, USA: IEEE, 2015: 3451-3459. [36] LU W, WAN K M, JEREMY N. 3D and Image Stitching with the Lytro Light-Field Camera[T/OL]. [2016-01-05]. http://jeremyneiman.com/projects/lytro_report.pdf. [37] WANNER S, MEISTER S, GOLDLUECKE B. Datasets and Benchmarks for Densely Sampled 4D Light Fields // BRONSTEIN M, FAVRE J, HORMANN K, eds. Vision, Modeling and Visualization. Geneva, Switzerland: The Eurographics Association, 2013: 225-226. [38] LEVOY M, HANRAHAN P. Light Field Rendering // Proc of the 23rd Annual Conference on Computer Graphics and Interactive Techniques. New York, USA: ACM Press, 1996: 31-42. [39] GHASEMI A, VETTERLI M. Scale-Invariant Representation of Light Field Images for Object Recognition and Tracking. Proc of SPIE, 2014: 9020.DOI: 10.1117/12.2041105. [40] AGARWALA A, DONTCHEVA M, AGRAWALA M, et al. Interactive Digital Photomontage // Proc of the 31st Annual Conference on Computer Graphics and Interactive Techniques. New York, USA: ACM Press, 2004: 294-302
2016, Vol. 29 
