Overview and Development of True 3D Display Technology: Principles and Perspectives
ZHANG Mei1, WANG Fei-Yue2,3, GUO Zhen1, TANG Lele1, WANG Xiao2,3
1. Institute of Systems Engineering, Academy of Military Sciences, Chinese People's Liberation Army, Beijing 100141; 2. State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190; 3. Qingdao Academy of Intelligent Industries, Qingdao 266000
Abstract The true three-dimensional(3D) display technology aims to reproduce real 3D scenes. It becomes more urgently needed than traditional 2D display systems due to the advantages of natural 3D visual perception and intuitive user experience. Since the true 3D display represents the future development trend of display area, it is the crucial technology to promote the development of metaverse area. The window of the information link between the virtual and the real world can be directly opened through the true 3D display technologies. And the immersive human-computer interaction 3D visual perception can be felt more strongly than other display technologies. In this paper, the principles and representative prototypes of various true three-dimensional display technologies are summarized. Then, the advantages and disadvantages of various main true three-dimensional display technologies are analyzed in detail. The potential development trends of the 3D display technology are proposed. Finally, the prospect forecast for true 3D display technology is provided. With the development of computer technology, optoelectronic technology, 5G communication and other technologies, the true 3D display system with high-quality display, powerful computing capabilities and intelligent perception and interactive functions will be highly developed in the future. Various applications will gradually increase in the areas of military affairs, medicine, teaching, etc.
Fund:Beijing Natural Science Foundation(No.J200012), Youth Growth Project of PLA General Hospital(No.QNC19026)
Corresponding Authors:GUO Zhen, Ph.D., senior engineer. His research interests include artificial intelligent, computer software and hardware.
About author:: ZHANG Mei, Ph.D., engineer. Her research interests include optical design, 3D imaging acquisition and 3D imaging display. WANG Fei-Yue, Ph.D., professor. His research interests include modeling, analysis, and control of intelligent systems and complex systems. TANG Lele, Ph.D., associate researcher. His research interests include computer software and integrated circuit design. WANG Xiao, Ph.D., associate researcher. Her research interests include social computing, knowledge automation, knowledge robots, social transportation and parallel intelligence.
ZHANG Mei,WANG Fei-Yue,GUO Zhen等. Overview and Development of True 3D Display Technology: Principles and Perspectives[J]. Pattern Recognition and Artificial Intelligence, 2022, 35(8): 701-717.
[1] GERSHUN A. The Light Field. Journal of Mathematics and Physics, 1939, 18(1/2/3/4): 51-151. [2] ADELSON E H, BERGEN J R.The Plenoptic Function and the Elements of Early Vision // LANDY M, MOVSHON J A, eds. Computational Models of Visual Processing. Cambridge, USA: MIT Press, 1991: 3-20. [3] LEVOY M, HANRAHAN P.Light Field Rendering // Proc of the 23rd Annual Conference on Computer Graphics and Interactive Techniques. New York, USA: ACM, 1996: 31-42. [4] SHIN D H, LEE B, KIM E S.Multidirectional Curved Integral Imaging with Large Depth by Additional Use of a Large-Aperture Lens. Applied Optics, 2006, 45(28): 7375-7381. [5] PARK G, HONG J, KIM Y, et al. Enhancement of Viewing Angle and Viewing Distance in Integral Imaging by Head Tracking[C/OL].[2022-01-30]. https://opg.optica.org/abstract.cfm?URI=DH-2009-DWB27. [6] XIE W, WANG Y Z, DENG H, et al. Viewing Angle-Enhanced Integral Imaging System Using Three Lens Arrays. Chinese Optics Letters, 2014, 12(1). DOI: 10.3788/COL201412.011101. [7] YU X B, SANG X Z, CHEN D, et al. 3D Display with Uniform Resolution and Low Crosstalk Based on Two Parallax Interleaved Barriers. Chinese Optics Letters, 2014, 12(12). DOI: 10.3788/COL201412.121001. [8] LEE S, JANG C, CHO J, et al. Viewing Angle Enhancement of an Integral Imaging Display Using Bragg Mismatched Reconstruction of Holographic Optical Elements. Applied Optics, 2016, 55(3): A95-A103. [9] HE M Y, ZHANG H L, DENG H, et al. Dual-View-Zone Tabletop 3D Display System Based on Integral Imaging. Applied Optics, 2018, 57(4): 952-958. [10] YU X B, SANG X Z, GAO X, et al. Large Viewing Angle Three-Dimensional Display with Smooth Motion Parallax and Accurate Depth Cues. Optics Express, 2015, 23(20): 25950-25958. [11] YANG S W, SANG X Z, GAO X, et al. Influences of the Pickup Process on the Depth of Field of Integral Imaging Display. Optics Communications, 2017, 386: 22-26. [12] XING S J, SANG X Z, YU X B, et al. High-Efficient Computer-Generated Integral Imaging Based on the Backward Ray-Tracing Technique and Optical Reconstruction. Optics Express, 2017, 25(1): 330-338. [13] LIU B Y, SANG X Z, YU X B, et al. Time-Multiplexed Light Field Display with 120-Degree Wide Viewing Angle. Optics Express, 2019, 27(24): 35728-35739. [14] TAKAKI Y, NAGO N.Multi-projection of Lenticular Displays to Construct a 256-View Super Multi-view Display. Optics Express, 2010, 18(9): 8824-8835. [15] TAKAKI Y, TANAKA K, NAKAMURA J.Super Multi-view Display with a Lower Resolution Flat-Panel Display. Optics Express, 2011, 19(5): 4129-4139. [16] WANG P R, SANG X Z, YU X B, et al. Demonstration of a Low-Crosstalk Super Multi-view Light Field Display with Natural Depth Cues and Smooth Motion Parallax. Optics Express, 2019, 27(23): 34442-34453. [17] BALOGH T, FORGÁCS T, AGÓCS T, et al. A Scalable Hardware and Software System for the Holographic Display of Interactive Graphics Applications[C/OL].[2022-01-30]. https://www.crs4.it/vic/data/papers/eg2005-holo.pdf. [18] AGOCS T, BALOGH T, FORGÁCS T, et al. A Large Scale Interactive Holographic Display // Proc of the IEEE Virtual Reality Conference. Washington, USA: IEEE, 2006. DOI: 10.1109/VR.2006.9. [19] ADHIKARLA V K, WOŹNIAK P, BARSI A, et al. Freehand Interaction with Large-Scale 3D Map Data // Proc of the 3DTV-Conference: The True Vision-Capture(Transmission and Display of 3D Video). Washington, USA: IEEE, 2014. DOI: 10.1109/3DTV.2014.6874711. [20] JONES A V, NAGANO K, LIU J, et al. Interpolating Vertical Parallax for an Autostereoscopic Three-Dimensional Projector Array. Journal of Electronic Imaging, 2014, 23(1). DOI: 10.1117/1.JEI.23.1.011005. [21] JONES A, UNGER J, NAGANO K, et al. An Automultiscopic Projector Array for Interactive Digital Humans // Proc of the ACM SIGGRAPH 2015 Emerging Technologies. New York, USA: ACM, 2015. DOI: 10.1145/2782782.2792494. [22] ZHONG Q, PENG Y F, LI H F, et al. Multiview and Light-Field Reconstruction Algorithms for 360° Multiple-Projector-Type 3D Display. Applied Optics, 2013, 52(19): 4419-4425. [23] PENG Y F, LI H F, ZHONG Q, et al. Large-Sized Light Field Three-Dimensional Display Using Multi-projectors and Directional Diffuser. Optical Engineering, 2013, 52(1). DOI: 10.1117/1.OE.52.1.017402. [24] YOSHIDA S. fVisiOn: 360-Degree Viewable Glasses-Free Tabletop 3D Display Composed of Conical Screen and Modular Projector Arrays. Optics Express, 2016, 24(12): 13194-13203. [25] YOSHIDA S.Virtual Multiplication of Light Sources for a 360°-Viewable Tabletop 3D Display. Optics Express, 2020, 28(22): 32517-32528. [26] JONES A, MCDOWALL I, YAMADA H, et al. Rendering for an Interactive 360° Light Field Display. ACM Transactions on Graphics, 2007, 26(3). DOI: 10.1145/1276377.1276427. [27] JONES A, LANG M, FYFFE G, et al. Achieving Eye Contact in a One-to-Many 3D Video Teleconferencing System. ACM Transactions on Graphics, 2009, 28(3). DOI:10.1145/1531326.1531370 [28] XIA X X, ZHENG Z R, LIU X, et al. Omnidirectional-View Three-Dimensional Display System Based on Cylindrical Selective-Diffusing Screen. Applied Optics, 2010, 49(26): 4915-4920. [29] XIA X X, WU J, YAN C J, et al. P-5: A New 360-Degree Holo-Views Display System with Multi-vertical Views. SID Symposium Digest of Technical Papers, 2010, 41(1): 1241-1244. [30] XIA X X, LIU X, LI H F, et al. A 360-Degree Floating 3D Display Based on Light Field Regeneration. Optics Express, 2013, 21(9): 11237-11247. [31] SU C, ZHONG Q, YU C, et al. 360° Multi-faced Tracking and Interaction Using a Panoramic Camera. SID Symposium Digest of Technical Papers, 2015, 46(1): 151-154. [32] SONG W T, ZHU Q D, LIU Y, et al. Omnidirectional-View Three-Dimensional Display Based on Rotating Selective-Diffusing Screen and Multiple Mini-Projectors. Applied Optics, 2015, 54(13): 4154-4160. [33] LANMAN D, HIRSCH M, KIM Y, et al. Content-Adaptive Parallax Barriers for Automultiscopic 3D Display // Proc of the ACM SIGGRAPH 2010 Talks. New York, USA: ACM, 2010. DOI: 10.1145/1837026.1837097. [34] LANMAN D, WETZSTEIN G, HIRSCH M, et al. Polarization Fields: Dynamic Light Field Display Using Multi-layer LCDs. ACM Transactions on Graphics, 2011, 30(6). DOI: 10.1145/2070781.2024220. [35] WETZSTEIN G, LANMAN D, HEIDRICH W, et al. Layered 3D: Tomographic Image Synthesis for Attenuation-Based Light Field and High Dynamic Range Displays. ACM Transactions on Graphics, 2011, 30(4). DOI: 10.1145/2010324.1964990. [36] WETZSTEIN G, LANMAN D, HIRSCH M, et al. Tensor Displays: Compressive Light Field Synthesis Using Multilayer Displays with Directional Backlighting. ACM Transactions on Graphics, 2012, 31(4). DOI: 10.1145/2185520.2185576. [37] HEIDE F, WETZSTEIN G, RASKAR R, et al. Adaptive Image Synthesis for Compressive Displays. ACM Transactions on Graphics, 2013, 32(4). DOI: 10.1145/2461912.2461925. [38] CHEN R J, MAIMONE A, FUCHS H, et al. Wide Field of View Compressive Light Field Display Using a Multilayer Architecture and Tracked Viewers. Journal of the Society for Information Display, 2014, 22(10): 525-534. [39] HUANG F C, CHEN K, WETZSTEIN G.The Light Field Stereoscope: Immersive Computer Graphics via Factored Near-Eye Light Field Displays with Focus Cues. ACM Transactions on Graphics, 2015, 34(4). DOI: 10.1145/2766922. [40] CAO X, GENG Z, ZHANG M, et al. Load-Balancing Multi-LCD Light Field Display. Proceedings of the SPIE, 2015, 9391. DOI: 10.1117/12.2078366. [41] CAO X, GENG Z, LI T T, et al. Accelerating Decomposition of Light Field Video for Compressive Multi-layer Display. Optics Express, 2015, 23(26): 34007-34022. [42] FATTAL D, PENG Z, TRAN T, et al. A Multi-directional Backlight for a Wide-Angle, Glasses-Free Three-Dimensional Display. Nature, 2013, 495: 348-351. [43] HUA J Y, HUA E, ZHOU F B, et al. Foveated Glasses-Free 3D Display with Ultrawide Field of View via a Large-Scale 2D-Metagrating Complex. Light: Science & Applications, 2021, 10. DOI: 10.1038/s41377-021-00651-1. [44] LEUNG M S, IVES N A, ENG G. Three-Dimensional Real-Image Volumetric Display System and Method: USA, 5745197, 1998-04-28. [45] SULLIVAN A.DepthCube Solid-State 3D Volumetric Display. Pro-ceeding of the SPIE, 2004, 5291: 279-284. [46] FENG Q B, TONG H, LIU T, et al. Single-DMD Based Solid-State Volumetric True 3D Display // Proc of the International Conference on Audio, Language and Image Processing. Washington, USA: IEEE, 2010: 552-556. [47] 芦云龙,盛杰超,方勇,等.固态体积式真三维立体显示效果优化.液晶与显示, 2016, 31(5): 518-523. (LU Y L, SHENG J C, FANG Y, et al. Optimizing Effect of Solid Volumetric True 3D Display. Chinese Journal of Liquid Crystals and Displays, 2016, 31(5): 518-523.) [48] 陈晓静. 40寸固态体积式真三维立体显示器光学引擎设计及样机开发.硕士学位论文.合肥:合肥工业大学, 2017. (CHEN X J.Optical Engine Design and Prototype Development of 40 Inch Solid-State Volumetric True 3D Display. Master Dissertation. Hefei, China: Hefei University of Technology, 2017.) [49] 方勇. 固态体积式真三维显示关键技术研究与实现.博士学位论文.合肥:合肥工业大学, 2017. (FANG Y.Research and Implementation on Key Technologies of Solid-State 3D Volumetric Display. Ph.D. Dissertation. Hefei, China: Hefei University of Technology, 2017.) [50] BAHR D, LANGHANS K, GERKEN M.FELIX: A Volumetric 3D Laser Display. Proceedings of the SPIE, 1996, 2650: 265-273. [51] LANGHANS K, BEZECNY D, HOMANN D, et al. New Portable FELIX 3D Display. Proceedings of the SPIE, 1998, 3296: 204-216. [52] LANGHANS K, BAHR D, BEZECNY D, et al. FELIX 3D Display: An Interactive Tool for Volumetric Imaging. Proceedings of the SPIE, 2002, 4660: 176-190. [53] FAVALORA G E, NAPOLI J, HALL D M, et al. 100-Million-Voxel Volumetric Display. Proceedings of the SPIE, 2002, 4712: 300-312. [54] 李莉. 体三维显示系统关键技术研究与实现.博士学位论文.南京:南京航空航天大学, 2008. (LI L.Research and Implementation on Key Technologies of Volumetric Display System. Ph.D. Dissertation. Nanjing, China: Nanjing University of Aeronautics and Astronautics, 2008.) [55] GENG J.A Volumetric 3D Display Based on a DLP Projection Engine. Displays, 2013, 34(1): 39-48. [56] DOWNING E, HESSELINK L, RALSTON J, et al. A Three-Color, Solid-State, Three-Dimensional Display. Science, 1996, 273(5279): 1185-1189. [57] NAYAR S K, ANAND V N.3D Display Using Passive Optical Scatters. Computer, 2007, 40(7): 54-63. [58] OCHIAI Y, KUMAGAI K, HOSHI T, et al. Fairy Lights in Femtoseconds: Aerial and Volumetric Graphics Rendered by Focused Femtosecond Laser Combined with Computational Holographic Fields. ACM Transactions on Graphics, 2016, 35(2). DOI: 10.1145/2850414. [59] GABOR D.A New Microscopic Principle. Nature, 1948, 161: 777-778. [60] BENTON S A, BOVE V M. Holographic Imaging. New Jersey, USA: Wiley, 2007. [61] TAY S, BLANCHE P A, VOORAKARANAM R, et al. An Updatable Holographic Three-Dimensional Display. Nature, 2008, 451: 694-698. [62] DING S H, CAO S Y, ZHENG Y F, et al. From Image Pair to a Computer Generated Hologram for a Real-World Scene. Applied Optics, 2016, 55(27): 7583-7592. [63] CHANG C L, QI Y J, XIA J, et al. Lensless Holographic 3D Display Based on Fast Calculated Computer-Generated Hologram. SID Symposium Digest of Technical Papers, 2017, 48(1): 811-814. [64] SANDO Y, SATOH K, BARADA D, et al. Real-Time Interactive Holographic 3D Display with a 360° Horizontal Viewing Zone. Applied Optics, 2019, 58(34): G1-G5. [65] GAO H, WANG Y X, FAN X H, et al. Dynamic 3D Meta-Holography in Visible Range with Large Frame Number and High Frame Rate. Science Advances, 2020, 6(28). DOI: 10.1126/sciadv.aba8595. [66] SU Y F, CAI Z J, WU K, et al. Projection-Type Multiview Holographic Three-Dimensional Display Using a Single Spatial Light Modulator and a Directional Diffractive Device. IEEE Photonics Journal, 2018, 10(5). DOI: 10.1109/JPHOT.2018.2871936. [67] DENG H, CHEN C, HE M Y, et al. High-Resolution Augmented Reality 3D Display with Use of a Lenticular Lens Array Holographic Optical Element. Journal of the Optical Society of America A(Optics, Image Science, and Vision), 2019, 36(4): 588-593. [68] JO Y, BANG K, YOO D, et al. Ultrahigh-Definition Volumetric Light Field Projection. Optics Letters, 2021, 46(17): 4212-4215. [69] HIRAYAMA R, PLASENCIA D M, MASUDA N, et al. A Volumetric Display for Visual, Tactile and Audio Presentation Using Acoustic Trapping. Nature, 2019, 575: 320-323. [70] YAMAGUCHI M.Full-Parallax Holographic Light-Field 3-D Display and Interactive 3-D Touch. Proceedings of the IEEE, 2017, 105(5): 947-959. [71] TIAN M Z, NI L X, XU L, et al. Multi-face Real-Time Tracking Based on Dual Panoramic Camera for Full-Parallax Light-Field Display. Optics Communications, 2019, 442: 19-26. [72] YU X B, SANG X Z, GAO X, et al. Dynamic Three-Dimensional Light-Field Display with Large Viewing Angle Based on Compound Lenticular Lens Array and Multi-Projectors. Optics Express, 2019, 27(11): 16024-16031. [73] AN J, WON K, LEE H S. Past, Current,Future of Holographic Video Display. Applied Optics, 2022, 61(5): B237-B245. [74] EL-GHOROURY H S, CHUANG C L, ALPASLAN Z Y. Quantum Photonic Imager(QPI): A Novel Display Technology that Enables more than 3D Applications. SID Symposium Digest of Technical Papers, 2015, 46(1). DOI: 10.1002/sdtp.10255. [75] WAN W Q, QIAO W, PU D L, et al. Holographic Sampling Display Based on Metagratings. iScience, 2020, 23(1). DOI: 10.1016/j.isci.2019.100773. [76] AN J, WON K, KIM Y, et al. Slim-Panel Holographic Video Display. Nature Communications, 2020, 11(1). DOI: 10.1038/s41467-020-19298-4. [77] REN H, WANG Q H, XING Y, et al. Super-Multiview Integral Imaging Scheme Based on Sparse Camera Array and CNN Super-Resolution. Applied Optics, 2019, 58(5): A190-A196. [78] GUO X, SANG X Z, CHEN D, et al. Real-Time Optical Reconstruction for a Three-Dimensional Light-Field Display Based on Path-Tracing and CNN Super-Resolution. Optics Express, 2021, 29(3): 37862-37876. [79] CHEN D, SANG X Z, PENG W, et al. Multi-parallax Views Synthesis for Three-Dimensional Light-Field Display Using Unsupervised CNN. Optics Express, 2018, 26(21): 27585-27598. [80] PEI R J, MA K, WANG F Y.3D Display Parallel System: Light Field Re-rendering and Depth Sense Optimization. SID Symposium Digest of Technical Papers, 2018, 49(1): 125-131. [81] 王飞跃. 计算实验方法与复杂系统行为分析和决策评估.系统仿真学报, 2004, 16(5): 893-897. (WANG F Y.Computational Experiments for Behavior Analysis and Decision Evaluation of Complex Systems. Journal of System Simulation, 2004, 16(5): 893-897.) [82] WANG F Y.Scanning the Issue and Beyond: Parallel Driving with Software Vehicular Robots for Safety and Smartness. IEEE Transactions on Intelligent Transportation Systems, 2014, 15(4): 1381-1387. [83] WANG F Y, ZHANG J, WEI Q L, et al. PDP: Parallel Dynamic Programming. IEEE/CAA Journal of Automatica Sinica, 2017, 4(1). DOI: 10.1109/JAS.2017.7510310. [84] WANG F Y.Computational Social Systems in a New Period: A Fast Transition into the Third Axial Age. IEEE Transactions on Computational Social Systems, 2017, 4(3): 52-53. [85] WANG F Y, WONG P K.Intelligent Systems and Technology for Integrative and Predictive Medicine: An ACP Approach. ACM Transactions on Intelligent Systems and Technology, 2013, 4(2). DOI: 10.1145/2438653.2438667. [86] 王飞跃,孟祥冰,杜思聪,等.平行光场:基本框架与流程.智能科学与技术学报, 2021, 3(1): 110-122. (WANG F Y, MENG X B, DU S C, et al. Parallel Light Field: The Framework and Processes. Chinese Journal of Intelligent Science and Technology, 2021, 3(1): 110-122.) [87] 王飞跃.平行光场与平行光学,从光学计算实验到光学引导智能[R/OL]. [2022-01-30]. http://www.sklmccs.ia.ac.cn/2018reports.html. (WANG F Y. Parallel Light Field and Parallel Optics, from Optical Computing Experiment to Optical Guided Intelligence[R/OL]. [2022-01-30]. http://www.sklmccs.ia.ac.cn/2018reports.html
2022, Vol. 35 
