Binocular Stereo Matching Algorithm Based on Labeled Matching Region Correction
ZHOU Jiali1,2, CHEN Yu1, WU Chao1, WU Min3
1. College of Science, Zhejiang University of Technology, Hangzhou 310023 2. Key Laboratory of Advanced Manufacturing Technology of Zhe-jiang Province, Zhejiang University, Hangzhou 310027 3. School of Sciences, Zhejiang University of Science and Technology, Hangzhou 310023
Abstract Binocular stereo vision is an accurate and effective measurement method. A binocular stereo matching method for label matching region correction is proposed. Based on the conventional graph cuts algorithm, the matching region is corrected by the spatial geometrical information of the block and a higher subpixel accuracy matching disparity map is obtained. Firstly, the correction transformation is determined by label and spatial geometrical information, and consequently the pixel information of the matching area of the left and right images are fully exploited. Then, the candidate 3D label is updated repeatedly to find the label minimizing the global energy. Finally, the left-right checking and the mean filtering are utilized to refine the disparity map. Experiments show that the proposed method is effective in finding a good, smooth piecewise linear disparity map with higher accuracy for edge region and occlusion region.
Fund:Supported by National Natural Science Foundation for Young Scientists of China(No.11301482), Key Research and Development Project of Zhejiang Province(No.2020C01005, 2020C0100 6), Graduate Teaching Reform Project of Zhejiang University of Technology(No.2018127)
Corresponding Authors:
WU Chao, Ph.D., lecturer. His research interests include pa-ttern recognition, image processing, differential geometry and its applications.
About author:: ZHOU Jiali, Ph.D., associate professor. His research interests include computer vision, pattern recognition, image processing, classification coding and industrial robotics. CHEN Yu, master student. His research interests include stereo vision.
[1] MENZE M, GEIGER A. Object Scene Flow for Autonomous Vehicles // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 3061-3070. [2] HAMZAH R A, KADMIN A F, HAMID M S, et al. Improvement of Stereo Matching Algorithm for 3D Surface Reconstruction. Signal Processing: Image Communication, 2018, 65: 165-172. [3] TANIAI T, MATSUSHITA Y, SATO Y, et al. Continuous 3D Label Stereo Matching Using Local Expansion Moves. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(11): 2725-2739. [4] HORNA L, FISHER R B. 3D Plane Labeling Stereo Matching with Content Aware Adaptive Windows // Proc of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. Washington, USA: IEEE. 2016: 162-171. [5] BESSE F, ROTHER C, FITZGIBBON A, et al. PMBP: PatchMatch Belief Propagation for Correspondence Field Estimation. International Journal of Computer Vision, 2014, 110(1): 2-13. [6] LI L C, ZHANG S L, YU X, et al. PMSC: PatchMatch-Based Superpixel Cut for Accurate Stereo Matching. IEEE Transactions on Circuits and Systems for Video Technology, 2018,28(3): 679-692. [7] BLEYER M, RHEMANN C, ROTHER C. PatchMatch Stereo-Ste-reo Matching with Slanted Support Windows // Proc of the British Machine Vision Conference. Berlin, Germany: Springer, 2011. DOI: 10.5244/C.25.14. [8] KOLMOGOROV V, ZABIN R. What Energy Functions Can Be Minimized via Graph Cuts? IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(2): 147-159. [9] FELZENSZWALB P F, HUTTENLOCHER D R. Efficient Belief Propagation for Early Vision // Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE. 2004: 261-268. [10] LU J B, YANG H S, MIN D B, et al. Patch Match Filter: Efficient Edge-Aware Filtering Meets Randomized Search for Fast Correspondence Field Estimation // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2013: 1854-1861. [11] BARNES C, SHECHTMAN E, FINKELSTEIN A, et al. Patch Match: A Randomized Correspondence Algorithm for Structural Image Editing. ACM Transactions on Graphics, 2009, 28(3): 24:1-24:11. [12] LEMPITSKY V, ROTHER C, ROTH S, et al. Fusion Moves for Markov Random Field Optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(8): 1392-1405. [13] HIRSCHMULLER H. Stereo Processing by Semiglobal Matching and Mutual Information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(2): 328-341. [14] ŽBONTAR J, LECUN Y. Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches. Journal of Machine Learning Research, 2016, 17: 1-32. [15] ZHANG S L, XIE W J, ZHANG G F, et al. Robust Stereo Mat-ching with Surface Normal Prediction // Proc of the IEEE International Conference on Robotics and Automation. Washington, USA: IEEE, 2017: 2540-2547. [16] 杜 英,刘 成,田 丹,等.弱纹理环境双目视觉稠密视差鲁棒性估计方法.光学精密工程, 2017, 25(4): 554-562. (DU Y K, LIU C, TIAN D, et al. Robust Estimation Method for Dense Disparity of Binocular Vision under Textureless Environment. Optics Precision Engineering, 2017, 25(4): 554-562.) [17] 王云峰,吴 炜,余小燕,等.基于自适应权重AD-Census变换的双目立体匹配.工程科学与技术, 2018, 50(4): 153-160. (WANG Y F, WU W, YU X Y, et al. A Stereo Matching System with the Adaptive Weight AD-Census. Advanced Engineering Sciences, 2018, 50(4): 153-160.) [18] 时 华,朱 虹.基于自适应匹配窗及多特征融合的立体匹配.模式识别与人工智能, 2016, 29(3): 193-202. (SHI H, ZHU H. Stereo Matching Based on Adaptive Matching Windows and Multi-feature Fusion. Pattern Recognition and Artificial Intelligence, 2016, 29(3): 193-202.) [19] 邬亚菲.基于多尺度超像素分割的立体匹配算法研究.硕士学位论文.杭州:浙江大学, 2017. (WU Y F. Research on Stereo Matching Based on Super-Pixel Segmentation. Master Dissertation. Hangzhou, China: Zhejiang University, 2017.) [20] 李玉洁.基于双目视觉的弱纹理场景三维重建.硕士学位论文.哈尔滨:哈尔滨工业大学, 2016. (LI Y J. Low-Textured Scene 3D Reconstruction Based on Binocular Stereo Vision. Master Dissertation. Harbin, China: Harbin Institute of Technology, 2016.) [21] RAZAK S S A, OTHMAN M A, KADMIN A F. The Effect of Adaptive Weighted Bilateral Filter on Stereo Matching Algorithm. International Journal of Engineering and Advanced Technology, 2019, 8(3): 284-287. [22] KITAGAWA M, SHIMIZU I, SARA R. High Accuracy Local Stereo Matching Using DoG Scale Map // Proc of the 15th IAPR International Conference on Machine Vision Applications. Washington, USA: IEEE, 2017: 258-261. [23] JELLAL R A, LANGE M, WASSERMANN B, et al. LS-ELAS: Line Segment Based Efficient Large-Scale Stereo Matching // Proc of the IEEE International Conference on Robotics and Automation. Washington, USA: IEEE. 2017: 146-152. [24] HAMZAH R A, IBRAHIM H, HASSAN A H A. Stereo Matching Algorithm Based on per Pixel Difference Adjustment, Iterative Guided Filter and Graph Segmentation. Journal of Visual Communication and Image Representation, 2017, 42: 145-160. [25] OLSSON C, ULÉN J, BOYKOV Y. In Defense of 3D-Label Stereo // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2013: 1730-1737. [26] 韩先君,刘艳丽,杨红雨.多元线性回归引导的立体匹配算法.计算机辅助设计与图形学学报, 2019, 31(1): 84-93. (HAN X J, LIU Y L, YANG H Y. A Stereo Matching Algorithm Guided by Multiple Linear Regression. Journal of Computer-Aided Design and Computer Graphics, 2019, 31(1): 84-93.) [27] HE K M, SUN J, TANG X O. Guided Image Filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(6): 1397-1409.
2020, Vol. 33 
