基于控制点和RGB向量差联合梯度Census变换的立体匹配算法

doi:10.16451/j.cnki.issn1003-6059.202201004

摘要
图/表
参考文献
相关文章 (8)

全文: PDF (5574 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要为了解决立体匹配算法在弱纹理区域及边界区域匹配精度较低的问题,文中提出基于控制点和RGB向量差联合梯度Census变换的立体匹配算法.首先,使用基于动态时间归整的行匹配算法,寻找最优匹配路径,经过扭曲对齐选取匹配特征点作为控制点.再使用基于RGB向量差代价联合基于梯度Census变换代价作为非控制点的匹配代价,基于梯度Census变换增加像素点的鲁棒性,RGB三向量保留图像三维色彩信息,使求得的代价精度更高.然后,融合求得的控制点与非控制点的匹配代价,作为初始匹配代价,基于RGB三向量差求得针对不同纹理区域的自适应窗口,在该窗口内以一次水平方向和竖直方向代价聚合的方式进行初始代价优化,同时使用多步骤优化减少视差错误率.最后,在Middlebury数据集上对不同区域测试视差错误率和在真实机器人场景上求取视差,将求得的视差利用三维成像原理进行三维重建测试.理论分析和实验表明,文中算法明显降低弱纹理区域和边界的误匹配率.

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	王森
	危辉
	孟令江

关键词 ：控制点, 动态时间归整(DTW), 梯度Census变换, 立体匹配

Abstract：Aiming at the low matching precision of weak texture region and boundary in traditional stereo matching algorithms, a stereo matching algorithm based on control points, RGB vector difference and gradient census transform is proposed. Firstly, a row matching algorithm based on dynamic time warping(DTW) is exploited to find the optimal matching path. The path is distorted and aligned to select matching control points. Then, RGB vector difference cost is combined with gradient Census transform cost as the matching cost of non-control points. The robustness of pixels is enhanced by gradient Census transform, and the three-dimensional color information of the image is retained by the RGB vector. Thus the higher matching precision is achieved. The combination of the matching cost of control points and non-control points is regarded as the initial matching cost. The RGB vector difference is utilized to acquire the adaptive window of different texture areas. The cost aggregation is calculated by the algorithm in a horizontal and vertical directions of an adaptive window. The cost aggregation is employed to optimize the initial cost. Multi-step optimization is employed to reduce the disparity error rate.Finally, the proposed algorithm is validated on Middlebury dataset for different regions and the disparity is calculated in the real scene of robot. The calculated disparity is verified for three-dimensional reconstruction based on the principle of 3D imaging. Both theoretical analysis and experimental results show that the proposed algorithm reduces the matching error rate of weak texture region and boundary significantly.

Key words： Control Points Dynamic Time Warping(DTW) Gradient Census Transform Stereo Matching

收稿日期: 2021-06-07

基金资助:国家自然科学基金项目(No.61771146)资助

通讯作者: 危辉,博士,教授,主要研究方向为认知科学、人工智能.E-mail:weihui@fudan.edu.cn.

作者简介: 王森,硕士研究生,主要研究方向为立体视觉.E-mail:19212010029@fudan.edu.cn.
孟令江,博士研究生,主要研究方向为立体视觉.E-mail: ljmeng17@fudan.edu.cn.

引用本文:

王森, 危辉, 孟令江. 基于控制点和RGB向量差联合梯度Census变换的立体匹配算法[J]. 模式识别与人工智能, 2022, 35(1): 37-50. WANG Sen, WEI Hui, MENG Lingjiang. Stereo Matching Algorithm Based on Control Points, RGB Vector Difference and Gradient Census Transform. Pattern Recognition and Artificial Intelligence, 2022, 35(1): 37-50.

链接本文:

http://manu46.magtech.com.cn/Jweb_prai/CN/10.16451/j.cnki.issn1003-6059.202201004 或 http://manu46.magtech.com.cn/Jweb_prai/CN/Y2022/V35/I1/37

[1] WEI H, XU C, JIN Z F.Binocular Matching Model Based on Hie-rarchical V1 and V2 Receptive Fields with Color, Orientation, and Region Feature Information. IEEE Transactions on Biomedical Engineering, 2020, 67(11): 3141-3150.
[2] GEIGER A, LENZ P, URTASUN R.Are We Ready for Autonomous Driving? the KITTI Vision Benchmark Suite // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2012: 3354-3361.
[3] HAMZAH R A, KADIN A F, HAMID M S, et al. Improvement of Stereo Matching Algorithm for 3D Surface Reconstruction. Signal Processing: Image Communication, 2018, 65: 165-172.
[4] ZHAI G D, ZHANG W T, HU W Y, et al. Coal Mine Rescue Robots Based on Binocular Vision: A Review of the State of the Art. IEEE Access, 2020, 8: 130561-130575.
[5] TSAI P, CHONDRO P, RUAN S J.Depth-Guided Pixel Dimming with Saliency-Oriented Power-Saving Transformation for Stereoscope AMOLED Displays. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 30(9): 3095-3105.
[6] CHONDRO P, YAO Z R, RUAN S J.Depth-Based Dynamic Lightness Adjustment Power-Saving Algorithm for AMOLED in Head-Mounted Display. Optics Express, 2018, 26(25): 33158-33165.
[7] LIU P P, KING I, LYU M R, et al. Flow2Stereo: Effective Self-Supervised Learning of Optical Flow and Stereo Matching // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 6647-6656.
[8] HAN X F, LEUNG T, JIA Y Q, et al. MatchNet: Unifying Feature and Metric Learning for Patch-Based Matching // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2015: 3279-3286.
[9] KENDALL A, MARTIROSYAN H, DASGUPTA S, et al. End-to-End Learning of Geometry and Context for Deep Stereo Regression // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2017: 66-75.
[10] MENZE M, GEIGER A.Object Scene Flow for Autonomous Vehicles // Proc of the IEEE Conference on Computer Vision and Pa-ttern Recognition. Washington, USA: IEEE, 2015: 3061-3070.
[11] SEKI A, POLLEFEYS M. Patch Based Confidence Prediction for Dense Disparity Map[C/OL]. [2021-05-26]. http://www.bmva.org/bmvc/2016/papers/paper023/paper023.pdf.
[12] CHENG X L, ZHONG Y R, HARANDI M, et al. Hierarchical Neural Architecture Search for Deep Stereo Matching[C/OL].[2021-05-26]. https://arxiv.org/pdf/2010.13501v1.pdf.
[13] SCHARSTEIN D, SZELISKI R.A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms. International Journal of Computer Vision, 2002, 47(1): 7-42.
[14] VANNE J, AHO E, HAMALAINEN T D, et al. A High-Perfor-mance Sum of Absolute Difference Implementation for Motion Estimation. IEEE Transactions on Circuits and Systems for Video Technology, 2006, 16(7): 876-883.
[15] LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single Shot Mul-tibox Detector // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 21-37.
[16] HIRSCHMÜLLER H, INNOCENT P R, GARIBALDI J. Real-Time Correlation-Based Stereo Vision with Reduced Border Errors. International Journal of Computer Vision, 2002, 47(1): 229-246.
[17] YOON K J, KWEON I S.Adaptive Support-Weight Approach for Correspondence Search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(4): 650-656.
[18] LEE J, JUN D, EEM C, et al. Improved Census Transform for Noise Robust Stereo Matching. Optical Engineering, 2016, 55(6). DOI: 10.1117/1.OE.55.6.063107.
[19] MA L, LI J J, MA J, et al. A Modified Census Transform Based on the Neighborhood Information for Stereo Matching Algorithm // Proc of the 7th International Conference on Image and Graphics. New York, USA: ACM, 2013: 533-538.
[20] LÜ C, LI J H, KOU Q Q, et al. Stereo Matching Algorithm Based on HSV Color Space and Improved Census Transform. Mathematical Problems in Engineering, 2021. DOI: 10.1155/2021/1857327.
[21] MÜLLER M. Dynamic Time Warping // MÜLLER M, ed. Information Retrieval for Music and Motion. Berlin, Germany: Sprin-ger, 2007: 69-84.
[22] HIRSCHMULLER H.Stereo Processing by Semiglobal Matching and Mutual Information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(2): 328-341.
[23] 张大禹,黄灿.基于WTA立体匹配算法的人体检测方法研究.电子测试, 2013(5): 66-68.
(ZHANG D Y, HUANG C.The Algorithm of WTA Stereo Matching Based on the Human Detection Method. Electronic Test, 2013(5): 66-68.)
[24] BOBICK A F, INTILLE S S. Large Occlusion Stereo. International Journal of Computer Vision, 1999, 33(3): 181-200.
[25] ZHANG K, LU J B, LAFRUIT G.Cross-Based Local Stereo Mat-ching Using Orthogonal Integral Images. IEEE Transactions on Circuits and Systems for Video Technology, 2009, 19(7): 1073-1079.
[26] MEI X, SUN X, ZHOU M C, et al. On Building an Accurate Ste-reo Matching System on Graphics Hardware // Proc of the IEEE International Conference on Computer Vision Workshops. Washington, USA: IEEE, 2011: 467-474.
[27] YANG Q X, WANG L, YANG R G, et al. Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation, and Occlusion Handling // Proc of the IEEE Computer Society Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2006: 2347-2354.
[28] SCHARSTEIN D, SZELISKI R, ZABIH R.A Taxonomy and Eva-luation of Dense Two-Frame Stereo Correspondence Algorithms // Proc of the IEEE Workshop on Stereo and Multi-baseline Vision. Washington, USA: IEEE, 2001: 131-140.
[29] SCHARSTEIN D, SZELISKI R.High-Accuracy Stereo Depth Maps Using Structured Light // Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2003: 195-202.
[30] YANG Q X.A Non-local Cost Aggregation Method for Stereo Mat-ching // Proc of the IEEE Conference on Computer Vision and Pa-ttern Recognition. Washington, USA: IEEE, 2012: 1402-1409.
[31] FACCIOLO G, DE FRANCHIS C, MEINHARDT E. MGM: A Significantly More Global Matching for Stereovision[C/OL]. [2021-05-26]. http://www.bmva.org/bmvc/2015/papers/paper090/paper090.pdf.
[32] ZHU S P, YAN L N.Local Stereo Matching Algorithm with Efficient Matching Cost and Adaptive Guided Image Filter. The Visual Computer, 2017, 33: 1087-1102.
[33] VEKSLER O. Stereo Correspondence by Dynamic Programming on a Tree // Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2005, II: 384-390.
[34] VANETTI M, GALLO I, BINAGHI E.Dense Two-Frame Stereo Correspondence by Self-Organizing Neural Network // Proc of the 15th International Conference on Image Analysis and Processing. Berlin, Germany: Springer, 2009: 1035-1042.
[35] MEI X, SUN X, DONG W M, et al. Segment-Tree Based Cost Aggregation for Stereo Matching // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2013: 313-320.