路端多源数据空间一致性数据集构建及评估方法研究

doi:10.16451/j.cnki.issn1003-6059.202311007

Abstract
Figure/Table
References (0)
Related Citation (1)

Download: PDF (6855 KB) HTML (1 KB)
Export: BibTeX | EndNote (RIS)

Abstract

The spatial consistency of multi-source sensing data is established as the foundation for the fusion of roadside multi-modal data, playing a crucial role in vehicle-to-infrastructure and roadside intelligence. However, existing roadside multi-modal datasets predominantly focus on recognition tasks such as object detection, lacking various spatial transformation information between multi-source sensors. This deficiency hinders the research into the spatial consistency problem of multi-source data at the roadside. Therefore, a dataset specifically designed for the study of the spatial consistency problem in roadside multi-source data is constructed in this paper-InfraCalib(https://github.com/chenzhiwei888/InfraCalib-Dataset). The dataset comprises over 230,000 frames of images and point cloud data, collected by two roadside smart mobile devices, covering diverse changes in scenes, modalities, lighting, device spatial positions and sensor postures. By matching feature key point pairs to correlate multi-modal data, a perspective-n-point(PnP) problem is constructed, and the extrinsic parameter matrix is solved using the minimum reprojection error method, serving as an approximate ground truth label. Finally, an experimental analysis of the classic feature matching algorithm is conducted on the InfraCalib dataset, and the discussion revolves around the quantitative evaluation indicators for the calibration of external parameters in multi-source sensors.

Key words： Multi-source Data Spatial Consistency Vehicle-to-Infrastructure Spatial Transformation Minimum Reprojection Error Evaluation Indicator

Received: 10 October 2023

ZTFLH:

TP181

Fund:

National Key Research & Development Program of China(No.2022YFB2502900), National Natural Science Foundation of China(No.62088102)

Corresponding Authors: CHEN Shitao, Ph.D.,assistant professor. His research interests include autonomous driving.

About author:: CHEN Zhiwei, master student.His research interests include intelligent transportation. ZHANG Haolin, Ph.D. candidate. His research interests include autonomous driving and vehicle-to-infrastructure. YAN Yuchen, Ph.D. candidate. His research interests include multi-sensors calibration and perception.

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	CHEN Zhiwei
	ZHANG Haolin
	YAN Yuchen
	CHEN Shitao

Cite this article:

CHEN Zhiwei,ZHANG Haolin,YAN Yuchen等. Construction of Roadside Multi-source Data Space Consistency Dataset and Research on Evaluation Methods[J]. Pattern Recognition and Artificial Intelligence, 2023, 36(11): 1041-1058.

URL:

http://manu46.magtech.com.cn/Jweb_prai/EN/10.16451/j.cnki.issn1003-6059.202311007 OR http://manu46.magtech.com.cn/Jweb_prai/EN/Y2023/V36/I11/1041

[1] CHEN C Y, SEFF A, KORNHAUSER A, et al. DeepDriving: Lear-ning Affordance for Direct Perception in Autonomous Driving // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2015: 2722-2730.
[2] XU H Z, GAO Y, YU F, et al. End-to-End Learning of Driving Models from Large-Scale Video Datasets // Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 3530-3538.
[3] BOJARSKI M, TESTA D D, DWORAKOWSKI D, et al. End to End Learning for Self-Driving Cars[C/OL]. [2023-09-16]. https://arxiv.org/pdf/1604.07316v1.pdf
[4] ONDRUŠKA P, POSNER I. Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks. Proceedings of the AAAI Confe-rence on Artificial Intelligence, 2016, 30(1): 3361-3367.
[5] CUI Y D, CHEN R, CHU W B, et al. Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review. IEEE Transactions on Intelligent Transportation Systems, 2021, 23(2): 722-739.
[6] LU N, CHENG N, ZHANG N, et al. Connected Vehicles: Solu-tions and Challenges. IEEE Internet of Things Journal, 2014, 1(4): 289-299.
[7] LUO R C, YIH C C, SU K L.Multisensor Fusion and Integration: Approaches, Applications, and Future Research Directions. IEEE Sensors journal, 2002, 2(2): 107-119.
[8] LOWE D G.Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision, 2004, 60: 91-110.
[9] RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: An Efficient Alternative to SIFT or SURF // Proc of the International Conference on Computer Vision. Washington, USA: IEEE, 2011: 2564-2571.
[10] BAY H, TUYTELAARS T, VAN GOOL L.SURF: Speeded up Robust Features // Proc of the 9th European Conference on Computer Vision. Berlin, Germany: Springer, 2006: 404-417.
[11] FENG M D, HU S X, ANG M H, et al. 2D3D-Matchnet: Lear-ning to Match Keypoints Across 2D Image and 3D Point Cloud // Proc of the International Conference on Robotics and Automation. Washington, USA: IEEE, 2019: 4790-4796.
[12] LOWE D G.Object Recognition from Local Scale-Invariant Features // Proc of the 7th IEEE International Conference on Compu-ter Vision. Washington, USA: IEEE, 1999. DOI: 10.1109/ICCV.1999.790410.
[13] ZHONG Y.Intrinsic Shape Signatures: A Shape Descriptor for 3D Object Recognition // Proc of the IEEE 12th International Confe-rence on Computer Vision Workshops. Washington, USA: IEEE, 2009: 689-696.
[14] LI J X, LEE G H.DeepI2P: Image-to-Point Cloud Registration via Deep Classification // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 15955-15964.
[15] BESL P J, MCKAY N D.A Method for Registration of 3-D Shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14(2): 239-256.
[16] BIBER P, STRAβER W. The Normal Distributions Transform: A New Approach to Laser Scan Matching // Proc of the IEEE/RSJ International Conference on Intelligent Robots and Systems. Washing-ton, USA: IEEE, 2003: 2743-2748.
[17] YANG J L, LI H D, JIA Y D.Go-ICP: Solving 3D Registration Efficiently and Globally Optimally // Proc of the IEEE Internatio-nal Conference on Computer Vision. Washington, USA: IEEE, 2013: 1457-1464.
[18] LU W X, WAN G W, ZHOU Y, et al. DeepVCP: An End-to-End Deep Neural Network for Point Cloud Registration // Proc of the IEEE/CVF International Conference on Computer Vision. Wa-shington, USA: IEEE, 2019: 12-21.
[19] YEW Z J, LEE G H.RPM-Net: Robust Point Matching Using Learned Features // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 11821-11830.
[20] LI J X, LEE G H.USIP: Unsupervised Stable Interest Point Detection from 3D Point Clouds // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 361-370.
[21] YEW Z J, LEE G H.3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 630-646.
[22] ZENG A, SONG S,NIEβNER M, et al. 3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 199-208.
[23] GOJCIC Z, ZHOU C F, WEGNER J D, et al. The Perfect Match: 3D Point Cloud Matching with Smoothed Densities // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 5540-5549.
[24] TANG D Q, HU T J, SHEN L C, et al. AprilTag Array-Aided Extrinsic Calibration of Camera-Laser Multi-sensor System. Robotics and Biomimetics, 2016. DOI: 10.1186/s40638-016-0044-0.
[25] DHALL A, CHELANI K, RADHAKRISHNAN V, et al. LiDAR-Camera Calibration Using 3D-3D Point Correspondences[C/OL].[2023-09-16]. https://arxiv.org/abs/1705.09785.
[26] LI X C, XIAO Y X, WANG B B, et al. Automatic Targetless LiDAR-Camera Calibration: A Survey. Artificial Intelligence Review, 2023, 56(9): 9949-9987.
[27] ZHANG Z.A Flexible New Technique for Camera Calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334.
[28] RICHARDSON A, STROM J, OLSON E.AprilCal: Assisted and Repeatable Camera Calibration // Proc of the IEEE/RSJ International Conference on Intelligent Robots and Systems. Washington, USA: IEEE, 2013: 1814-1821.
[29] YE X Q, SHU M, LI H Y, et al. Rope3D: The Roadside Perception Dataset for Autonomous Driving and Monocular 3D Object Detection Task // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 21309-21318.
[30] YU H B, LUO Y Z, SHU M, et al. DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection // Proc of the IEEE/CVF Conference on Computer Vision and Pa-ttern Recognition. Washington, USA: IEEE, 2022: 21329-21338.
[31] YU H B, YANG W X, RUAN H Z, et al. V2X-Seq: A Large-Scale Sequential Dataset for Vehicle-Infrastructure Cooperative Perception and Forecasting // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2023: 5486-5495.
[32] CREβ C, ZIMMER W, STRAND L, et al. A9-Dataset: Multisen-sor Infrastructure-Based Dataset for Mobility Research // Proc of the IEEE Intelligent Vehicles Symposium. Washington, USA: IEEE, 2022: 965-970.
[33] GEIGER A, LENZ P, URTASUN R.Are We Ready for Autonomous Driving? the KITTI Vision Benchmark Suite // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2012: 3354-3361.
[34] CAESAR H, BANKITI V, LANG A H, et al. nuScenes: A Multimodal Dataset for Autonomous Driving // Proc of the IEEE/CVF conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 11618-11628.
[35] MEI J R, ZHU A Z, YAN X C, et al. Waymo Open Dataset: Pa-noramic Video Panoptic Segmentation // Proc of the European Conference on Computer Vision. Berlin, Germany: Springer, 2022: 53-72.
[36] CHANG M F, LAMBERT J, SANGKLOY P, et al. Argoverse: 3D Tracking and Forecasting with Rich Maps // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 8740-8749.
[37] MAO J G, NIU M Z, JIANG C H, et al. One Million Scenes for Autonomous Driving: ONCE Dataset[C/OL].[2023-09-16]. https://arxiv.org/pdf/2106.11037.pdf.
[38] P?IBYL B, ZEM?ÍK P, ?ADÍK M. Absolute Pose Estimation from Line Correspondences Using Direct Linear Transformation. Computer Vision and Image Understanding, 2017, 161: 130-144.
[39] RANGANATHAN A. The Levenberg-Marquardt Algorithm[C/OL]. [2023-09-16]. http://www.ananth.in/docs/lmtut.pdf.
[40] GAO X S, HOU X R, TANG J L, et al. Complete Solution Classification for the Perspective-Three-Point Problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(8): 930-943.
[41] LEPETIT V, MORENO-NOGUER F, FUA P.EPnP: An Accurate O(n) Solution to the PnP Problem. International Journal of Computer Vision, 2009, 81: 155-166.
[42] PENATE-SANCHEZ A, ANDRADE-CETTO J, MORENO-NOGUER F.Exhaustive Linearization for Robust Camera Pose and Focal Length Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(10): 2387-2400.
[43] FISCHLER M A, BOLLES R C.Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Communications of the ACM, 1981, 24(6): 381-395.
[44] VISWANATHAN D G.Features from Accelerated Segment Test(FA-ST) [C/OL]. [2023-09-16].https://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/AV1011/AV1FeaturefromAcceleratedSegmentTest.pdf.
[45] CALONDER M, LEPETIT V, STRECHA C, et al. BRIEF: Binary Robust Independent Elementary Features // Proc of the 11th European Conference on Computer Vision. Berlin, Germany: Sprin-ger, 2010: 778-792.