Multi-modality Sensing Aided Beam Prediction for mmWave V2V Communications
WEN Weibo1, ZHANG Haotian1, GAO Shijian2, CHENG Xiang1, YANG Liuqing3,4,5
1. School of Electronics, Peking University, Beijing 100871; 2. Samsung Semiconductor, Samsung SoC Research and Deve-lopment Lab, San Diego, CA 92121, USA; 3. Intelligent Transportation Thrust, The Hong Kong University of Science and Technology(Guangzhou), Guangzhou 511455; 4. Internet of Things Thrust, The Hong Kong University of Science and Technology(Guangzhou), Guangzhou 511455; 5. Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong 999077
Abstract To ensure the transmission reliability of vehicular communication network, precisely aligned beamforming of millimeter-wave communication using massive multi-input multi-output(mMIMO) technology is urgently required. In highly dynamic vehicular communication scenarios, traditional beam alignment schemes incur significant resource overhead and struggle to establish reliable links within the coherence time. To address this critical challenge, a scheme of multi-modality sensing aided beam prediction for mmWave V2V communications is proposed. Two non-RF sensing modalities, vision and ranging(LiDAR) point cloud, are integrated, and deep neural networks are employed for feature extraction and integration of multi-modal information. Accurate matching and deep fusion of image space semantic information and physical space location information are achieved through perspective projection. By collaborative sensing coordinate calibration and vehicle position prediction, the features of physical environment are accurately mapped to the angular-domain channel, enabling real-time and precise beam prediction. The experimental results on the mixed multi-modal sensing-communication dataset(M3SC)show that the proposed scheme achieves high angle tracking accuracy and achievable communication rate.
Fund:National Key Research and Development Program of China(No.2020AAA0108101), National Natural Science Foundation of China(No.62125101,62341101,62001018,62301011,U23A20339), New Cornerstone Science Foundation through the XPLORER PRIZE, Municipal Science and Technology Project of Guangzhou(No.2023A03J0011), Guangdong Provincial Department of Education Major Research Project(No.2023ZDZX1037)
Corresponding Authors:GAO Shijian, Ph.D.,senior engineer. His research interests include wireless communications and statistical signal processing.
About author:: WEN Weibo, Ph.D. candidate. His research interests include multi-modal sensing-assisted transceiver design in communication systems. ZHANG Haotian, Ph.D. candidate. His research interests include multi-modal sen-sing-assisted transceiver design in communication systems. CHENG Xiang, Ph.D., professor. His research interests include data-driven intelligent network and networked intelligence. YANG Liuqing, Ph.D., professor. Her research interests include wireless communication networks, multi-agent systems and integrated communication and sensing.
[1] CHENG X, GAO S J, YANG L Q. mmWave Massive MIMO Vehi-cular Communications. Berlin, Germany: Springer, 2023. [2] 任开明,李纪舟,刘玲艳,等.车联网通信技术发展现状及趋势研究.通信技术, 2015, 48(5): 507-513. (REN K M, LI J Z, LIU L Y, et al. Development Status and Ten-dency of IoV Communication Technology. Communications Techno-logy, 2015, 48(5): 507-513.) [3] XIAO Z Y, HE T, XIA P F, et al. Hierarchical Codebook Design for Beamforming Training in Millimeter-Wave Communication. IEEE Transactions on Wireless Communications, 2016, 15(5): 3380-3392. [4] ELTAYEB M E, ALKHATEEB A, HEATH R W, et al. Opportu-nistic Beam Training with Hybrid Analog/Digital Codebooks for mmWave Systems // Proc of the IEEE Global Conference on Signal and Information Processing. Washington, USA: IEEE, 2015: 315-319. [5] GAO S J, CHENG X, FANG L Y, et al. Model Enhanced Learning Based Detectors(Me-LeaD) for Wideband Multi-user 1-bit mmWave Communications. IEEE Transactions on Wireless Communications, 2021, 20(7): 4646-4656. [6] MA W Y, QI C H, ZHANG Z C, et al. Sparse Channel Estimation and Hybrid Precoding Using Deep Learning for Millimeter Wave Massive MIMO. IEEE Transactions on Communications, 2020, 68(5): 2838-2849. [7] GAO S J, CHENG X, YANG L Q.Estimating Doubly-Selective Channels for Hybrid mmWave Massive MIMO Systems: A Doubly-Sparse Approach. IEEE Transactions on Wireless Communications, 2020, 19(9): 5703-5715. [8] SHEN W Q, BU X Y, GAO X Y, et al. Beamspace Precoding and Beam Selection for Wideband Millimeter-Wave MIMO Relying on Lens Antenna Arrays. IEEE Transactions on Signal Processing, 2019, 67(24): 6301-6313. [9] VA V, VIKALO H, HEATH R W.Beam Tracking for Mobile Millimeter Wave Communication Systems // Proc of the IEEE Global Conference on Signal and Information Processing. Washington, USA: IEEE, 2016: 743-747. [10] LIU F, YUAN W J, MASOUROS C, et al. Radar-Assisted Predictive Beamforming for Vehicular Links: Communication Served by Sensing. IEEE Transactions on Wireless Communications, 2020, 19(11): 7704-7719. [11] LAREW S G, LOVE D J.Adaptive Beam Tracking with the Unscented Kalman Filter for Millimeter Wave Communication. IEEE Signal Processing Letters, 2019, 26(11): 1658-1662. [12] ZHANG Q X, JI K J, FENG Z Y, et al. Vehicle Behavior-Cogni-tion-Based Particle-Filter-Enabled mmWave Beam Tracking for Co-nnected Automated Vehicles. IEEE Internet of Things Journal, 2022, 9(21): 21292-21304. [13] ZHU D L, CHOI J, CHENG Q, et al. High-Resolution Angle Tracking for Mobile Wideband Millimeter-Wave Systems with Antenna Array Calibration. IEEE Transactions on Wireless Communications, 2018, 17(11): 7173-7189. [14] CHENG X, DUAN D L, GAO S J, et al. Integrated Sensing and Communications(ISAC) for Vehicular Communication Networks(VCN). IEEE Internet of Things Journal, 2022, 9(23): 23441-23451. [15] CHENG X, ZHANG H T, ZHANG J N, et al. Intelligent Multi-Modal Sensing-Communication Integration: Synesthesia of Machines. IEEE Communications Surveys and Tutorials, 2023. DOI: 10.1109/COMST.2023.3336917. [16] ALRABEIAH M, HREDZAK A, ALKHATEEB A.Millimeter Wave Base Stations with Cameras: Vision-Aided Beam and Blockage Prediction // Proc of the IEEE 91st Vehicular Technology Confe-rence. Washington, USA: IEEE, 2020. DOI: 10.1109/VTC2020-Spring48590.2020.9129369. [17] DEMIRHAN U, ALKHATEEB A.Radar Aided 6G Beam Prediction: Deep Learning Algorithms and Real-World Demonstration // Proc of the IEEE Wireless Communications and Networking Con-ference. Washington, USA: IEEE, 2022: 2655-2660. [18] KLAUTAU A, GONZÁLEZ-PRELCIC N, HEATH R W. LIDAR Data for Deep Learning-Based mmWave Beam-Selection. IEEE Wireless Communications Letters, 2019, 8(3): 909-912. [19] WU S Y, CHAKRABARTI C, ALKHATEEB A.LiDAR-Aided Mo-bile Blockage Prediction in Real-World Millimeter Wave Systems // Proc of the IEEE Wireless Communications and Networking Conference. Washington, USA: IEEE, 2022: 2631-2636. [20] CHARAN G, ALRABEIAH M, ALKHATEEB A.Vision-Aided 6G Wireless Communications: Blockage Prediction and Proactive Handoff. IEEE Transactions on Vehicular Technology, 2021, 70(10): 10193-10208. [21] CHEN S Z, LIANG Y C, SUN S H, et al. Vision, Requirements, and Technology Trend of 6G: How to Tackle the Challenges of System Coverage, Capacity, User Data-Rate and Movement Speed. IEEE Wireless Communications, 2020, 27(2): 218-228 [22] FAN Y J, GAO S J, DUAN D L, et al. Radar Integrated MIMO Communications for Multi-hop V2V Networking. IEEE Wireless Communications Letters, 2023, 12(2): 307-311. [23] ZHANG H T, GAO S J, CHENG X, et al. Integrated Sensing and Communications towards Proactive Beamforming in mmWave V2I via Multi-modal Feature Fusion(MMFF)[C/OL].[2023-09-22]. https://arxiv.org/abs/2310.02561. [24] 3GPP. Study on enhancement of 3GPP Support for 5G V2X Services[EB/OL].[2023-09-22]. https://www.3gpp.org/dynateport/22886.htm. [25] 陈山枝,时岩,胡金玲.蜂窝车联网(C-V2X)综述.中国科学基金, 2020, 34(2): 179-185. (CHEN S Z, SHI Y, HU J L.Cellular Vehicle to Everything(C-V2X): A Review. Bulletin of National Natural Science Foundation of China, 2020, 34(2): 179-185.) [26] LUO H, DEMIRHAN U, ALKHATEEB A.Millimeter Wave V2V Beam Tracking Using Radar: Algorithms and Real-World Demonstration[C/OL]. [2023-09-22].https://arxiv.org/pdf/2308.01558.pdf. [27] OSMAN T, CHARAN G, ALKHATEEB A.Vehicle Cameras Guide mmWave Beams: Approach and Real-World V2V Demonstration[C/OL]. [2023-09-22].https://arxiv.org/pdf/2308.10362.pdf. [28] REDMON J, DIVVALA S, GIRSHICK R, et al. You Only Look Once: Unified, Real-Time Object Detection // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 779-788. [29] CHENG X, HUANG Z W, BAI L, et al. M3SC: A Generic Dataset for Mixed Multi-modal(MMM) Sensing and Communication Integration. China Communications, 2023, 20(11): 13-29. [30] FAN Y J, GAO S J, CHENG X, et al. Wideband Generalized Beamspace Modulation(wGBM) for mmWave Massive MIMO over Doubly-Selective Channels. IEEE Transactions on Vehicular Technology, 2021, 70(7): 6869-6880.
2023, Vol. 36 
