模式识别与人工智能
Sunday, Apr. 13, 2025 Home      About Journal      Editorial Board      Instructions      Ethics Statement      Contact Us                   中文
Pattern Recognition and Artificial Intelligence  2023, Vol. 36 Issue (11): 1029-1040    DOI: 10.16451/j.cnki.issn1003-6059.202311006
Current Issue| Next Issue| Archive| Adv Search |
Spatial-Channel Attention Multi-sensor Fusion Based on Bird's-Eye View
JI Yuzhe1, CHEN Yijie2, YANG Liuqing1,2, ZHENG Xinhu2
1. Internet of Things Thrust, The Hong Kong University of Science and Technology(Guangzhou), Guangzhou 511455;
2. Intelligent Transportation Thrust, The Hong Kong University of Science and Technology(Guangzhou), Guangzhou 511455

Download: PDF (1983 KB)   HTML (1 KB) 
Export: BibTeX | EndNote (RIS)      
Abstract  

Object perception based on bird's-eye view(BEV) is one of hot issues, but studies on multi-sensor fusion for BEV are still insufficient. Therefore, a multi-sensor fusion module based on spatial-channel attention is proposed. Spatial errors between multiple sensors can be effectively corrected by adding local attention mechanisms to features of different modalities. By using transpose attention operations, the image and point cloud data are fully integrated to resolve the heterogeneity between different modal semantics. Consequently, the fused BEV features achieves more comprehensive and accurate perception by effectively combining the unique information of each sensor without introducing spatial misalignment. Experiment on nuScenes dataset and extensive ablation experiments show that the proposed fusion module effectively improves the accuracy of object detection. Visualization results demonstrate that the fused features can capture more complete and accurate information, especially in distant objects detection.

Key wordsBird's-Eye View(BEV)      Multi-sensor Fusion      Attention Mechanism      Object Detection     
Received: 18 October 2023     
ZTFLH: TP391.41  
Fund:

General Program of National Natural Science Foundation of China(No.62373315), Science and Technology Projects of Guangzhou(No.2023A03J0683,2023A03J0011)

About author:: JI Yuzhe, Ph.D, candidate. His research interests include multimodal fusion perception and multi-vehicle cooperative perception. CHEN Yijie, master student. His research interests include multi-agent cooperative perception and vehicle-road cooperative systems. YANG Liuqing, Ph.D., professor. Her research interests include wireless communication networks, multi-agent systems and integrated communication and sensing. ZHENG Xinhu, Ph.D., assistant profe-ssor. His research interests include multi-modal perception, multi-agent cooperative perception and networked intelligence.
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
JI Yuzhe
CHEN Yijie
YANG Liuqing
ZHENG Xinhu
Cite this article:   
JI Yuzhe,CHEN Yijie,YANG Liuqing等. Spatial-Channel Attention Multi-sensor Fusion Based on Bird's-Eye View[J]. Pattern Recognition and Artificial Intelligence, 2023, 36(11): 1029-1040.
URL:  
http://manu46.magtech.com.cn/Jweb_prai/EN/10.16451/j.cnki.issn1003-6059.202311006      OR     http://manu46.magtech.com.cn/Jweb_prai/EN/Y2023/V36/I11/1029
Copyright © 2010 Editorial Office of Pattern Recognition and Artificial Intelligence
Address: No.350 Shushanhu Road, Hefei, Anhui Province, P.R. China Tel: 0551-65591176 Fax:0551-65591176 Email: bjb@iim.ac.cn
Supported by Beijing Magtech  Email:support@magtech.com.cn