Abstract The existing lane detection neural networks mainly adopt independent single frame image for detection, and therefore they cannot handle the complex and practical application scenarios, such as short-term occlusion of lane and light and shade changes of ground. To solve these problems, a parallel lane detection network based on image sequence is proposed according to the scene characteristics that the continuous images can be obtained in the normal driving process of vehicles. Firstly, a parallel feature extraction structure is designed. A single frame network with high accuracy is employed to extract the features of the current frame image. A lightweight multi-frame network is designed to extract the features of low resolution multi-frame sequential images. The cyclic neural network module is utilized to fuse the extracted sequential features to obtain multi-frame features. Then, the fusion module of single frame feature and multi-frame feature is designed, and the feature map of the lane line is output through upsampling network. The experimental results show that the objective detection accuracy and subjective effect of the proposed network are significantly improved.
Fund:Supported by Natural Science Foundation of Zhejiang Province(No.Y21F010051), National Natural Science Foundation of China(No.61773345), Open Fund of State Key Laboratory of Automotive Simulation and Control(No.20171103)
Corresponding Authors:
ZHU Wei, Ph.D., associate professor. His research interests include machine vision.
About author:: OU Quanlin, master student. His research interests include intelligent visual proce-ssing.HONG Lidong, master student. His research interests include intelligent robot system.HE Defeng, Ph.D., professor. His research interests include intelligent driving and safety control.
[1] BADRINARAYANAN V,KENDALL A,CIPOLLA R.SegNet:A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation.IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(12):2481-2495. [2] RONNEBERGER O,FISCHER P,BROX T.U-Net:Convolutional Networks for Biomedical Image segmentation//Proc of the International Conference on Medical Image Computing and Computer-Assisted Intervention.Berlin,Germany:Springer,2015:234-241. [3] NEVEN D,DE BRABANDERE B,GEORGOULIS S,et al.Towards End-to-End Lane Detection:An Instance Segmentation Approach//Proc of the IEEE Intelligent Vehicles Symposium.Washington,USA:IEEE,2018:286-291. [4] LEE S,KIM J,YOON J S,et al.VPGRET:Vanishing Point Guided Network for Lane and Road Marking Detection and Recognition//Proc of the IEEE International Conference on Computer Vision.Washington,USA:IEEE,2017:1965-1973. [5] PAN X G,SHI J P,LUO P,et al.Spatial as Deep:Spatial CNN for Traffic Scene Understanding//Proc of the 32nd AAAI Confe-rence on Artificial Intelligence.Palo Alto,USA:AAAI Press,2018:7276-7283. [6] GARNETT N,COHEN R,PE′ER T,et al.3D-LaneNet:End-to-End 3D Multiple Lane Detection//Proc of the IEEE/CVF International Conference on Computer Vision.Washington,USA:IEEE,2019:2921-2930. [7] ZOU Q,JIANG H W,DAI Q Y,et al.Robust Lane Detection From Continuous Driving Scenes Using Deep Neural Networks.IEEE Transactions on Vehicular Technology,2019,69(1):41-54. [8] SHI X J,CHEN Z R,WANG H,et al. Convolutional LSTM Network:A Machine Learning Approach for Precipitation Nowcasting//Proc of the 28th International Conference on Neural Information Processing Systems.Cambridge,USA:The MIT Press,2015:802-810. [9] GERS F A,SCHMIDHUBER J,CUMMINS F.Learning to Forget:Continual Prediction with LSTM.Neural Computation,2000,12(10):2451-2471. [10] HE K M,ZHANG X Y,REN S Q,et al.Deep Residual Learning for Image Recognition//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2016:770-778. [11] MEHTA S,RASTEGARI M,SHAPIRO L,et al. ESPNetv2:A Light-Weight,Power Efficient,and General Purpose Convolutional Neural Network//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2019:9190-9200. [12] ZHAO H S,QI X J,SHEN X Y,et al.ICNet for Real-Time Semantic Segmentation on High-Resolution Images//Proc of the European Conference on Computer Vision.Berlin,Germany:Springer,2018:405-420. [13] ROMERA E,ÁLVAREZ J M,BERGASA L M,et al. ERFNet:Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation.IEEE Transactions on Intelligent Transportation Systems,2018,19(1):263-272. [14] SANDLER M,HOWARD A,ZHU M L,et al. MobileNetV2:Inverted Residuals and Linear Bottlenecks//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2018:4510-4520. [15] MEHTA S,RASTEGARI M,CASPI A,et al. ESpnet:Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation//Proc of the European Conference on Computer Vision.Berlin,Germany:Springer,2018:561-580. [16] MA N N,ZHANG X Y,ZHENG H T,et al.ShuffleNetV2:Practical Guidelines for Efficient CNN Architecture Design//Proc of the European Conference on Computer Vision.Berlin,Germany:Springer,2018:122-138. [17] CHOLLET F.Xception:Deep Learning with Depthwise Separable Convolutions//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2017:1800-1807. [18] ZHAO H S,SHI J P,QI X J,et al.Pyramid Scene Parsing Network//Proc of the IEEE Conference on Computer Vision and Pa-ttern Recognition.Washington,USA:IEEE,2017:6230-6239. [19] YU F,KOLTUN V.Multi-scale Context Aggregation by Dilated Convolutions[C/OL].[2020-08-05].https://arxiv.org/pdf/1511.07122.pdf. [20] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[C/OL].[2020-08-05].https://arxiv.org/pdf/1409.1556.pdf. [21] SZEGEDY C,LIU W,JIA Y Q,et al.Going Deeper with Convolutions//Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2015.DOI:10.1109/CVPR.2015.7298594. [22] GLOROT X,BORDES A,BENGIO Y.Deep Sparse Rectifier Neu-ral Networks//Proc of the 14th International Conference on Artificial Intelligence and Statistics.New York,USA:ACM,2011:315-323. [23] HE K M,ZHANG X Y,REN S Q,et al.Delving Deep into Rectifiers:Surpassing Human-Level Performance on ImageNet Classification//Proc of the IEEE International Conference on Computer Vision.Washington,USA:IEEE,2015:1026-1034.
2021, Vol. 34 
