1. College of Liangjiang Artificial Intelligence, Chongqing University of Technology, Chongqing 401135; 2. School of Computer Science and Technology, Harbin Institute of Technology(Shenzhen), Shenzhen 518055; 3. Shenzhen Key Laboratory of Visual Object Detection and Re-cognition, Harbin Institute of Technology(Shenzhen), Shen-zhen 518055
Abstract In natural scenes, the accuracy of fire detection is affected by weather conditions, light intensity and background interference. To achieve real-time accurate fire detection in complex scenarios, a real-time efficient fire detection method based on improved YOLOv5 is proposed. The proposed method is combined with Focal Loss, complete intersection over union loss function and multi-feature fusion to detect fires in real time. The focal loss function is introduced to alleviate the imbalance between positive and negative samples and make full use of the information of difficult samples. Meanwhile, combining the static and dynamic features of fires, a multi-feature fusion method is designed to eliminate false alarm fires. Aiming at the lack of fire datasets at home and abroad, a large-scale and high-quality fire dataset of 100 000 magnitude is constructed(http://www.yongxu.org/databases.html). Experiments show that the accuracy, speed, precision and generalization ability of the proposed method are significantly improved.
Fund:National Natural Science Foundation of China(No.61971078), Postdoctoral Innovative Talents Support Program of China(No.BX20190100)
Corresponding Authors:
XIAO Hanguang, Ph.D., professor. His research interests include image processing, machine learning, multi-mode perception and intelligent information processing.
About author:: ZHANG Dasheng, master student. His research interests include computer vision, image processing and object detection. WEN Jie, Ph.D., assistant professor. His research interests include machine learning and pattern recognition. XU Yong, Ph.D., professor. His research interests include pattern recognition, image processing, deep learning, biometric recognition and bio-informatics.
[1] CHEN J, HE Y P, WANG J. Multi-feature Fusion Based Fast Video Flame Detection. Building and Environment, 2010, 45(5): 1113-1122. [2] FOGGIA P, SAGGESE A, VENTO M. Real-Time Fire Detection for Video-Surveillance Applications Using a Combination of Experts Based on Color, Shape, and Motion. IEEE Transactions on Circuits and Systems for Video Technology, 2015, 25(9): 1545-1556. [3] PREMA C E, VINSLEY S S, SURESH S. Efficient Flame Detection Based on Static and Dynamic Texture Analysis in Forest Fire Detection. Fire Technology, 2018, 54(1): 255-288. [4] ZHU L P, LI H Q, WANG F H, et al. A Flame Detection Method Based on Novel Gradient Features. Journal of Intelligent Systems, 2018, 29(1): 773-786. [5] CHEN J Q, MU X H, SONG Y L, et al. Flame Recognition in Vi-deo Images with Color and Dynamic Features of Flames. Journal of Autonomous Intelligence, 2019, 2(1): 30-45. [6] ZHANG Q, LIU X J, HUANG L. Video Image Fire Recognition Based on Color Space and Moving Object Detection // Proc of the International Conference on Artificial Intelligence and Computer Engineering. Washington, USA: IEEE, 2020: 367-371. [7] 吴帅,徐勇,赵东宁.基于深度卷积网络的目标检测综述.模式识别与人工智能, 2018, 31(4): 335-346. (WU S, XU Y, ZHAO D N. Survey of Object Detection Based on Deep Convolutional Network. Pattern Recognition and Artificial Intelligence, 2018, 31(4): 335-346.) [8] JOSEPH K J, KHAN S, KHAN F S, et al. Towards Open World Object Detection // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 5826-5836. [9] MUHAMMAD K, AHMAD J, MEHMOOD I, et al. Convolutional Neural Networks Based Fire Detection in Surveillance Videos. IEEE Access, 2018, 6: 18174-18183. [10] CAI Y H, GUO Y J, LI Y Y, et al. Fire Detection Method Based on Improved Deep Convolution Neural Network // Proc of the 8th International Conference on Computing and Pattern Recognition. New York, USA: ACM, 2019: 466-470. [11] KIM B, LEE J. A Video-Based Fire Detection Using Deep Lear-ning Models. Applied Sciences, 2019, 9(14). DOI: 10.3390/app9142862. [12] REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. [13] SHEN Y K, TAN S, SORDONI A, et al. Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks[C/OL].[2022-01-15]. https://arxiv.org/pdf/1810.09536.pdf. [14] MUHAMMAD K, AHMAD J, LÜ Z H, et al. Efficient Deep CNN-Based Fire Detection and Localization in Video Surveillance Applications. IEEE Transactions on Systems, Man, and Cybernetics(Systems), 2019, 49(7): 1419-1434. [15] WANG Z Y, WEI D, HU X Q. Research on Two Stage Flame Detection Algorithm Based on Fire Feature and Machine Learning // Proc of the International Conference on Robotics, Intelligent Control and Artificial Intelligence. New York, USA: ACM, 2019: 574-578. [16] 白岩,徐泽堃,黄森,等.基于卷积神经网络室内火焰烟雾识别.计算机科学与应用, 2019, 9(6): 1183-1191. (BAI Y, XU Z K, HUANG S, et al. Indoor Flame Smoke Identification Based on Convolutional Neural Network. Computer Science and Application, 2019, 9(6): 1183-1191.) [17] CHAOXIA C Y, SHANG W W, ZHANG F. Information-Guided Flame Detection Based on Faster R-CNN. IEEE Access, 2020, 8: 58923-58932. [18] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation // Proc of the IEEE Conference on Computer Vision and Pa-ttern Recognition. Washington, USA: IEEE, 2014: 580-587. [19] CHOUEIRI S, DAOUD D, HARB S, et al. Fire and Smoke Detection Using Artificial Neural Networks // Proc of the 14th International Conference on Open Source Systems and Technologies. Washington, USA: IEEE, 2020. DOI: 10.1109/ICOSST51357.2020.9332990. [20] ZHANG L K, LU L. Video Flame Detection Method Based on Improved Fast Robust Feature // Proc of the 4th International Conference on Machine Vision and Information Technology. Bristol, UK: IOP, 2020. DOI: 10.1088/1742-6596/1518/1/012065. [21] SAPONARA S, ELHANASHI A, GAGLIARDI A. Real-Time Vi-deo Fire/Smoke Detection Based on CNN in Antifire Surveillance Systems. Journal of Real-Time Image Processing, 2021, 18: 889-900. [22] REDMON J, FARHADI A. YOLO9000: Better, Faster, Stronger // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 6517-6525. [23] SHAHID M, CHIEN I F, SARAPUGDI W,et al. Deep Spatial-Temporal Networks for Flame Detection. Multimedia Tools and App-lications, 2020, 81: 35297-35318. [24] 李欣健,张大胜,孙利雷,等.复杂场景下基于CNN的轻量火焰检测方法.模式识别与人工智能, 2021, 34(5): 415-422. (LI X J, ZHANG D S, SUN L L, et al. CNN-Based Lightweight Flame Detection Method in Complex Scenes. Pattern Recognition and Artificial Intelligence, 2021, 34(5): 415-422.) [25] XU R J, LIN H F, LU K J, et al. A Forest Fire Detection System Based on Ensemble Learning. Forests, 2021, 12(2). DOI: 10.3390/f12020217. [26] BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: Optimal Speed and Accuracy of Object Detection[C/OL].[2022-01-15]. https://arxiv.org/pdf/2004.10934.pdf. [27] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal Loss for Dense Object Detection // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2017: 2999-3007. [28] ZHENG Z H, WANG P, LIU W, et al. Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression. Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020, 34(7): 12993-13000. [29] CHEN T H, WU P H, CHIOU Y C.An Early Fire-Detection Me-thod Based on Image Processing // Proc of the International Conference on Image Processing. Washington, USA: IEEE, 2004, III: 1707-1710. [30] 戴鑫,黄愐,张进.帧间差法在视频目标检测的仿真应用.计算机与多媒体技术, 2021(21): 110-111. (DAI X, HUANG M, ZHANG J. Simulation Application Based on Frame Difference Method in Video Object Detection. Computer and Multimedia Technology, 2021(21): 110-111.) [31] KO B C, HAM S J, NAM J Y. Modeling and Formalization of Fuzzy Finite Automata for Detection of Irregular Fire Flames. IEEE Transactions on Circuits and Systems for Video Technology, 2011, 21(12): 1903-1912. [32] CHINO D Y T, AVALHAIS L P S, RODRIGUES J F, et al. BoWFire: Detection of Fire in Still Images by Integrating Pixel Co-lor and Texture Analysis // Proc of the 28th SIBGRAPI Conference on Graphics, Patterns and Images. Washington, USA: IEEE, 2015: 95-102.
2022, Vol. 35 
