Multi-stage Perception Guided Adaptive Ensemble for Concept Drift
GUO Husheng1,2,3, WANG Yao1, ZHANG Xu1, ZHENG Wendong1, XU Hang1,2, WANG Wenjian2,3
1. School of Computer and Information Technology, Shanxi University, Taiyuan 030006; 2. Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, Shanxi University, Taiyuan 030006; 3. Key Laboratory of Data Intelligence and Cognitive Computing of Shanxi Province, Shanxi University, Taiyuan 030006
Abstract：Concept drift is a critical factor that limits the performance improvement of models in data stream mining tasks in open environments. Due to the continuous evolution of data distributions in data streams, the discrepancies between historical and current data distributions result in difficulties in fully exploiting historical knowledge, thereby undermining model stability and adaptability to drifting environments. To address this issue, a multi-stage perception guided adaptive ensemble for concept drift(MPAE) is proposed. Temporal profiling based on model performance over consecutive data blocks is performed, and the key historical data consistent with the current distribution are selected accordingly. Loss sequences and statistical significance testing are further combined to adaptively filter current data. Dynamic sample reconstruction is conducted based on temporal perception. In addition, a dynamic ensemble stopping criterion is designed based on the loss decay rate, allowing drift-perception adjustment of the ensemble process. According to the estimated drift intensity, an adaptive ensemble model is constructed to achieve effective coordination between the data and the model. Experiments demonstrate that MPAE exploits the historical data effectively, improves the drift adaptability of the model, and achieves favorable generalization performance.
[1] Read J, Zliobaite I. Supervised learning from data streams: an over-view and update[J/OL]. ACM Computing Surveys, 2025, 57(12). https://doi.org/10.1145/373727. [2] Cacciarelli D, Kulahci M.Active learning for data streams: a survey[J]. Machine Learning, 2024, 113(1): 185-239. [3] Karimian M, Beigy H.Concept drift handling: a domain adaptation perspective[J/OL]. Expert Systems with Applications, 2023, 224. https://doi.org/10.1016/j.eswa.2023.119946. [4] 赵鹏,周志华.基于决策树模型重用的分布变化流数据学习[J]. 中国科学(信息科学), 2021, 51(1): 1-12. (Zhao P, Zhou Z H.Learning from distribution-changing data streams via decision tree model reuse[J]. Scientia Sinica Informationis, 2021, 51(1): 1-12.) [5] Xiang Q Y, Zi L L, Cong X, et al. Concept drift adaptation me-thods under the deep learning framework: a literature review[J/OL]. Applied Sciences Concept drift adaptation me-thods under the deep learning framework: a literature review[J/OL]. Applied Sciences, 2023, 13(11). https://doi.org/10.3390/app13116515. [6] Hoi S C H, Sahoo D, Lu J, et al. Online learning: a comprehensive survey[J]. Neurocomputing, 2021, 459: 249-289. [7] Lu Y, Cheung Y M, Tang Y Y.Adaptive chunk-based dynamic weighted majority for imbalanced data streams with concept drift[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(8): 2764-2778. [8] Weinberg A I, Last M.EnHAT-synergy of a tree-based ensemble with Hoeffding adaptive tree for dynamic data streams mining[J]. Information Fusion, 2023, 89: 397-404. [9] Li Z, HUANG W C, XIONG Y,et al. Incremental learning im-balanced data streams with concept drift: the dynamic updated ensemble algorithm[J/OL]. Knowledge-Based Systems, 2020, 195. https://doi.org/10.1016/j.knosys.2020.105694. [10] Woźniak M, Zyblewski P, Ksieniewicz P.Active weighted aging ensemble for drifted data stream classification[J]. Information Sciences, 2023, 630: 286-304. [11] Guo H S, Zhang Y, Wang W J.Dynamical targeted ensemble learning for streaming data with concept drift[J]. IEEE Transac-tions on Knowledge and Data Engineering, 2024, 36(12): 8023-8036. [12] Barboza E V L, de Almeida P R L, de Souza B A, et al. IncA-DES: an incremental and adaptive dynamic ensemble selection approach using online K-d tree neighborhood search for data streams with concept drift[J/OL]. Information Fusion, 2025, 123. https://doi.org/10.1016/j.inffus.2025.103272. [13] Li W D, Yang X, Liu W Q, et al. DDG-DA: data distribution generation for predictable concept drift adaptation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(4): 4092-4100. [14] Guo H S, Zhang S, Wang W J.Selective ensemble-based online adaptive deep neural networks for streaming data with concept drift[J]. Neural Networks, 2021, 142(C): 437-456. [15] Zhou H Y, Zhang S H, Peng J Q, et al. Informer: beyond efficient transformer for long sequence time-series forecasting[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(12): 11106-11115. [16] Nie Y Q, Nguyen N H, Sinthong P, et al. A time series is worth 64 words: long-term forecasting with Transformers[EB/OL]. [2026-04-05]. https://openreview.net/pdf?id=Jbdc0vTOcol. [17] Buzzega P, Boschini M, Porrello A, et al.Dark experience for general continual learning: a strong, simple baseline[C]//Procee-dings of the 34th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2020: 15920-15930. [18] Guo Y W, Yao A B, Chen Y R.Dynamic network surgery for effi-cient DNNs[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2016: 1389-1395. [19] Srivastava R K, Greff K, Schmidhuber J.Training very deep net-works[C]//Proceedings of the 29th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2015, II: 2377-2385. [20] Sahoo D, Pham Q, Lu J, et al. Online deep learning: learning deep neural networks on the fly[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence. San Francisco, USA: IJCAI, 2018: 2660-2666. [21] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 770-778. [22] Huang G, Liu Z, Pleiss G, et al. Convolutional networks with dense connectivity[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(12): 8704-8716. [23] Frías-Blanco I, del Campo-Ávila J, Ramos-Jiménez G, et al. Online and non-parametric drift detection methods based on Hoeffding's bounds[J]. IEEE Transactions on Knowledge and Data Enginee-ring, 2015, 27(3): 810-823. [24] Ben-David S, Blitzer J, Crammer K, et al. A theory of learning from different domains[J]. Machine Learning, 2010, 79(1): 151-175. [25] Brzezinski D, Stefanowski J.Prequential AUC for classifier evalua-tion and drift detection in evolving data streams[C]//Proceedings of the 3rd International Workshop on New Frontiers in Mining Complex Patterns. Berlin, Germany: Springer, 2014: 87-101. [26] Bifet A, Holmes G, Pfahringer B, et al. MOA: massive online analysis, a framework for stream classification and clustering[C]//Proceedings of the 1st Workshop on Applications of Pattern Analysis. San Diego, USA: JMLR, 2010: 44-50. [27] Gomes H M, Bifet A, Read J, et al. Adaptive random forests for evolving data stream classification[J]. Machine Learning, 2017, 106(9): 1469-1495. [28] Lu Y, Cheung Y M, Tang Y Y.Dynamic weighted majority for incremental learning of imbalanced data streams with concept drift[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence. San Francisco, USA: IJCAI, 2017: 2393-2399. [29] Bifet A, Holmes G, Pfahringer B.Leveraging bagging for evolving data streams[C]//Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases. Berlin, Germany: Springer, 2010: 135-150. [30] Gomes H M, Read J, Bifet A.Streaming random patches for evolving data stream classification[J]//Proceedings of the IEEE International Conference on Data Mining. Washington, USA: IEEE, 2019: 240-249. [31] Wang B Y, Pineau J.Online bagging and boosting for imbalanced data streams[J]. IEEE Transactions on Knowledge and Data Engineering, 2016, 28(12): 3353-3366. [32] Cano A, Krawczyk B.ROSE: robust online self-adjusting en-semble for continual learning on imbalanced drifting data streams[J]. Machine Learning, 2022, 111(7): 2561-2599. [33] Demšar J.Statistical comparisons of classifiers over multiple data sets[J]. Journal of Machine Learning Research, 2006, 7: 1-30.
2026, Vol. 39 
