Hierarchical Topic Model Based on Chain of Thought and Semantic Decoupling
WANG Zhihua1, LI Yang2, LI Deyu1,3, WANG Suge1,3
1. School of Computer and Information Technology, Shanxi University, Taiyuan 030006; 2. School of Finance, Shanxi University of Finance and Economics, Taiyuan 030006; 3. Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, Shanxi University, Taiyuan 030006
Abstract：Hierarchical topic models can uncover latent topics in documents and model the hierarchical relationships between topics, providing technical support for applications such as data governance, information retrieval, content classification, and knowledge management. A hierarchical topic model based on chain of thought and semantic decoupling(CoT-SDHT-M) is proposed in this paper. First, a hierarchical topic generation module based on a chain of thought is established. An initial hierarchical topic structure is generated by a large language model(LLM) under the guidance of hierarchical topic generation chain of thought. Then, a topic similarity discrimination mechanism based on LLM is introduced to generate refined topics and to guide the LLM in merging topics through examples, thereby improving the quality of the generated topics. Finally, a hierarchical topic optimization module based on transport planning and semantic decoupling is designed. It incorporates the initial hierarchical structure as a topic prior for downstream modeling. The relationships between topics are modeled as an optimal transport problem, and parent-child topic decoupling is performed based on the keywords of upper-layer and lower-layer topics to optimize the hierarchical topic structure. The experiments on various standard public datasets, including NeurIPS, ACL and 20 Newsgroups, demonstrate that CoT-SDHT-M significantly outperforms existing baseline models in terms of topic quality metrics and hierarchical metrics.
王志华, 李旸, 李德玉, 王素格. 基于思维链和语义解耦的层次化主题模型[J]. 模式识别与人工智能, 2025, 38(7): 613-626.
WANG Zhihua, LI Yang, LI Deyu, WANG Suge. Hierarchical Topic Model Based on Chain of Thought and Semantic Decoupling. Pattern Recognition and Artificial Intelligence, 2025, 38(7): 613-626.
[1] BLEI D M, NG A Y, JORDAN M I. Latent Dirichlet Allocation. Journal of the Machine Learning Research, 2003, 3(1): 993-1022. [2] BLEI D M, JORDAN M I, GRIFFITHS T L, et al. Hierarchical Topic Models and the Nested Chinese Restaurant Process // Proc of the 17th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2003: 17-24. [3] 万常选,张奕韬,刘德喜,等.主题方面共享的领域主题层次模型.软件学报, 2024, 35(4): 1790-1818. (WAN C X, ZHANG Y T, LIU D X, et al. Domain Topic Hierar-chy Model for Topic Aspect Sharing. Journal of Software, 2024, 35(4): 1790-1818.) [4] KINGMA D P, WELLING M. Auto-Encoding Variational Bayes[C/OL]. [2025-05-17].https://arxiv.org/pdf/1312.6114v3. [5] ISONUMA M, MORI J, BOLLEGALA D, et al. Tree-Structured Neural Topic Model // Proc of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2020: 800-806. [6] CHEN Z Y, DING C, ZHANG Z S, et al. Tree-Structured Topic Modeling with Nonparametric Neural Variational Inference // Proc of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(Long Papers). Stroudsburg, USA: ACL, 2021: 2343-2353. [7] CHEN Z Y, DING C, RAO Y H, et al. Hierarchical Neural Topic Modeling with Manifold Regularization. World Wide Web, 2021, 24(6): 2139-2160. [8] XU Y S, WANG D S, CHEN B, et al. HyperMiner: Topic Taxonomy Mining with Hyperbolic Embedding // Proc of the 36th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2022: 31557-31570. [9] LIN Z C, CHEN H G, LU Y Y, et al. Hierarchical Topic Modeling via Contrastive Learning and Hyperbolic Embedding // Proc of the Joint International Conference on Computational Linguistics, Language Resources and Evaluation. Stroudsburg, USA: ACL, 2024: 8133-8143. [10] CHEN H G, MAO P G, LU Y Y, et al. Nonlinear Structural Equa-tion Model Guided Gaussian Mixture Hierarchical Topic Modeling // Proc of the 61st Annual Meeting of the Association for Computational Linguistics(Long Papers). Stroudsburg, USA: ACL, 2023: 10377-10390. [11] LIU J Y, CHEN H G, ZHU C J, et al. Unsupervised Hierarchical Topic Modeling via Anchor Word Clustering and Path Guidance // Findings of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2024: 7505-7517. [12] WU X B, PAN F J, NGUYEN T, et al. On the Affinity, Rationa-lity, and Diversity of Hierarchical Topic Modeling. Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38(17): 19261-19269. [13] STAMMBACH D, ZOUHAR V, HOYLE A, et al. Revisiting Automated Topic Model Evaluation with Large Language Models // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2023: 9348-9357. [14] RIJCKEN E, SCHEEPERS F, ZERVANOU K, et al. Towards Interpreting Topic Models with ChatGPT[C/OL].[2025-05-17]. https://pure.tue.nl/ws/portalfiles/portal/300364784/IFSA_InterpretingTopicModelsWithChatGPT.pdf. [15] MU Y D, DONG C, BONTCHEVA K, et al. Large Language Mo-dels Offer an Alternative to the Traditional Approach of Topic Mo-delling // Proc of the Joint International Conference on Computational Linguistics, Language Resources and Evaluation. Stroudsburg, USA: ACL, 2024: 10160-10171. [16] PHAM C M, HOYLE A, SUN S M, et al. TopicGPT: A Prompt-Based Topic Modeling Framework // Proc of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies(Long Papers). Stroudsburg, USA: ACL, 2024: 2956-2984. [17] ZHANG Y W, WANG Z H, SHANG J B. ClusterLLM: Large Language Models as a Guide for Text Clustering // Proc of the Confe-rence on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2023: 13903-13920. [18] PEYRÉ G, CUTURI M. Computational Optimal Transport. Foun-dations and Trends® in Machine Learning, 2019, 11(5/6): 355-607. [19] SINKHORN R. A Relationship between Arbitrary Positive Matrices and Doubly Stochastic Matrices. The Annals of Mathematical Statistics, 1964, 35(2): 876-879. [20] TEAM GLM. ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools[C/OL]. [2025-05-17].https://arxiv.org/pdf/2406.12793. [21] DUAN Z B, WANG D S, CHEN B, et al. Sawtooth Factorial To-pic Embeddings Guided Gamma Belief Network. Proceedings of the Machine Learning Research, 2021, 139: 2903-2913. [22] LI Y W, WANG C J, DUAN Z B, et al. Alleviating "Posterior Collapse″ in Deep Topic Models via Policy Gradient // Proc of the 36th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2022: 22562-22575. [23] DUAN Z B, LIU X Y, SU Y D, et al. Bayesian Progressive Deep Topic Model with Knowledge Informed Textual Data Coarsening Process. Proceedings of the Machine Learning Research, 2023, 202: 8731-8746.
2025, Vol. 38 
