Abstract In multi-label learning, the performance of a learning algorithm can be improved by discovering and making use of the dependencies within the labels. In this paper, an innovated algorithm for multi-label learning based on the classifier chain model is proposed. This algorithm mainly consists of two steps. The dependencies are quantified firstly using mutual information, and then a tree structure of labels is derived to depict the relationship within labels. Thus, the randomness of dependencies in classifier chain is weakened, and the linear dependency is generalized to a tree structure one. To further utilize the dependencies, ensemble technique is used to learn and aggregate multiple trees of labels. The experimental results show that the proposed algorithm is also competitive alternative and it improves the performance significantly after ensemble learning especially, hence it can learn the dependencies within labels more effectively.
[1] Cheng W,Hullermeier E.Combining Instance-Based Learning and Logistic Regression for Multilabel Classification.Machine Learning,2009,76( 2/3): 211-225 [2] Tsoumakas G,Katakis I,Vlahavas I.Mining Multi-Label Data // Oded M,Lior R,eds.Data Mining and Knowledge Discovery Handbook.New York,USA: Springer,2010: 667-685 [3] Read J.Multi-Label Classification Using Ensembles of Pruned Sets // Proc of the IEEE International Conference on Data Mining.Pisa,Italy,2008: 995-1000 [4] Dembczynski K,Cheng W,Hullermeier E.Bayes Optimal Multilabel Classification via Probabilistic Classifier Chains // Proc of the 27th International Conference on Machine Learning.Haifa,Israel,2010: 279-286 [5] Zhang Minling,Zhang Kun.Multi-Label Learning by Exploiting Label Dependency // Proc of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.Washington,USA,2010: 999-1008 [6] Read J,Pfahringer B,Holmes G,et al.Classifier Chains for Multi-Label Classification // Proc of the European Conference on Machine Learning and Knowledge Discovery in Databases.Bled,Slovenia,2009: 254-269 [7] Schapire R E,Singer Y.Boostexter: A Boosting-Based System for Text Categorization.Machine Learning,2000,39(2/3): 135-168 [8] Clare A,King R D.Knowledge Discovery in Multi-Label Phenotype Data // Proc of the 5th European Conference on Principles of Data Mining and Knowledge Discovery.Freiburg,Germany,2001: 42-53 [9] Zhang Minling,Zhou Zhihua.ML-KNN: A Lazy Learning Approach to Multi-Label Learning.Pattern Recognition,2007,40(7): 2038-2048 [10] McCallum A K.Multi-Label Text Classification with a Mixture Model Trained by EM [EB/OL].[1999-12-10].http: // www.kyriakides.net/CBCL/references/Papers/mccallum99 multilabel.pdf [11] Elisseeff A,Weston J.A Kernel Method for Multi-Labelled Classification // Oietterich T G,Becker S,Ghahramani,eds.Advances in Neural Information Processing Systems.Cambridge USA: MIT Press,2002,XIV: 681-687 [12] Thabtah F A,Cowling P,Peng Y.MMAC: A New Multi-Class,Multi-Label Associative Classification Approach // Proc of the 4th International Conference on Data Mining.Borighton,UK,2004: 217-224 [13] Veloso A,Wagner M J,Goncalves M,et al.Multi-Label Lazy Associative Classification // Proc of the 11th European Conference on Principles and Practice of Knowledge Discovery in Databases.Warsaw,Poland,2007: 605-612 [14] Chen Weizhu,Yan Jun,Zhang Benyu,et al.Document Transformation for Multi-Label Feature Selection in Text Categorization // Proc of the 7th IEEE International Conference on Data Mining.Omaha,USA,2007: 451-456 [15] Tsoumakas G,Vlahavas I.Random k-Labelsets: An Ensemble Method for Multilabel Classification // Proc of the 18th European Conference on Machine Learning.Warsaw,Poland,2007: 406-417 [16] Boutell M,Luo J,Shen X,et al.Learning Multi-Label Scene Classification.Pattern Recognition,2004,37(9): 1757-1771 [17] Hullermeier E,Furnkranz J,Cheng W,et al.Label Ranking by Learning Pairwise Preferences.Artificial Intelligence,2008,172(16/17): 1897-1916 [18] Furnkranz J,Hullermeier E,Mencia E,et al.Multilabel Classification via Calibrated Label Ranking.Machine Learning,2008,73(2): 133-153 [19] Gaag L,Waal P.Multi-Dimensional Bayesian Network Classifiers // Proc of the 3rd European Workshop on Probabilistic Graphical Models.Prague,Czech Republic,2006: 107-114 [20] Battiti R.Using Mutual Information for Selecting Features in Supervised Neural Net Learning.IEEE Trans on Neural Networks,1994,5(4): 537-550 [21] Guo B,Nixon M S.Gait Feature Subset Selection by Mutual Information.IEEE Trans on Systems,Man and Cybernetics,2009,39(1): 36-46 [22] Witten I,Frank E.Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations.2nd Edition.San Francisco,USA: Morgan Kaufmann Publishers,2000
2012, Vol. 25 
