|
|
|
| Gated Dynamic Attention Mechanism towards Aspect Extraction |
| CHENG Meng1, HONG Yu1, TANG Jian1, ZHANG Jiashuo1, ZOU Bowei1, YAO Jianmin1 |
| 1.School of Computer Science and Technology, Soochow University, Suzhou 215006 |
|
|
|
|
Abstract In the current aspect extraction researches, the attention modeling and training are fixed, and the sentence is modeled in one time step. However, the semantics of the words vary in contexts, and a fixed attention distribution lacks dynamic adaptability. Therefore, a gated dynamic attention mechanism towards aspect extraction is proposed in this paper. A bidirectional long short term memory network is exploited to obtain hidden representations of words in a target sentence. Then, a specific attention distribution is computed according to the target word and its context while the attention model labelling words. Thus, the attention-weight distribution can be automatically adjusted according to the changes of contexts. Next, a gate is adopted to adjust the quantities of information flowing to the next units. Finally, conditional random field is utilized to label the aspect. The official datasets of 2014-2016 semantic evaluation are employed to verify the effectiveness of the proposed method, and F1 scores are increased.
|
|
Received: 21 October 2018
|
|
|
| Fund:Supported by National Key R&D Program of China(No.2017YFB1002104), National Natural Science Foundation of China(No.61672367,61672368) |
| About author:: (CHENG Meng, master student. His research interests include aspect based sentiment analysis and aspect extraction.) (HONG Yu(Corresponding author), Ph.D., associate professor. His research interests include information retrieval and information extraction.) (TANG Jian, master student. His research interests include machine translation.) (ZHANG Jiashuo, master student. His research interests include image captioning.) (ZOU Bowei, Ph.D. His research interests include information extraction and discourse analysis.) (YAO Jianmin, Ph.D., associate profe-ssor. His research interests include machine translation.) |
|
|
|
[1] PONTIKI M, GALANIS D, PAVLOPOULOS J, et al. SemEval-2014 Task 4: Aspect Based Sentiment Analysis // Proc of the 8th International Workshop on Semantic Evaluation. Stroudsburg, USA: ACL, 2014: 27-35.
[2] HU M Q, LIU B.Mining and Summarizing Customer Reviews // Proc of the 10th ACM SIGKDD International Conference on Know-ledge Discovery and Data Mining. New York, USA: ACM, 2004: 168-177.
[3] ZHUANG L, JING F, ZHU X Y.Movie Review Mining and Su-mmarization // Proc of the 15th ACM International Conference on Information and Knowledge Management. New York, USA: ACM, 2006: 43-50.
[4] BLAIR-GOLDENSOHN S, HANNAN K, MCDONALD R, ,et al. Building a Sentiment Summarizer for Local Service Reviews // Proc of the Workshop on NLP in the Information Explosion Era. Stroudsburg. Building a Sentiment Summarizer for Local Service Reviews // Proc of the Workshop on NLP in the Information Explosion Era. Stroudsburg, USA: ACL, 2008, XIV: 339-348.
[5] WANG B, WANG H F. Bootstrapping Both Product Features and Opinion Words from Chinese Customer Reviews with Cross-Inducing[C/OL]. [2018-05-30]. http://www.aclweb.org/anthology/I/I08/I08-1038.pdf.
[6] MEI Q Z, LING X, WONDRA M, et al. Topic Sentiment Mixture: Modeling Facets and Opinions in Weblogs // Proc of the 16th International Conference on World Wide Web. New York, USA: ACM, 2007: 171-180.
[7] TITOV I, Mcdonald R.Modeling Online Reviews with Multi-grain Topic Models // Proc of the 17th International Conference on World Wide Web. New York, USA: ACM, 2008: 111-120.
[8] LIN C H, HE Y L.Joint Sentiment/Topic Model for Sentiment Analysis // Proc of the 18th ACM Conference on Information and Knowledge Management. New York, USA: ACM, 2009: 375-384.
[9] MUKHERJEE A, LIU B.Aspect Extraction through Semi-supervised Modeling // Proc of the 50th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2012: 339-348.
[10] JIN W, HO H H, SRIHARI R K.OpinionMiner: A Novel Machine Learning System for Web Opinion Mining and Extraction // Proc of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2009: 1195-1204.
[11] JAKOB N, GUREVYCH I.Extracting Opinion Targets in a Single-and Cross-Domain Setting with Conditional Random Fields // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2010: 1035-1045.
[12] LI F T, HAN C, HUANG M L, et al. Structure-Aware Review Mining and Summarization // Proc of the 23rd International Conference on Computational Linguistics. Stroudsburg, USA: ACL, 2010: 653-661.
[13] XU L H, LIU K, ZHAO J.Joint Opinion Relation Detection Using One-Class Deep Neural Network // Proc of the 25th International Conference on Computational Linguistics. Stroudsburg, USA: ACL, 2014: 677-687.
[14] LIU P F, JOTY S, MENG H L.Fine-Grained Opinion Mining with Recurrent Neural Networks and Word Embeddings // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2015: 1433-1443.
[15] LI X, LAM W.Deep Multi-task Learning for Aspect Term Extraction with Memory Interaction // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2017: 2886-2892.
[16] TOH Z Q, SU J.NLANGP at SemEval-2016 Task 5: Improving Aspect Based Sentiment Analysis Using Neural Network Features // Proc of the 10th International Workshop on Semantic Evaluation. Stroudsburg, USA: ACL, 2016: 282-288.
[17] BAHDANAU D, CHO K, BENGIO Y.Neural Machine Translation by Jointly Learning to Align and Translate[C/OL]. [2018-05-30].https://arxiv.org/pdf/1409.0473v7.pdf.
[18] WANG W Y, PAN S J, DAHLMEIER D, et al.Coupled Multi-layer Attentions for Co-extraction of Aspect and Opinion Terms // Proc of the 31st AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2017: 3316-3322.
[19] YANG Z C, YANG D Y, DYER C, et al. Hierarchical Attention Networks for Document Classification // Proc of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, USA: ACL, 2016: 1480-1489.
[20] SCHUSTER M, PALIWAL K K.Bidirectional Recurrent Neural Networks. IEEE Transactions on Signal Processing, 1997, 45(11): 2673-2681.
[21] COLLINS M. The Forward-Backward Algorithm[C/OL]. [2018-05-30]. http://www.cs.columbia.edu/~mcollins/fb.pdf.
[22] HUANG Z H, XU W, YU K.Bidirectional LSTM-CRF Models for Sequence Tagging[C/OL]. [2018-05-30].https://arxiv.org/pdf/1508.01991.pdf.
[23] PONTIKI M, GALANIS D, PAPAGEORGIOU H, et al. Semeval-2015 Task 12: Aspect Based Sentiment Analysis // Proc of the 9th International Workshop on Semantic Evaluation. Stroudsburg, USA: ACL, 2015: 486-495.
[24] PONTIKI M, GALANIS D, PAPAGEORGIOU H, et al. SemEval-2016 Task 5: Aspect Based Sentiment Analysis // Proc of the 10th International Workshop on Semantic Evaluation. Stroudsburg, USA: ACL, 2016: 19-30.
[25] GREFF K, SRIVASTAVA R K, KOUTNÍk J, et al. LSTM: A Search Space ODYSSEY. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(10): 2222-2232.
[26] CHERNYSHEVICH M.IHS R&D Belarus: Cross-Domain Extraction of Product Features Using Conditional Random Fields // Proc of the 8th International Workshop on Semantic Evaluation. Stroudsburg, USA: ACL, 2014: 309-313.
[27] TOH Z Q, WANG W T.DLIREC: Aspect Term Extraction and Term Polarity Classification System // Proc of the 8th International Workshop on Semantic Evaluation. Stroudsburg, USA: ACL, 2014: 235-240.
[28] YIN Y C, WEI F R, DONG L, et al.Unsupervised Word and Dependency Path Embeddings for Aspect Term Extraction // Proc of the 25th International Joint Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2016: 2979-2985.
[29] VICENTE I S, SARALEGI X, AGERRI R.EliXa: A Modular and Flexible ABSA Platform // Proc of the 9th International Workshop on Semantic Evaluation. Stroudsburg, USA: ACL, 2015: 748-752.
[30] ZAREMBA W, SUTSKEVER I, VINYALS O. Recurrent Neural Network Regularization[C/OL]. [2018-05-30]. https://arxiv.org/pdf/1409.2329.pdf. |
|
|
|
2019, Vol. 32 
