Skip to main content
SpringerLink
Log in

Graph Enhanced Transformer for Aspect Category Detection

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Aspect category detection is one challenging subtask of aspect based sentiment analysis, which categorizes a review sentence into a set of predefined aspect categories. Most existing methods regard the aspect category detection as a flat classification problem. However, aspect categories are inter-related, and they are usually organized with a hierarchical tree structure. To leverage the structure information, this paper proposes a hierarchical multi-label classification model to detect aspect categories and uses a graph enhanced transformer network to integrate label dependency information into prediction features. Experiments have been conducted on four widely-used benchmark datasets, showing that the proposed model outperforms all strong baselines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S. SemEval-2014 task 4: Aspect based sentiment analysis. In Proc. the 8th International Workshop on Semantic Evaluation, Aug. 2014, pp.27–35. https://doi.org/10.3115/v1/s14-2004.

  2. Pontiki M, Galanis D, Papageorgiou H, Manandhar S, Androutsopoulos I. SemEval-2015 task 12: Aspect based sentiment analysis. In Proc. the 9th International Workshop on Semantic Evaluation, Jun. 2015, pp.486–495. https://doi.org/10.18653/v1/s15-2082.

  3. Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, Al-Smadi M, Al-Ayyoub M, Zhao Y Y, Qin B, De Clercq O, Hoste V, Apidianaki M, Tannier X, Loukachevitch N, Kotelnikov E, Bel N, Jiménez-Zafra S M, Eryiğit G. SemEval-2016 task 5: Aspect based sentiment analysis. In Proc. the 10th International Workshop on Semantic Evaluation, Jun. 2016, pp.19–30. https://doi.org/10.18653/v1/s16-1002.

  4. Ganu G, Elhadad N, Marian A. Beyond the stars: Improving rating predictions using review text content. In Proc. the 12th International Workshop on the Web and Databases, Jun. 2009.

  5. Toh Z, Su J. NLANGP at SemEval-2016 task 5: Improving aspect based sentiment analysis using neural network features. In Proc. the 10th International Workshop on Semantic Evaluation, Jun. 2016, pp.282–288. https://doi.org/10.18653/v1/s16-1045.

  6. Xue W, Zhou W B, Li T, Wang Q. MTNA: A neural multi-task model for aspect category classification and aspect term extraction on restaurant reviews. In Proc. the 8th International Joint Conference on Natural Language Processing, Nov. 2017, pp.151–156.

  7. Hu M Y, Zhao S W, Zhang L, Cai K K, Su Z, Cheng R H, Shen X W. CAN: Constrained attention networks for multi-aspect sentiment analysis. In Proc. the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Nov. 2019. pp.4601–4610. https://doi.org/10.18653/v1/D19-1467.

  8. Movahedi S, Ghadery E, Faili H, Shakery A. Aspect category detection via topic-attention network. arXiv: 1901.01183, 2019. https://arxiv.org/abs/1901.01183, May 2023.

  9. Ghadery E, Movahedi S, Faili H, Shakery A. MNCN: A multilingual ngram-based convolutional network for aspect category detection in online reviews. In Proc. the 33rd AAAI Conference on Artificial Intelligence, Jul. 2019, pp.6441–6448. https://doi.org/10.1609/aaai.v33i01.33016441.

  10. Ghadery E, Movahedi S, Sabet M J, Faili H, Shakery A. LICD: A language-independent approach for aspect category detection. In Proc. the 41st European Conference on Information Retrieval, Apr. 2019, pp.575–589. https://doi.org/10.1007/978-3-030-15712-8_37.

  11. Vens C, Struyf J, Schietgat L, Džeroski S, Blockeel H. Decision trees for hierarchical multi-label classification. Machine Learning, 2008, 73(2): 185–214. https://doi.org/10.1007/s10994-008-5077-3.

    Article  MATH  Google Scholar 

  12. Wang S F, Wang J, Wang Z Y, Ji Q. Enhancing multi-label classification by modeling dependencies among labels. Pattern Recognition, 2014, 47(10): 3405–3413. https://doi.org/10.1016/j.patcog.2014.04.009.

    Article  Google Scholar 

  13. Toh Z, Su J. NLANGP: Supervised machine learning system for aspect category classification and opinion target extraction. In Proc. the 9th International Workshop on Semantic Evaluation, Jun. 2015, pp.496–501. https://doi.org/10.18653/v1/s15-2083.

  14. Zhou X J, Wan X J, Xiao J G. Representation learning for aspect category detection in online reviews. In Proc. the 29th AAAI Conference on Artificial Intelligence, Jan. 2015, pp.417–423. https://doi.org/10.1609/aaai.v29i1.9194.

  15. Xenos D, Theodorakakos P, Pavlopoulos J, Malakasiotis P, Androutsopoulos I. AUEB-ABSA at SemEval-2016 task 5: Ensembles of classifiers and embeddings for aspect based sentiment analysis. In Proc. the 10th International Workshop on Semantic Evaluation, Jun. 2016, pp.312–317. https://doi.org/10.18653/v1/s16-1050.

  16. Silla C N Jr, Freitas A A. A survey of hierarchical classification across different application domains. Data Mining and Knowledge Discovery, 2011, 22(1/2): 31–72. https://doi.org/10.1007/s10618-010-0175-9.

  17. Cerri R, Barros R C, de Carvalho A C P L E, Jin Y C. Reduction strategies for hierarchical multi-label classification in protein function prediction. BMC Bioinformatics, 2016, 17(1): Article No. 373. https://doi.org/10.1186/s12859-016-1232-1.

  18. Wehrmann J, Barros R C, das Dôres S N, Cerri R. Hierarchical multi-label classification with chained neural networks. In Proc. the Symposium on Applied Computing, Apr. 2017, pp.790–795. https://doi.org/10.1145/3019612.3019664.

  19. Peng H, Li J X, He Y, Liu Y P, Bao M J, Wang L H, Song Y Q, Yang Q. Large-scale hierarchical text classification with recursively regularized deep graph-CNN. In Proc. the 2018 World Wide Web Conference, Apr. 2018, pp.1063–1072. https://doi.org/10.1145/3178876.3186005.

  20. Wang P F, Fan Y, Niu S Z, Yang Z, Zhang Y F, Guo J F. Hierarchical matching network for crime classification. In Proc. the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Jul. 2019, pp.325–334. https://doi.org/10.1145/3331184.3331223.

  21. Bahdanau D, Cho K H, Bengio Y. Neural machine translation by jointly learning to align and translate. In Proc. the 3rd International Conference on Learning Representations, May 2015.

  22. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I. Attention is all you need. In Proc. the 31st International Conference on Neural Information Processing Systems, Dec. 2017, pp.6000–6010. https://doi.org/10.5555/3295222.3295349.

  23. Wang X, Tu Z P, Wang L Y, Shi S M. Self-attention with structural position representations. In Proc. the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Nov. 2019, pp.1403–1409. https://doi.org/10.18653/v1/D19-1145.

  24. Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y. Graph attention networks. In Proc. the 6th International Conference on Learning Representations, Apr. 30–May 3, 2018.

  25. Shaw P, Uszkoreit J, Vaswani A. Self-attention with relative position representations. In Proc. the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Jun. 2018, pp.464–468. https://doi.org/10.18653/v1/n18-2074.

  26. Busbridge D, Sherburn D, Cavallo P, Hammerla N Y. Relational graph attention networks. arXiv: 1904.05811, 2019. https://arxiv.org/abs/1904.05811, May 2023.

  27. Guo Q P, Qiu X P, Liu P F, Shao Y F, Xue X Y, Zhang Z. Star-Transformer. In Proc. the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Jun. 2019, pp.1315–1325. https://doi.org/10.18653/v1/n19-1133.

  28. Cho K, van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proc. the 2014 Conference on Empirical Methods in Natural Language Processing, Oct. 2014, pp.1724–1734. https://doi.org/10.3115/v1/d14-1179.

  29. He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Dec. 2016, pp.770–780. https://doi.org/10.1109/CVPR.2016.90.

  30. Ba J L, Kiros J R, Hinton G E. Layer normalization. arXiv: 1607.06450, 2016. https://arxiv.org/abs/1607.06450, May 2023.

  31. Kingma D P, Ba J. Adam: A method for stochastic optimization. In Proc. the 3rd International Conference on Learning Representations, May 2015.

  32. Liu L Q, Mu F N, Li P Y, Mu X, Tang J, Ai X S, Fu R, Wang L F, Zhou X. NeuralClassifier: An open-source neural hierarchical multi-label text classification Toolkit. In Proc. the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, Jul. 2019, pp.87–92. https://doi.org/10.18653/v1/p19-3015.

  33. Lai S W, Xu L H, Liu K, Zhao J. Recurrent convolutional neural networks for text classification. In Proc. the 29th AAAI Conference on Artificial Intelligence, Jan. 2015, pp.2267–2273. https://doi.org/10.1609/aaai.v29i1.9513.

  34. Johnson R, Zhang T. Deep pyramid convolutional neural networks for text categorization. In Proc. the 55th Annual Meeting of the Association for Computational Linguistics, Jul. 2017, pp.562–570. https://doi.org/10.18653/v1/P17-1052.

  35. Liu P F, Qiu X P, Huang X J. Recurrent neural network for text classification with multi-task learning. In Proc. the 25th International Joint Conference on Artificial Intelligence, Jul. 2016, pp.2873–2879.

  36. Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks. In Proc. the 5th International Conference on Learning Representations, Apr. 2017.

Download references

Author information

Authors and Affiliations

  1. Office of the Cyberspace Affairs Commission, Peking University, Beijing, 100871, China

    Chen Chen

  2. School of Computer Science, Peking University, Beijing, 100871, China

    Hou-Feng Wang

  3. School of Software and Microelectronics, Peking University, Beijing, 100871, China

    Qing-Qing Zhu

  4. National Engineering Research Center for Software Engineering, Peking University, Beijing, 100871, China

    Jun-Fei Liu

Authors
  1. Chen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hou-Feng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qing-Qing Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun-Fei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hou-Feng Wang.

Supplementary Information

ESM 1

(PDF 143 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, C., Wang, HF., Zhu, QQ. et al. Graph Enhanced Transformer for Aspect Category Detection. J. Comput. Sci. Technol. 38, 612–625 (2023). https://doi.org/10.1007/s11390-021-1000-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-021-1000-1

Keywords

Search

Navigation