Jie Liu
Abstract
Social media produces large amounts of content every day. How to predict the potential influences of the contents from a social reply feedback perspective is a key issue that has not been explored. Thus, we propose a novel task named reply keyword prediction in social media, which aims to predict the keywords in the potential replies in as many aspects as possible. One prerequisite challenge is that the accessible social media datasets labeling such keywords remain absent. To solve this issue, we propose a new dataset,1 to study the reply keyword prediction in social media. This task could be seen as a single-turn dialogue keyword prediction for open-domain dialogue system. However, existing methods for dialogue keyword prediction cannot be adopted directly, which has two main drawbacks. First, they do not provide an explicit mechanism to model topic complementarity between keywords which is crucial in social media to controllably model all aspects of replies. Second, the collocations of keywords are not explicitly modeled, which also makes it less controllable to optimize for fine-grained prediction since the context information is much less than that in dialogue. To address these issues, we propose a two-stage disentangled framework, which can optimize the complementarity and collocation explicitly in a disentangled fashion. In the first stage, we use a sequence-to-set paradigm via multi-label prediction and determinantal point processes, to generate a set of keyword seeds satisfying the complementarity. In the second stage, we adopt a set-to-sequence paradigm via seq2seq model with the keyword seeds guidance from the set, to generate the more-fine-grained keywords with collocation. Experiments show that this method can generate not only a more diverse set of keywords but also more relevant and consistent keywords. Furthermore, the keywords obtained based on this method can achieve better reply generation results in the retrieval-based system than others.
- [1] . 2019. Topic-aware neural keyphrase generation for social media language. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2516–2526.Google Scholar
Cross Ref
- [2] . 2018. One size does not fit all: Generating and evaluating variable number of keyphrases. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 7961–7975.Google Scholar
- [3] . 2021. Adaptive beam search decoding for discrete keyphrase generation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 13082–13089.Google ScholarCross Ref
- [4] . 2021. One2Set: Generating diverse keyphrases as a set. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, Online, 4598–4608.
DOI: Google ScholarCross Ref - [5] . 2019. Target-guided open-domain conversation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 5624–5634.Google ScholarCross Ref
- [6] . 2022. Retrieval-free knowledge-grounded dialogue response generation with adapters. In Proceedings of the Second DialDoc Workshop on Document-grounded Dialogue and Conversational Question Answering. 93–107.Google Scholar
- [7] . 2009. Determinantal point processes. arXiv preprint arXiv:0911.1153 (2009).Google Scholar
- [8] . 2018. Fast greedy MAP inference for determinantal point process to improve recommendation diversity. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS’18). Curran Associates Inc., Red Hook, NY, USA, 5627–5638.Google ScholarDigital Library
- [9] . 2019. GDPP: Learning diverse generations using determinantal point processes. In Proceedings of International Conference on Machine Learning.Google Scholar
- [10] . 2015. Neural responding machine for short-text conversation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 1577–1586.Google ScholarCross Ref
- [11] . 1971. Measuring nominal scale agreement among many raters. Psychological Bulletin 76, 5 (1971), 378.Google ScholarCross Ref
- [12] . 2019. ERNIE: Enhanced representation through knowledge integration. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 1441–1451.Google Scholar
- [13] . 2021. Familia: A configurable topic modeling framework for industrial text engineering. In Database Systems for Advanced Applications - 26th International Conference, DASFAA 2021, Taipei, Taiwan, April 11–14, 2021, Proceedings, Part III (LNCS), Vol. 12683. Springer, 516–528.
DOI: Google ScholarDigital Library - [14] . 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP’14). 1746–1751.Google ScholarCross Ref
- [15] . 2022. Multi-label softmax networks for pulmonary nodule classification using unbalanced and dependent categories. IEEE Transactions on Medical Imaging (2022).Google Scholar
- [16] Y. E. Lee and M. Lee. 2020. Decoding visual responses based on deep neural networks with ear-EEG signals. 8th International Winter Conference on Brain-Computer Interface (BCI’20), IEEE, 1–6.Google Scholar
- [17] . 2022. DQ-BART: Efficient sequence-to-sequence model via joint distillation and quantization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, Dublin, Ireland, 203–211.
DOI: Google ScholarCross Ref - [18] . 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).Google Scholar
- [19] . 2017. Attention is all you need. In Proceedings of Advances in Neural Information Processing Systems. 5998–6008.Google Scholar
- [20] . 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. arXiv preprint arXiv:1703.10960 (2017).Google Scholar
- [21] . 2016. Domain separation networks. In Proceedings of Advances in Neural Information Processing Systems. 343–351.Google Scholar
- [22] . 2018. Keyphrase generation with correlation constraints. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 4057–4066.Google ScholarCross Ref
- [23] . 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics Volume 1 (Long Papers). 1073–1083.Google ScholarCross Ref
- [24] . 2022. WR-One2Set: Towards well-calibrated keyphrase generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, , , and (Eds.). Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 7283–7293.
DOI: Google ScholarCross Ref - [25] . 2018. Directional skip-gram: Explicitly distinguishing left and right context for word embeddings. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers). Association for Computational Linguistics, 175–180. http://aclweb.org/anthology/N18-2028Google ScholarCross Ref
- [26] . 2009. On the use of the Adjusted Rand Index as a metric for evaluating supervised classification. In Proceedings of International Conference on Artificial Neural Networks. Springer, 175–184.Google ScholarDigital Library
- [27] . 2005. Agreement, the F-Measure, and reliability in information retrieval. Journal of the American Medical Informatics Association JAMIA (2005).Google Scholar
- [28] . 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 4171–4186.Google Scholar
- [29] . 2017. Topic aware neural response generation. In Proceedings of Thirty-First AAAI Conference on Artificial Intelligence.Google ScholarDigital Library
- [30] . 2015. A neural conversational model. arXiv preprint arXiv:1506.05869 (2015).Google Scholar
- [31] Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Association for Computational Linguistics, San Diego, CA, 110–119.Google Scholar
- [32] . 2016. Sequence to backward and forward sequences: A content-introducing approach to generative short-text conversation. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. 3349–3358.Google Scholar
- [33] . 2017. Multiresolution recurrent neural networks: An application to dialogue response generation. In Proceedings of Thirty-First AAAI Conference on Artificial Intelligence.Google ScholarCross Ref
- [34] . 2019. Order-sensitive keywords based response generation in open-domain conversational systems. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP) 19, 2 (2019), 1–18.Google Scholar
- [35] . 2019. Neural keyphrase generation via reinforcement learning with adaptive rewards. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2163–2174.Google ScholarCross Ref
- [36] . 2023. Task-optimized adapters for an end-to-end task-oriented dialogue system. In Findings of the Association for Computational Linguistics: ACL 2023, , , and (Eds.). Association for Computational Linguistics, Toronto, Canada, 7355–7369.
DOI: Google ScholarCross Ref - [37] . 2023. End-to-end knowledge retrieval with multi-modal queries. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), , , and (Eds.). Association for Computational Linguistics, Toronto, Canada, 8573–8589.
DOI: Google ScholarCross Ref
Index Terms
- Seq2Set2Seq: A Two-stage Disentangled Method for Reply Keyword Generation in Social Media
Recommendations
Social media user classification: based on social capital expectation, susceptibility, and compulsion loop
ICEC '17: Proceedings of the International Conference on Electronic CommerceSocial media such as Facebook, Instagram and Twitter are originally developed as communication tools among individuals for private conversations. Through the platforms, people share photos, stories and news with their social media friends to interact ...
One Social Movement, Two Social Media Sites: A Comparative Study of Public Discourses
Social media have become central places where public discourses are generated, sustained, and circulated around public events. So far, much research has examined large-scale dissemination patterns of prominent statements, opinions, and slogans ...
Comments