Zhiyuan Wen
Skip Abstract Section
Abstract
Generating appropriate emotions for responses is essential for dialog systems to provide human-like interaction in various application scenarios. Most previous dialog systems tried to achieve this goal by learning empathetic manners from anonymous conversational data. However, emotional responses generated by those methods may be inconsistent, which will decrease user engagement and service quality. Psychological findings suggest that the emotional expressions of humans are rooted in personality traits. Therefore, we propose a new task, Personality-affected Emotion Generation, to generate emotion based on the personality given to the dialog system and further investigate a solution through the personality-affected mood transition. Specifically, we first construct a daily dialog dataset, Personality EmotionLines Dataset (PELD), with emotion and personality annotations. Subsequently, we analyze the challenges in this task, i.e., (1) heterogeneously integrating personality and emotional factors and (2) extracting multi-granularity emotional information in the dialog context. Finally, we propose to model the personality as the transition weight by simulating the mood transition process in the dialog system and solve the challenges above. We conduct extensive experiments on PELD for evaluation. Results suggest that by adopting our method, the emotion generation performance is improved by 13% in macro-F1 and 5% in weighted-F1 from the BERT-base model.
- Elisabeth André, Martin Klesen, Patrick Gebhard, Steve Allen, and Thomas Rist. 1999. Integrating models of personality and emotions into lifelike characters. In International Workshop on Affective Interactions. Springer, 150–165.Google Scholar
- Nabiha Asghar, Pascal Poupart, Jesse Hoey, Xin Jiang, and Lili Mou. 2018. Affective neural response generation. In European Conference on Information Retrieval. Springer, 154–166.Google ScholarCross Ref
- G Ball and J Breese. 2000. Emotion and personality ina conversational character. Embodied Conversational Agents. MIT Press, Cambridge (2000).Google Scholar
- C Bauerhenne, A Gammoudi, M Moussaoui, J Reichelt, and J Wang. 2020. Emotional states and personality profiles in Conversational AI. (2020).Google Scholar
- Sven Buechel and Udo Hahn. 2016. Emotion analysis as a regression problem–dimensional models and their implications on emotion representation and metrical evaluation. In ECAI 2016. IOS Press, 1114–1122.Google Scholar
- Sven Buechel and Udo Hahn. 2017. Emobank: Studying the impact of annotation perspective and representation format on dimensional emotion analysis. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers. 578–585.Google ScholarCross Ref
- Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth S Narayanan. 2008. IEMOCAP: Interactive emotional dyadic motion capture database. Language resources and evaluation 42, 4 (2008), 335–359.Google Scholar
- Kaitlin E Cannava, Andrew C High, Susanne M Jones, and Graham D Bodie. 2018. The stuff that verbal person-centered support is made of: Identifying linguistic markers of more and less supportive conversations. Journal of Language and Social Psychology 37, 6 (2018), 656–679.Google ScholarCross Ref
- Kenneth Mark Colby. 1975. Artificial paranoia: a computer simulation of paranoid process. Pergamon Press.Google Scholar
- Pierre Colombo, Wojciech Witon, Ashutosh Modi, James Kennedy, and Mubbasir Kapadia. 2019. Affect-driven dialog generation. arXiv preprint arXiv:1904.02793(2019).Google Scholar
- Paul T Costa and Robert R McCrae. 1992. Normal personality assessment in clinical practice: The NEO Personality Inventory.Psychological assessment 4, 1 (1992), 5.Google Scholar
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805(2018).Google Scholar
- Paul Ed Ekman and Richard J Davidson. 1994. The nature of emotion: Fundamental questions.Oxford University Press.Google Scholar
- Howard S Friedman and Margaret L Kern. 2014. Personality, well-being, and health. Annual review of psychology 65 (2014), 719–742.Google Scholar
- Deepanway Ghosal, Navonil Majumder, Soujanya Poria, Niyati Chhaya, and Alexander Gelbukh. 2019. Dialoguegcn: A graph convolutional neural network for emotion recognition in conversation. arXiv preprint arXiv:1908.11540(2019).Google Scholar
- Meng-Ju Han, Chia-How Lin, and Kai-Tai Song. 2010. A design for smooth transition of robotic emotional states. In 2010 IEEE Workshop on Advanced Robotics and its Social Impacts. IEEE, 13–18.Google ScholarCross Ref
- Meng-Ju Han, Chia-How Lin, and Kai-Tai Song. 2012. Robotic emotional expression generation based on mood transition and personality model. IEEE transactions on cybernetics 43, 4 (2012), 1290–1303.Google Scholar
- Devamanyu Hazarika, Soujanya Poria, Rada Mihalcea, Erik Cambria, and Roger Zimmermann. 2018. Icon: Interactive conversational memory network for multimodal emotion detection. In Proceedings of the 2018 conference on empirical methods in natural language processing. 2594–2604.Google ScholarCross Ref
- Devamanyu Hazarika, Soujanya Poria, Amir Zadeh, Erik Cambria, Louis-Philippe Morency, and Roger Zimmermann. 2018. Conversational memory network for emotion recognition in dyadic dialogue videos. In Proceedings of the conference. Association for Computational Linguistics. North American Chapter. Meeting, Vol. 2018. NIH Public Access, 2122.Google ScholarCross Ref
- Minlie Huang, Xiaoyan Zhu, and Jianfeng Gao. 2020. Challenges in building intelligent open-domain dialog systems. ACM Transactions on Information Systems (TOIS) 38, 3 (2020), 1–32.Google ScholarDigital Library
- Chika Itoh, Shohei Kato, and Hidenori Itoh. 2009. Mood-transition-based emotion generation model for the robot’s personality. In 2009 IEEE International Conference on Systems, Man and Cybernetics. IEEE, 2878–2883.Google ScholarCross Ref
- Hang Jiang, Xianzhe Zhang, and Jinho D Choi. 2019. Automatic Text-based Personality Recognition on Monologues and Multiparty Dialogues Using Attentive Networks and Contextual Embeddings. arXiv preprint arXiv:1911.09304(2019).Google Scholar
- Michael Johns and Barry G Silverman. 2001. How emotions and personality effect the utility of alternative decisions: a terrorist target selection case study. Center for Human Modeling and Simulation(2001), 10.Google Scholar
- Henrik Kessler, Alexander Festini, Harald C Traue, Suzanne Filipic, Michael Weber, and Holger Hoffmann. 2008. SIMPLEX–simulation of personal emotion experience. Affective Computing (2008), 255–270.Google Scholar
- Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980(2014).Google Scholar
- Kwan Min Lee, Wei Peng, Seung-A Jin, and Chang Yan. 2006. Can robots manifest personality?: An empirical test of personality recognition, social responses, and social presence in human–robot interaction. Journal of communication 56, 4 (2006), 754–772.Google ScholarCross Ref
- Changmao Li and Jinho D Choi. 2020. Transformers to Learn Hierarchical Contexts in Multiparty Dialogue for Span-based Question Answering. arXiv preprint arXiv:2004.03561(2020).Google Scholar
- Jiwei Li, Michel Galley, Chris Brockett, Georgios P Spithourakis, Jianfeng Gao, and Bill Dolan. 2016. A persona-based neural conversation model. arXiv preprint arXiv:1603.06155(2016).Google Scholar
- Jia Li, Xiao Sun, Xing Wei, Changliang Li, and Jianhua Tao. 2019. Reinforcement Learning Based Emotional Editing Constraint Conversation Generation. arXiv preprint arXiv:1904.08061(2019).Google Scholar
- Minglei Li, Qin Lu, Yunfei Long, and Lin Gui. 2017. Affective State Prediction of Contextualized Concepts. In IJCAI 2017 Workshop on Artificial Intelligence in Affective Computing. PMLR, 45–57.Google Scholar
- Minglei Li, Qin Lu, Yunfei Long, and Lin Gui. 2017. Inferring affective meanings of words from word embedding. IEEE Transactions on Affective Computing 8, 4 (2017), 443–456.Google ScholarCross Ref
- Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. Dailydialog: A manually labelled multi-turn dialogue dataset. arXiv preprint arXiv:1710.03957(2017).Google Scholar
- Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision. 2980–2988.Google ScholarCross Ref
- Zijie Lin, Bin Liang, Yunfei Long, Yixue Dang, Min Yang, Min Zhang, and Ruifeng Xu. 2022. Modeling Intra-and Inter-Modal Relations: Hierarchical Graph Contrastive Learning for Multimodal Sentiment Analysis. In Proceedings of the 29th International Conference on Computational Linguistics. 7124–7135.Google Scholar
- Zhaojiang Lin, Andrea Madotto, Jamin Shin, Peng Xu, and Pascale Fung. 2019. MoEL: Mixture of Empathetic Listeners. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, 121–132. https://doi.org/10.18653/v1/D19-1012Google ScholarCross Ref
- Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692(2019).Google Scholar
- Yukun Ma, Khanh Linh Nguyen, Frank Z Xing, and Erik Cambria. 2020. A survey on empathetic dialogue systems. Information Fusion 64(2020), 50–70.Google ScholarCross Ref
- François Mairesse and Marilyn A Walker. 2011. Controlling user perceptions of linguistic style: Trainable generation of personality traits. Computational Linguistics 37, 3 (2011), 455–488.Google ScholarDigital Library
- Navonil Majumder, Pengfei Hong, Shanshan Peng, Jiankun Lu, Deepanway Ghosal, Alexander Gelbukh, Rada Mihalcea, and Soujanya Poria. 2020. MIME: MIMicking emotions for empathetic response generation. arXiv preprint arXiv:2010.01454(2020).Google Scholar
- Navonil Majumder, Soujanya Poria, Devamanyu Hazarika, Rada Mihalcea, Alexander Gelbukh, and Erik Cambria. 2019. Dialoguernn: An attentive rnn for emotion detection in conversations. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 6818–6825.Google ScholarDigital Library
- Robert P Marinier and John E Laird. 2007. Computational modeling of mood and feeling from emotion. In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol. 29.Google Scholar
- Naoki Masuyama and Chu Kiong Loo. 2015. Robotic emotional model with personality factors based on Pleasant-Arousal scaling model. In 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). IEEE, 19–24.Google ScholarDigital Library
- Naoki Masuyama, Chu Kiong Loo, and Manjeevan Seera. 2018. Personality affected robotic emotional model with associative memory for human-robot interaction. Neurocomputing 272(2018), 213–225.Google ScholarDigital Library
- John D Mayer. 2004. (2004) What is Emotional Intelligence?(2004).Google Scholar
- Albert Mehrabian. 1996. Analysis of the big-five personality factors in terms of the PAD temperament model. Australian journal of Psychology 48, 2 (1996), 86–92.Google ScholarCross Ref
- Albert Mehrabian. 1996. Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology 14, 4 (1996), 261–292.Google ScholarCross Ref
- Saif Mohammad. 2018. Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 english words. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 174–184.Google ScholarCross Ref
- Gregory Park, H Andrew Schwartz, Johannes C Eichstaedt, Margaret L Kern, Michal Kosinski, David J Stillwell, Lyle H Ungar, and Martin EP Seligman. 2015. Automatic personality assessment through social media language.Journal of personality and social psychology 108, 6(2015), 934.Google Scholar
- Sungjoon Park, Jiseon Kim, Seonghyeon Ye, Jaeyeol Jeon, Hee Young Park, and Alice Oh. 2019. Dimensional Emotion Detection from Categorical Emotion. arXiv preprint arXiv:1911.02499(2019).Google Scholar
- Tim Polzehl. 2016. PERSONALITY IN SPEECH.Springer.Google Scholar
- Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. 2018. Meld: A multimodal multi-party dataset for emotion recognition in conversations. arXiv preprint arXiv:1810.02508(2018).Google Scholar
- Soujanya Poria, Navonil Majumder, Rada Mihalcea, and Eduard Hovy. 2019. Emotion recognition in conversation: Research challenges, datasets, and recent advances. IEEE Access 7(2019), 100943–100953.Google ScholarCross Ref
- Paulo Quaglio. 2009. Television dialogue: The sitcom Friends vs. natural conversation. Amsterdam, NL.Google Scholar
- Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2018. Towards empathetic open-domain conversation models: A new benchmark and dataset. arXiv preprint arXiv:1811.00207(2018).Google Scholar
- James A Russell. 1991. Culture and the categorization of emotions.Psychological bulletin 110, 3 (1991), 426.Google Scholar
- James A Russell and Albert Mehrabian. 1977. Evidence for a three-factor theory of emotions. Journal of research in Personality 11, 3 (1977), 273–294.Google ScholarCross Ref
- H Andrew Schwartz, Johannes C Eichstaedt, Margaret L Kern, Lukasz Dziurzynski, Stephanie M Ramones, Megha Agrawal, Achal Shah, Michal Kosinski, David Stillwell, Martin EP Seligman, et al. 2013. Personality, gender, and age in the language of social media: The open-vocabulary approach. PloS one 8, 9 (2013), e73791.Google ScholarCross Ref
- Roman Shantala, Gennadiv Kyselov, and Anna Kyselova. 2018. Neural dialogue system with emotion embeddings. In 2018 IEEE First International Conference on System Analysis & Intelligent Computing (SAIC). IEEE, 1–4.Google ScholarCross Ref
- Ashish Sharma, Adam S Miner, David C Atkins, and Tim Althoff. 2020. A computational approach to understanding empathy expressed in text-based mental health support. arXiv preprint arXiv:2009.08441(2020).Google Scholar
- JiaXing Shen, Mingyu Derek Ma, Rong Xiang, Qin Lu, Elvira Perez Vallejos, Ge Xu, Chu-Ren Huang, and Yunfei Long. 2020. Dual memory network model for sentiment analysis of review text. Knowledge-Based Systems 188 (2020), 105004.Google ScholarDigital Library
- Xiaohui Song, Longtao Huang, Hui Xue, and Songlin Hu. 2022. Supervised prototypical contrastive learning for emotion recognition in conversation. arXiv preprint arXiv:2210.08713(2022).Google Scholar
- Xiao Sun, Xinmiao Chen, Zhengmeng Pei, and Fuji Ren. 2018. Emotional human machine conversation generation based on SeqGAN. In 2018 First Asian Conference on Affective Computing and Intelligent Interaction (ACII Asia). IEEE, 1–6.Google ScholarCross Ref
- Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: LIWC and computerized text analysis methods. Journal of language and social psychology 29, 1 (2010), 24–54.Google ScholarCross Ref
- Timothy J Trull, Marika B Solhan, Sarah L Tragesser, Seungmin Jahng, Phillip K Wood, Thomas M Piasecki, and David Watson. 2008. Affective instability: measuring a core feature of borderline personality disorder with ecological momentary assessment.Journal of abnormal psychology 117, 3 (2008), 647.Google Scholar
- A Wagner and E Briscoe. 2017. Psychological modeling of humans by assistive robots. In Human Modelling for Bio-Inspired Robotics. Elsevier, 273–296.Google Scholar
- Steffen Walter, Jonghwa Kim, David Hrabal, Stephen Clive Crawcour, Henrik Kessler, and Harald C Traue. 2013. Transsituational individual-specific biopsychological classification of emotions. IEEE Transactions on Systems, Man, and Cybernetics: Systems 43, 4(2013), 988–995.Google ScholarCross Ref
- Yan Wang, Jiayu Zhang, Jun Ma, Shaojun Wang, and Jing Xiao. 2020. Contextualized emotion recognition in conversation as sequence tagging. In Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue. 186–195.Google ScholarCross Ref
- Jason Wei and Kai Zou. 2019. Eda: Easy data augmentation techniques for boosting performance on text classification tasks. arXiv preprint arXiv:1901.11196(2019).Google Scholar
- Wei Wei, Jiayi Liu, Xianling Mao, Guibing Guo, Feida Zhu, Pan Zhou, and Yuchong Hu. 2019. Emotion-aware Chat Machine: Automatic Emotional Response Generation for Human-like Emotional Interaction. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 1401–1410.Google ScholarDigital Library
- Bernard L Welch. 1947. The generalization of STUDENT’S problem when several different population varlances are involved. Biometrika 34, 1-2 (1947), 28–35.Google ScholarCross Ref
- Zhiyuan Wen, Jiannong Cao, Ruosong Yang, Shuaiqi Liu, and Jiaxing Shen. 2021. Automatically Select Emotion for Response via Personality-affected Emotion Transition. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 5010–5020.Google ScholarCross Ref
- Zhiyuan Wen, Jiannong Cao, Ruosong Yang, and Senzhang Wang. 2020. Decode with template: Content preserving sentiment transfer. In Proceedings of the Twelfth Language Resources and Evaluation Conference. 4671–4679.Google Scholar
- Zhiyuan Wen, Jiannong Cao, Yu Yang, Haoli Wang, Ruosong Yang, and Shuaiqi Liu. 2023. DesPrompt: Personality-descriptive prompt tuning for few-shot personality recognition. Information Processing & Management 60, 5 (2023), 103422.Google ScholarDigital Library
- Tal Yarkoni. 2010. Personality in 100,000 words: A large-scale analysis of personality and word use among bloggers. Journal of research in personality 44, 3 (2010), 363–373.Google ScholarCross Ref
- Sayyed M Zahiri and Jinho D Choi. 2017. Emotion detection on tv show transcripts with sequence-based convolutional neural networks. arXiv preprint arXiv:1708.04299(2017).Google Scholar
- Rohola Zandie and Mohammad H Mahoor. 2020. Emptransfo: A multi-head transformer architecture for creating empathetic dialog systems. arXiv preprint arXiv:2003.02958(2020).Google Scholar
- Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. arXiv preprint arXiv:1703.10960(2017).Google Scholar
- Chujie Zheng, Yong Liu, Wei Chen, Yongcai Leng, and Minlie Huang. 2021. Comae: a multi-factor hierarchical framework for empathetic response generation. arXiv preprint arXiv:2105.08316(2021).Google Scholar
- Peixiang Zhong, Chen Zhang, Hao Wang, Yong Liu, and Chunyan Miao. 2020. Towards persona-based empathetic conversational models. arXiv preprint arXiv:2004.12316(2020).Google Scholar
- Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. 2018. Emotional chatting machine: Emotional conversation generation with internal and external memory. In Thirty-Second AAAI Conference on Artificial Intelligence.Google ScholarCross Ref
- Li Zhou, Jianfeng Gao, Di Li, and Heung-Yeung Shum. 2020. The design and implementation of xiaoice, an empathetic social chatbot. Computational Linguistics 46, 1 (2020), 53–93.Google ScholarDigital Library
Index Terms
- Personality-affected Emotion Generation in Dialog Systems
Recommendations
Quantifying the Links Between Personality Sub-traits and the Basic Emotions
Computational Science and Its Applications – ICCSA 2020AbstractThis article describes an exploratory study that aimed to analyse the relationship between personality traits and emotions. In particular, it investigates to what extent the sub-traits of the Five Factor Model has an empirically quantifiable ...
Towards Conversationally Intelligent Dialog Systems
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing SystemsSpoken dialog systems, lacking the means to address the complex phenomena of spontaneous speech and conversational dynamics, force users into a constrained mode of dialog that resembles text-based interaction more closely than spoken conversation. Turn-...
Are emoji valid indicators of in-the-moment mood?
AbstractDespite widespread assumptions that emoji represent emotions, research findings show that we do not process emoji in a way which we would expect for emotional stimuli. As such, we might be better placed to consider them more in line with mood ...
Highlights- The Emoji PANAS scale can reduce linguistic confounds when reporting current affective state
- Emoji are unlikely to be a valid means for reporting mood for those high in emotionality
- Emoji may be better consider mood indicators ...
Comments