Skip Abstract Section
Abstract
The identification of emotions is an open research area and has a potential leading role in the improvement of socio-emotional skills such as empathy, sensitivity, and emotion recognition in humans. The current study aimed to use Event Related Potential (ERP) components (N100, N200, P200, P300, early Late Positive Potential (LPP), middle LPP, and late LPP) of EEG data for the classification of emotional states (positive, negative, neutral). EEG data were collected from 62 healthy individuals over 18 electrodes. An emotional paradigm with pictures from the International Affective Picture System (IAPS) was used to record the EEG data. A linear Support Vector Machine (C=0.1) was used to classify emotions, and a forward feature selection approach was used to eliminate irrelevant features. The early LPP component, which was the most discriminative among all ERP components, had the highest classification accuracy (70.16%) for identifying negative and neutral stimuli. The classification of negative versus neutral stimuli had the best accuracy (79.84%) when all ERP components were used as a combined feature set, followed by positive versus negative stimuli (75.00%) and positive versus neutral stimuli (68.55%). Overall, the combined ERP component feature sets outperformed single ERP component feature sets for all stimulus pairings in terms of accuracy. These findings are promising for further research and development of EEG-based emotion recognition systems.
- Nandini K. Bhandari and Manish Jain. 2020. Emotion Recognition And Classification Using Eeg: A Review. International Journal of Scientific & Technology Research 9, 2 (February 2020), 1827–1836.Google Scholar
- William James. 2013. What is an Emotion?. Simon and Schuster.Google Scholar
- Carl Georg Lange.1912. The mechanism of the emotion. (B. Rand, Trans.). In Om sindsbevaegelser: Eine psycho-physiologische studie [On Emotions: A psycho-physiological study]. In B. Rand (Ed.)., The classical psychologists, 672-684. (Original work published 1885; translated 1912).Google Scholar
- Morteza Zangeneh Soroush, Keivan Maghooli, Seyed Kamaledin Setarehdan, and Ali Motie Nasrabadi. 2017. A Review on EEG Signals Based Emotion Recognition. International Clinical Neuroscience Journal 4, 4 (October 2017), 118–129. DOI:https://doi.org/10.15171/icnj.2017.01Google ScholarCross Ref
- Nazmi Sofian Suhaimi, James Mountstephens, and Jason Teo. 2020. EEG-Based Emotion Recognition: A State-of-the-Art Review of Current Trends and Opportunities. Computational Intelligence and Neuroscience 2020, (September 2020), 1–19. DOI:https://doi.org/10.1155/2020/8875426Google ScholarDigital Library
- Rania Alhalaseh and Suzan Alasasfeh. 2020. Machine-Learning-Based Emotion Recognition System Using EEG Signals. Computers 9, 4 (November 2020), 95. DOI:https://doi.org/10.3390/computers9040095Google ScholarCross Ref
- Felipe Zago Canal, Tobias Rossi Müller, Jhennifer Cristine Matias, Gustavo Gino Scotton, Antonio Reis de Sa Junior, Eliane Pozzebon, and Antonio Carlos Sobieranski. 2022. A survey on facial emotion recognition techniques: A state-of-the-art literature review. Information Sciences 582, (January 2022), 593–617. DOI:https://doi.org/10.1016/j.ins.2021.10.005Google ScholarDigital Library
- Tomasz Sapiński, Dorota Kamińska, Adam Pelikant, and Gholamreza Anbarjafari. 2019. Emotion Recognition from Skeletal Movements. Entropy 21, 7 (June 2019), 646. DOI:https://doi.org/10.3390/e21070646Google ScholarCross Ref
- Andrius Dzedzickis, Artūras Kaklauskas, and Vytautas Bucinskas. 2020. Human Emotion Recognition: Review of Sensors and Methods. Sensors (Basel, Switzerland) 20, 3 (January 2020), 542. DOI:https://doi.org/10.3390/s20030592Google ScholarCross Ref
- Essam H. Houssein, Asmaa Hammad, and Abdelmgeid A. Ali. 2022. Human emotion recognition from EEG-based brain–computer interface using machine learning: a comprehensive review. Neural Computing and Applications 34, (May 2022), 12527–12557. DOI:https://doi.org/10.1007/s00521-022-07292-4Google ScholarDigital Library
- Haoran Liu, Ying Zhang, Yujun Li, and Xiangyi Kong. 2021. Review on Emotion Recognition Based on Electroencephalography. Frontiers in Computational Neuroscience 15, (October 2021), 758212. DOI:https://doi.org/10.3389/fncom.2021.758212Google ScholarCross Ref
- Jonas K. Olofsson, Steven Nordin, Henrique Sequeira, and John Polich. 2008. Affective picture processing: An integrative review of ERP findings. Biological Psychology 77, 3 (March 2008), 247–265. DOI:https://doi.org/10.1016/j.biopsycho.2007.11.006Google ScholarCross Ref
- M. Jehna, C. Neuper, A. Ischebeck, M. Loitfelder, S. Ropele, C. Langkammer, F. Ebner, S. Fuchs, R. Schmidt, F. Fazekas, and C. Enzinger. 2011. The functional correlates of face perception and recognition of emotional facial expressions as evidenced by fMRI. Brain Research 1393, (June 2011), 73–83. DOI:https://doi.org/10.1016/j.brainres.2011.04.007Google ScholarCross Ref
- Mojtaba Khomami Abadi, Ramanathan Subramanian, Seyed Mostafa Kia, Paolo Avesani, Ioannis Patras, and Nicu Sebe. 2015. DECAF: MEG-Based Multimodal Database for Decoding Affective Physiological Responses. IEEE Transactions on Affective Computing 6, 3 (2015), 209–222. DOI:https://doi.org/10.1109/TAFFC.2015.2392932Google ScholarDigital Library
- Bahar Güntekin and Erol Başar. 2014. A review of brain oscillations in perception of faces and emotional pictures. Neuropsychologia 58, (May 2014), 33–51. DOI:https://doi.org/10.1016/j.neuropsychologia.2014.03.014Google ScholarCross Ref
- Martin Eimer and Amanda Holmes. 2007. Event-related brain potential correlates of emotional face processing. Neuropsychologia 45, 1 (2007), 15–31. DOI:https://doi.org/10.1016/j.neuropsychologia.2006.04.022Google ScholarCross Ref
- Tracy A. Dennis and Greg Hajcak. 2009. The late positive potential: a neurophysiological marker for emotion regulation in children. Journal of Child Psychology and Psychiatry 50, 11 (November 2009), 1373–1383. DOI:https://doi.org/10.1111/j.1469-7610.2009.02168.xGoogle ScholarCross Ref
- Mizhi Hua, Zhuo Rachel Han, Siyi Chen, Meng Yang, Renlai Zhou, and Sengqi Hu. 2014. Late positive potential (LPP) modulation during affective picture processing in preschoolers. Biological Psychology 101, (September 2014), 77–81. DOI:https://doi.org/10.1016/j.biopsycho.2014.06.006Google ScholarCross Ref
- Greg Hajcak, Annmarie MacNamara, and Doreen M. Olvet. 2010. Event-Related Potentials, Emotion, and Emotion Regulation: An Integrative Review. Developmental Neuropsychology 35, 2 (February 2010), 129–155. DOI:https://doi.org/10.1080/87565640903526504Google ScholarCross Ref
- Maxime Résibois, Philippe Verduyn, Pauline Delaveau, Jean-Yves Rotgé, Peter Kuppens, Iven Van Mechelen, and Philippe Fossati. 2017. The neural basis of emotions varies over time: different regions go with onset- and offset-bound processes underlying emotion intensity. Social cognitive and affective neuroscience 12, 8 (2017), 1261–1271. DOI:https://doi.org/10.1093/scan/nsx051Google ScholarCross Ref
- Christos A Frantzidis, Charalampos Bratsas, Christos L Papadelis, Evdokimos Konstantinidis, Costas Pappas, and Panagiotis D Bamidis. 2010. Toward Emotion Aware Computing: An Integrated Approach Using Multichannel Neurophysiological Recordings and Affective Visual Stimuli. IEEE Transactions on Information Technology in Biomedicine 14, 3 (May 2010), 589–597. DOI:https://doi.org/10.1109/titb.2010.2041553Google ScholarDigital Library
- Ahmad Tauseef Sohaib, Shahnawaz Qureshi, Johan Hagelbäck, Olle Hilborn, and Petar Jerčić. 2013. Evaluating Classifiers for Emotion Recognition Using EEG. Foundations of Augmented Cognition (2013), 492–501. DOI:https://doi.org/10.1007/978-3-642-39454-6_53Google ScholarCross Ref
- Mostafa Mohammadpour, Seyyed Mohammad Reza Hashemi, and Negin Houshmand. 2017. Classification of EEG-based emotion for BCI applications. 2017 Artificial Intelligence and Robotics (IRANOPEN) (April 2017). DOI:https://doi.org/10.1109/rios.2017.7956455Google ScholarCross Ref
- Haiyan Xu and Konstantinos N. Plataniotis. 2012. Affect recognition using EEG signal. 2012 IEEE 14th International Workshop on Multimedia Signal Processing (MMSP) (September 2012). DOI:https://doi.org/10.1109/mmsp.2012.6343458Google ScholarCross Ref
- N. G. Mathieu, S. Bonnet, S. Harquel, E. Gentaz, and A. Campagne. 2013. Single-trial ERP classification of emotional processing. 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER) 2013, (November 2013), 101-104. DOI:https://doi.org/10.1109/ner.2013.6695881Google ScholarCross Ref
- Jingjing Tong, Shuang Liu, Yufeng Ke, Bin Gu, Feng He, Baikun Wan, and Dong Ming. 2017. EEG-based emotion recognition using nonlinear feature. 2017 IEEE 8th International Conference on Awareness Science and Technology (iCAST) 2017, (November 2017), 55-59. DOI:https://doi.org/10.1109/icawst.2017.8256518Google ScholarCross Ref
- Soraia M. Alarcao and Manuel J. Fonseca. 2019. Emotions Recognition Using EEG Signals: A Survey. IEEE Transactions on Affective Computing 10, 3 (July 2019), 374–393. DOI:https://doi.org/10.1109/taffc.2017.2714671Google ScholarDigital Library
- Jiang Wang and Mei Wang. 2021. Review of the emotional feature extraction and classification using EEG signals. Cognitive Robotics 1, (2021), 29–40. DOI:https://doi.org/10.1016/j.cogr.2021.04.001Google ScholarCross Ref
- Md. Mustafizur Rahman, Ajay Krishno Sarkar, Md. Amzad Hossain, Md. Selim Hossain, Md. Rabiul Islam, Md. Biplob Hossain, Julian M.W. Quinn, and Mohammad Ali Moni. 2021. Recognition of human emotions using EEG signals: A review. Computers in Biology and Medicine 136, (September 2021), 104696. DOI:https://doi.org/10.1016/j.compbiomed.2021.104696Google ScholarDigital Library
- Vempati, Raveendrababu, and Lakhan Dev Sharma. 2023. A Systematic Review on Automated Human Emotion Recognition using Electroencephalogram Signals and Artificial Intelligence. Results in Engineering, 2023, 101027.Google Scholar
- Moon Inder Singh and Mandeep Singh. 2017. Development of a real time emotion classifier based on evoked EEG. Biocybernetics and Biomedical Engineering 37, 3 (2017), 498–509. DOI:https://doi.org/10.1016/j.bbe.2017.05.004Google ScholarCross Ref
- Danny Plass-Oude Bos. 2006. EEG-based emotion recognition. The influence of visual and auditory stimuli, 56,3, 1-17Google Scholar
- Yi-Hung Liu, Wei-Teng Cheng, Yu-Tsung Hsiao, Chien-Te Wu, and Mu-Der Jeng. 2014. EEG-based emotion recognition based on kernel Fisher’s discriminant analysis and spectral powers. 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC) 2014, (October 2014), 2221-2225. DOI:https://doi.org/10.1109/smc.2014.6974254Google ScholarCross Ref
- Aayush Bhardwaj, Ankit Gupta, Pallav Jain, Asha Rani, and Jyoti Yadav. 2015. Classification of human emotions from EEG signals using SVM and LDA Classifiers. 2015 2nd International Conference on Signal Processing and Integrated Networks (SPIN) 2015, (February 2015), 180-185. DOI:https://doi.org/10.1109/spin.2015.7095376Google ScholarCross Ref
- R. E. J. Yohanes, Wee Ser, and Guang-bin Huang. 2012. Discrete Wavelet Transform coefficients for emotion recognition from EEG signals. 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society 2012, (August 2012), 2251-2254. DOI:https://doi.org/10.1109/embc.2012.6346410Google ScholarCross Ref
- Seyyed Abed Hosseini and Mohammad Bagher Naghibi-Sistani. 2011. Emotion recognition method using entropy analysis of EEG signals. International Journal of Image, Graphics and Signal Processing 3, 5 (August 2011), 30–36. DOI:https://doi.org/10.5815/ijigsp.2011.05.05Google ScholarCross Ref
- Pragati Patel, Raghunandan R, and Ramesh Naidu Annavarapu. 2021. EEG-based human emotion recognition using entropy as a feature extraction measure. Brain Informatics 8, 1 (October 2021). DOI:https://doi.org/10.1186/s40708-021-00141-5Google ScholarCross Ref
- Yisi Liu and Olga Sourina. 2014. Real-Time Subject-Dependent EEG-Based Emotion Recognition Algorithm. Transactions on Computational Science XXIII (2014), 199–223. DOI:https://doi.org/10.1007/978-3-662-43790-2_11Google ScholarCross Ref
- Katerina Giannakaki, Giorgos Giannakakis, Christina Farmaki, and Vangelis Sakkalis. 2017. Emotional State Recognition Using Advanced Machine Learning Techniques on EEG Data. 2017 IEEE 30th International Symposium on Computer-Based Medical Systems (CBMS) (June 2017). DOI:https://doi.org/10.1109/cbms.2017.156Google ScholarCross Ref
- Andrea Apicella, Pasquale Arpaia, Giovanna Mastrati, and Nicola Moccaldi. 2021. EEG-based detection of emotional valence towards a reproducible measurement of emotions. Scientific Reports 11, 1 (November 2021). DOI:https://doi.org/10.1038/s41598-021-00812-7Google ScholarCross Ref
- Peter J. Lang, Margaret M. Bradley, and Bruce N. Cuthbert. 2008. International Affective Picture System (IAPS): Instruction manual and affective ratings, Technical Report A-8. Gainesville: The Center for Research in Psychophysiology, University of Florida.Google Scholar
- Bahar Güntekin and Elif Tülay. 2014. Event related beta and gamma oscillatory responses during perception of affective pictures. Brain Research 1577, (August 2014), 45–56. DOI:https://doi.org/10.1016/j.brainres.2014.06.029Google ScholarCross Ref
- Sebastiaan Mathôt, Daniel Schreij, and Jan Theeuwes. 2011. OpenSesame: An open-source, graphical experiment builder for the social sciences. Behavior Research Methods 44, 2 (November 2011), 314–324. DOI:https://doi.org/10.3758/s13428-011-0168-7Google ScholarCross Ref
- Margaret M. Bradley and Peter J. Lang. 1994. Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry 25, 1 (March 1994), 49–59. DOI:https://doi.org/10.1016/0005-7916(94)90063-9Google ScholarCross Ref
- Katie E. Garrison, Adrienne L. Crowell, Anna J. Finley, and Brandon J. Schmeichel. 2017. Effects of prior mental effort on picture processing: An ERP investigation. Psychophysiology 54, 11 (July 2017), 1714–1725. DOI:https://doi.org/10.1111/psyp.12914Google ScholarCross Ref
- Xinmei Deng, Biao Sang, Yixuan Ku, and Liyang Sai. 2019. Age-Related Differences in the Late Positive Potential during Emotion Regulation between Adolescents and Adults. Scientific Reports 9, 1 (April 2019). DOI:https://doi.org/10.1038/s41598-019-42139-4Google ScholarCross Ref
- P. Pudil, J. Novovičová, and J. Kittler. 1994. Floating search methods in feature selection. Pattern Recognition Letters 15, 11 (November 1994), 1119–1125. DOI:https://doi.org/10.1016/0167-8655(94)90127-9Google ScholarDigital Library
- Stephen B. R. E. Brown, Henk van Steenbergen, Guido P. H. Band, Mischa de Rover, and Sander Nieuwenhuis. 2012. Functional significance of the emotion-related late positive potential. Frontiers in Human Neuroscience 6, (2012),33. DOI:https://doi.org/10.3389/fnhum.2012.00033Google ScholarCross Ref
- Raja Majid Mehmood and Hyo Jong Lee. 2016. A novel feature extraction method based on late positive potential for emotion recognition in human brain signal patterns. Computers & Electrical Engineering 53, (July 2016), 444–457. DOI:https://doi.org/10.1016/j.compeleceng.2016.04.009Google ScholarDigital Library
- Hans Revers, Katrijn Van Deun, Wim Strijbosch, Jean Vroomen, and Marcel Bastiaansen. 2022. Decoding the neural responses to experiencing disgust and sadness. Brain Research 1793, (October 2022), 148034. DOI:https://doi.org/10.1016/j.brainres.2022.148034Google ScholarCross Ref
- Moon Inder Singh and Mandeep Singh. 2021. Emotion Recognition: An Evaluation of ERP Features Acquired from Frontal EEG Electrodes. Applied Sciences 11, 9 (April 2021), 4131. DOI:https://doi.org/10.3390/app11094131Google ScholarCross Ref
- Moon Inder Singh and Mandeep Singh. 2020. Development of emotion classifier based on absolute and differential attributes of averaged signals of visually stimulated event related potentials. Transactions of the Institute of Measurement and Control 42, 11 (March 2020), 2057–2067. DOI:https://doi.org/10.1177/0142331220904889Google ScholarCross Ref
- Christos A. Frantzidis, Chrysa D. Lithari, Ana B. Vivas, Christos L. Papadelis, Costas Pappas, and Panagiotis D. Bamidis. 2008. Towards emotion aware computing: A study of arousal modulation with multichannel event-related potentials, delta oscillatory activity and skin conductivity responses. 2008 8th IEEE International Conference on BioInformatics and BioEngineering (October 2008). DOI:https://doi.org/10.1109/bibe.2008.4696823Google ScholarCross Ref
- Mandeep Singh, Mooninder Singh, and Nikhil Singhal. 2013. Emotion Recognition along Valence Axis Using Naïve Bayes Classifier. International Journal of Information Technology & Knowledge Management 7, (2013), 51–55.Google Scholar
- Mandeep Singh, Mooninder Singh, and Ankita Sandel. 2014. Emotion Classification along Valence Axis Using Averaged ERP Signals. International Journal of Information Technology & Knowledge Management 7, (2014), 153–161.Google Scholar
- Money Goyal, Mooninder Singh, and Mandeep Singh. 2015. Classification of emotions based on ERP feature extraction. 2015 1st International Conference on Next Generation Computing Technologies (NGCT) 2015, (September 2015), 660 -662. DOI:https://doi.org/10.1109/ngct.2015.7375203Google ScholarCross Ref
- Greg Hajcak, Anna Weinberg, Annmarie MacNamara, and Dan Foti. 2011. ERPs and the Study of Emotion. Oxford University Press. In EJ Kappenman, SJ Luck (Editors). The Oxford Handbook of Event-Related Potential Components: Oxford University Press, 2011. pp.516-554. DOI:https://doi.org/10.1093/oxfordhb/9780195374148.013.0222Google ScholarCross Ref
- Jeff P Hamm, Blake W Johnson, and Ian J Kirk. 2002. Comparison of the N300 and N400 ERPs to picture stimuli in congruent and incongruent contexts. Clinical Neurophysiology 113, 8 (August 2002), 1339–1350. DOI:https://doi.org/10.1016/s1388-2457(02)00161-xGoogle ScholarCross Ref
- Rui Ding, Ping Li, Wei Wang, and Wenbo Luo. 2017. Emotion Processing by ERP Combined with Development and Plasticity. Neural Plasticity 2017, (2017), 1–15. DOI:https://doi.org/10.1155/2017/5282670Google ScholarCross Ref
- Bahar Güntekin, Banu Femir, Bilge Turp Gölbaşı, Elif Tülay, and Erol Başar. 2017. Affective pictures processing is reflected by an increased long-distance EEG connectivity. Cognitive Neurodynamics 11, 4 (April 2017), 355–367. DOI:https://doi.org/10.1007/s11571-017-9439-zGoogle ScholarCross Ref
- Jianhai Zhang, Hongyan Xu, Li Zhu, Wanzeng Kong, and Zheng Ma. 2019. Gender recognition in emotion perception using EEG features. 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) 2019, (November 2019), 2883-2887. DOI:https://doi.org/10.1109/bibm47256.2019.8983332Google ScholarCross Ref
- Yasser F. Alharbi and Yousef A. Alotaibi. 2021. The Correlate of Emotion and Gender Classification Using EEG Signals. 2021 IEEE 6th International Conference on Signal and Image Processing (ICSIP) 2021, (October 2021), 790-794. DOI:https://doi.org/10.1109/icsip52628.2021.9688884Google ScholarCross Ref
Index Terms
- Decoding Functional Brain Data for Emotion Recognition: A Machine Learning Approach
Recommendations
Building predictive models of emotion with functional near-infrared spectroscopy
Goal of this work is to classify affective states of Valence and Arousal using functional Near Infra-Red Spectroscopy.Use Russel's circumplex model as the representation of affect.Suggest novel method of using Brain ``Regions of Interest as a feature ...
Application of Support Vector Machine for Emotion Classification
ICGEC '12: Proceedings of the 2012 Sixth International Conference on Genetic and Evolutionary ComputingEmotions are a great source of information in communication and interaction among people. There is a continuous interaction between emotions, thoughts and behavior, in such a way that they constantly influence each other. in this paper, we propose an ...
Decoding Emotion Dimensions Arousal and Valence Elicited on EEG Responses to Videos and Images: A Comparative Evaluation
Brain InformaticsAbstractThis study aims to compare the automatic classification of emotions based on the self-reported level of arousal and valence with the Self-Assessment Manikin (SAM) when subjects were exposed to videos or images. The classification is performed on ...
Comments