Skip to main content
SpringerLink
Log in

Multiple-source distribution deep adaptive feature norm network for EEG emotion recognition

  • Research Article
  • Published:
Cognitive Neurodynamics Aims and scope Submit manuscript

Abstract

Electroencephalogram (EEG) emotion recognition plays an important role in human–computer interaction, and a higher recognition accuracy can improve the user experience. In recent years, domain adaptive methods in transfer learning have been used to construct a general emotion recognition model to deal with domain difference among different subjects and sessions. However, it is still challenging to effectively reduce domain difference in domain adaptation. In this paper, we propose a Multiple-Source Distribution Deep Adaptive Feature Norm Network for EEG emotion recognition, which reduce domain difference by improving the transferability of task-specific features. In detail, the domain adaptive method of our model employs a three-layer network topology, inserts Adaptive Feature Norm to self-supervised adjustment between different layers, and combines a multiple-kernel selection approach to mean embedding matching. The method proposed in this paper achieves the best classification performance in the SEED and SEED-IV datasets. In SEED dataset, the average accuracy of cross-subject and cross-session experiments is 85.01 and 91.93%, respectively. In SEED-IV dataset, the average accuracy is 58.81% in cross-subject experiments and 59.51% in cross-session experiments. The experimental results demonstrate that our method can effectively reduce the domain difference and improve the emotion recognition accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

The datasets analyzed during the current study are available in the SEED repository, https://bcmi.sjtu.edu.cn/ *seed/index.html.

References

  • Ahmed F, Bari ASMH, Gavrilova ML (2019) Emotion recognition from body movement. IEEE Access 8:11761–11781

    Article  Google Scholar 

  • Borgwardt KM, Gretton A, Rasch MJ et al (2006) Integrating structured biological data by Kernel Maximum Mean Discrepancy. Bioinformatics 22(14):e49–e57

    Article  CAS  PubMed  Google Scholar 

  • Bozhokin SV, Suslova IB (2015) Wavelet-based analysis of spectral rearrangements of EEG patterns and of non-stationary correlations[J]. Physica A 421:151–160

    Article  MathSciNet  Google Scholar 

  • Chai X, Wang Q, Zhao Y et al (2016) Unsupervised domain adaptation techniques based on auto-encoder for non-stationary EEG-based emotion recognition[J]. Comput Biol Med 79:205–214

    Article  PubMed  Google Scholar 

  • Chen H, Jin M, Li Z et al (2021) MS-MDA: Multisource Marginal Distribution Adaptation for Cross-subject and Cross-session EEG Emotion Recognition [J]. Front Neurosci 15:778488

    Article  PubMed  PubMed Central  Google Scholar 

  • Chen H, Li Z, Jin M, et al (2021) MEERNet: multi-source EEG-based emotion recognition network for generalization across subjects and sessions[C]//2021 In: 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, pp 6094–6097

  • Cowie R, Douglas-Cowie E, Tsapatsoulis N et al (2001) Emotion recognition in human-computer interaction [J]. IEEE Signal Process Mag 18(1):32–80

    Article  ADS  Google Scholar 

  • Dolan RJ (2002) Emotion, cognition, and behavior [J]. Science 298(5596):1191–1194

    Article  ADS  CAS  PubMed  Google Scholar 

  • Duan RN, Zhu JY, Lu BL (2013) Differential entropy feature for EEG-based emotion classification[C]//2013 In: 6th International IEEE/EMBS Conference on Neural Engineering (NER). IEEE, pp 81–84

  • Feng X, Cong P, Dong L et al (2023) Channel attention convolutional aggregation network based on video-level features for EEG emotion recognition [J]. Cognit Neurodynamics. https://doi.org/10.1007/s11571-023-10034-4

    Article  Google Scholar 

  • Ganin Y, Ustinova E, Ajakan H et al (2016) Domain-adversarial training of neural networks [J]. J Mach Learn Res 17(1):1–35

    MathSciNet  Google Scholar 

  • Ghare PS, Paithane AN (2016) Human emotion recognition using non-linear and non-stationary EEG signal[C]//2016 In: International Conference on Automatic Control and Dynamic Optimization Techniques (ICACDOT). IEEE, pp 1013–1016

  • Gu X, Cai W, Gao M et al (2022) Multi-source domain transfer discriminative dictionary learning modeling for electroencephalogram-based emotion recognition [J]. IEEE Trans Comput Soc Syst 9(6):1604–1612

    Article  Google Scholar 

  • Hang W, Feng W, Du R et al (2019) Cross-subject EEG signal recognition using deep domain adaptation network[J]. IEEE Access 7:128273–128282

    Article  Google Scholar 

  • Hinton G, van der Maaten L (2008) Visualizing data using t-SNE. J Mach Learn Res [J] 9:2579

    Google Scholar 

  • Imran A, Athitsos V M (2021) Adaptive feature norm for unsupervised subdomain adaptation [C]//International Symposium on Visual Computing. Springer, Cham, pp 341-352

  • Jin YM, Luo YD, Zheng WL, et al (2017) EEG-based emotion recognition using domain adaptation network[C]//2017 In: international conference on orange technologies (ICOT). IEEE, pp 222–225

  • Kingma DP, Ba J (2014) Adam: a method for stochastic optimization [J]. arXiv preprint arXiv:1412.6980

  • LiH, Jin YM, Zheng WL, et al (2018) Cross-subject emotion recognition using deep adaptation networks[C]//In: International conference on neural information processing. Springer, Cham, pp 403–413.

  • Liu ZT, Xie Q, Wu M et al (2018) Speech emotion recognition based on an improved brain emotion learning model [J]. Neurocomputing 309:145–156

    Article  Google Scholar 

  • Long M, Wang J (2015) Learning transferable features with deep adaptation networks[C]// JMLR.org. JMLR.org

  • Lotte F, Guan C (2010) Learning from other subjects helps reducing brain-computer interface calibration time[C]//2010 In: IEEE International conference on acoustics, speech and signal processing. IEEE, pp 614–617

  • Mithbavkar SA, Shah MS (2021) Analysis of EMG based emotion recognition for multiple people and emotions[C]//2021 In: IEEE 3rd Eurasia Conference on Biomedical Engineering, Healthcare and Sustainability (ECBIOS). IEEE, pp 1–4

  • Shin Y, Lee S, Ahn M et al (2015) Noise robustness analysis of sparse representation based classification method for non-stationary EEG signal classification[J]. Biomed Signal Process Control 21:8–18

    Article  Google Scholar 

  • Tzeng E, Hoffman J, Zhang N, et al (2014) Deep domain confusion: maximizing for domain invariance [J]. arXiv preprint arXiv:1412.3474

  • Tzeng E, Hoffman J, Saenko K, et al (2017) Adversarial discriminative domain adaptation [C]//In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 7167–7176

  • Wu G, Liu G, Hao M (2010) The analysis of emotion recognition from GSR based on PSO[C]//2010 In: International symposium on intelligence information processing and trusted computing. IEEE, pp 360–363

  • Xu R, Li G, Yang J, et al (2018) Larger norm more transferable: an adaptive feature norm approach for unsupervised domain adaptation[J]

  • Yosinski J, Clune J, Bengio Y, et al (2014) How transferable are features in deep neural networks? [J]. MIT Press

  • Zhao L M, Yan X, Lu B L (2021) Plug-and-play domain adaptation for cross-subject EEG-based emotion recognition[C]//In: Proceedings of the AAAI Conference on Artificial Intelligence., vol 35(1): pp 863–870

  • Zheng WL, Lu BL (2015) Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks[J]. IEEE Trans Auton Ment Dev 7(3):162–175

    Article  Google Scholar 

  • Zheng WL, Liu W, Lu Y et al (2018) Emotionmeter: a multimodal framework for recognizing human emotions[J]. IEEE Trans Cybern 49(3):1110–1122

    Article  PubMed  Google Scholar 

  • Zheng W L, Zhang Y Q, Zhu J Y, et al (2015) Transfer components between subjects for EEG-based emotion recognition[C]//2015 In: international conference on affective computing and intelligent interaction (ACII). IEEE, pp 917–922

  • Zhou R, Zhang Z, Fu H, et al (2023) PR-PL: a novel prototypical representation based pairwise learning framework for emotion recognition using eeg signals[J]. IEEE Transactions on Affective Computing

Download references

Acknowledgements

This work was supported by the Key Research and Development Project of Zhejiang Province (2020C04009) and Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province (2020E10010)

Author information

Authors and Affiliations

  1. School of Automation, Hangzhou Dianzi University, Hangzhou, 310000, China

    Lei Zhu, Fei Yu, Wangpan Ding, Aiai Huang & Nanjiao Ying

  2. School of Computer Science, Hangzhou Dianzi University, Hangzhou, 310000, China

    Jianhai Zhang

Authors
  1. Lei Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fei Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wangpan Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aiai Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nanjiao Ying
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jianhai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lei Zhu.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, L., Yu, F., Ding, W. et al. Multiple-source distribution deep adaptive feature norm network for EEG emotion recognition. Cogn Neurodyn (2024). https://doi.org/10.1007/s11571-024-10092-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11571-024-10092-2

Keywords

Search

Navigation