Skip to main content
SpringerLink
Log in

An improved CapsNet based on data augmentation for driver vigilance estimation with forehead single-channel EEG

  • Research Article
  • Published:
Cognitive Neurodynamics Aims and scope Submit manuscript

Abstract

Various studies have shown that it is necessary to estimate the drivers’ vigilance to reduce the occurrence of traffic accidents. Most existing EEG-based vigilance estimation studies have been performed on intra-subject and multi-channel signals, and these methods are too costly and complicated to implement in practice. Hence, aiming at the problem of cross-subject vigilance estimation of single-channel EEG signals, an estimation algorithm based on capsule network (CapsNet) is proposed. Firstly, we propose a new construction method of the input feature maps to fit the characteristics of CapsNet to improve the algorithm accuracy. Meanwhile, the self-attention mechanism is incorporated in the algorithm to focus on the key information in feature maps. Secondly, we propose substituting the traditional multi-channel signals with the single-channel signals to improve the utility of algorithm. Thirdly, since the single-channel signals carry fewer dimensions of the information compared to the multi-channel signals, we use the conditional generative adversarial network to improve the accuracy of single-channel signals by increasing the amount of data. The proposed algorithm is verified on the SEED-VIG, and Root-mean-square-error (RMSE) and Pearson Correlation Coefficient (PCC) are used as the evaluation metrics. The results show that the proposed algorithm improves the computing speed while the RMSE is reduced by 3%, and the PCC is improved by 12% compared to the mainstream algorithm. Experiment results prove the feasibility of using forehead single-channel EEG signals for cross-subject vigilance estimation and offering the possibility of lightweight EEG vigilance estimation devices for practical applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. https://www.smivision.com/eye-tracking/products/mobile-eye-tracking/.

  2. https://figshare.com/articles/dataset/Multi-channel_EEG_recordings_during_a_sustained-attention_driving_task_preprocessed_dataset_/7666055/3.

References

  • Alioua N, Amine A, Rziza M (2014) Driver’s fatigue detection based on yawning extraction. Int J Veh Technol 2014:47–75

    Google Scholar 

  • Bergasa LM, Nuevo J, Sotelo MA, Vhzquez M (2006) Real-time system for monitoring driver vigilance. IEEE Trans Intell Transp Syst 7:63–77

    Article  Google Scholar 

  • Buendia R, Forcolin F, Karlsson J, Sjöqvist BA, Anund A, Candefjord S (2019) Deriving heart rate variability indices from cardiac monitoring-an indicator of driver sleepiness. Traffic Inj Prev 20:249–254

    Article  PubMed  Google Scholar 

  • Cao Z, Chuang C-H, King J-K, Lin C-T (2019) Multi-channel eeg recordings during a sustained-attention driving task. Sci Data

  • Chao H, Dong L, Liu Y, Lu B (2019) Emotion recognition from multiband eeg signals using capsnet. Sensors (Switzerland) 19

  • Chen S, Kaili X, Yao X, Ge J, Li L, Zhu S, Li Z (2021) Information fusion and multi-classifier system for miner fatigue recognition in plateau environments based on electrocardiography and electromyography signals. Comput Methods Programs Biomed 211:106451

    Article  PubMed  Google Scholar 

  • Chuang CH, Ko LW, Lin YP, Jung TP, Lin CT (2014) Independent component ensemble of eeg for brain-computer interface. IEEE Trans Neural Syst Rehabil Eng 22:230–238

    Article  PubMed  Google Scholar 

  • Dong Y, Hu Z, Uchimura K, Murayama N (2011) Driver inattention monitoring system for intelligent vehicles: A review. IEEE Trans Intell Transp Syst 12:596–614

    Article  Google Scholar 

  • Flores MJ, Armingol JM, Adl Escalera (2010) Driver drowsiness warning system using visual information for both diurnal and nocturnal illumination conditions. In: Ad Hoc networks

  • Gao Z, Wang X, Yang Y, Chaoxu M, Cai Q, Dang W, Zuo S (2019) Eeg-based spatio-temporal convolutional neural network for driver fatigue evaluation. IEEE Trans Neural Netw Learn Syst 30:2755–2763

    Article  PubMed  Google Scholar 

  • Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Neural information processing systems

  • Guarda L, Tapia J, Droguett EL, Ramos M (2022) A novel capsule neural network based model for drowsiness detection using electroencephalography signals. Expert Syst Appl 201

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778

  • Jianchao L, Zheng X, Tang L, Zhang T, Sheng QZ, Wang C, Jin J, Shui Yu, Zhou W (2021) Can steering wheel detect your driving fatigue? IEEE Trans Veh Technol 70:5537–5550

    Article  Google Scholar 

  • Jiang Y, Zhang Y, Lin C, Dongrui W, Lin C-T (2021) Eeg-based driver drowsiness estimation using an online multi-view and transfer tsk fuzzy system. IEEE Trans Intell Transp Syst 22(3):1752–1764

    Article  Google Scholar 

  • Jiao Y, Deng Y, Luo Y, Lu BL (2020) Driver sleepiness detection from eeg and eog signals using gan and lstm networks. Neurocomputing 408:100–111

    Article  Google Scholar 

  • Kamakura Y, Ohsuga M, Inoue Y, Noguchi Y (2007) Classification of blink waveforms towards the assessment of driver’s arousal level. Trans Soc Autom Engineers Jpn 38:173–178

    Google Scholar 

  • Kingma D, Ba J (2014) Adam: a method for stochastic optimization. Comput Sci

  • Ko LW, Komarov O, Lai WK, Liang WG, Jung TP (2020) Eyeblink recognition improves fatigue prediction from single-channel forehead eeg in a realistic sustained attention task. J Neural Eng 17:036015 ((12pp))

    Article  PubMed  Google Scholar 

  • Kong W, Zhou Z, Jiang B, Babiloni F, Borghini G (2017) Assessment of driving fatigue based on intra/inter-region phase synchronization. Neurocomputing 219:474–482

    Article  Google Scholar 

  • Ko W, Oh K, Jeon E, Suk H-I (2020) Vignet: a deep convolutional neural network for eeg-based driver vigilance estimation. In: 2020 8th International Winter Conference on Brain-Computer Interface (BCI), pp 1–3

  • Liu Y, Lan Z, Cui J, Sourina O, Muller-Wittig W (2019) Eeg-based cross-subject mental fatigue recognition. In: Proceedings—2019 international conference on cyberworlds, CW 2019, pp 247–252

  • Lu BL, Li H, Zheng WL (2018) Multimodal vigilance estimation with adversarial domain adaptation networks. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp 1–6

  • Luo H, Qiu T, Liu C, Huang P (2019) Research on fatigue driving detection using forehead eeg based on adaptive multi-scale entropy. Biomed Signal Process Control 51:50–58

    Article  Google Scholar 

  • Maaten L, Geoffrey H (2008) Visualizing data using t-sne. J Mach Learn Res 9:2579–2605

    Google Scholar 

  • Ma B, Li H, Luo Y, Lu BL (2019) Depersonalized cross-subject vigilance estimation with adversarial domain generalization. In: 2019 International joint conference on neural networks (IJCNN), pp 1–8

  • Maskeliunas R, Damasevicius R, Martisius I, Vasiljevas M (2016) Consumer-grade eeg devices: Are they usable for control tasks? PeerJ

  • Murphy-Chutorian E, Trivedi MM (2010) Head pose estimation and augmented reality tracking: an integrated system and evaluation for monitoring driver awareness. IEEE Trans Intell Transp Syst 11:300–311

    Article  Google Scholar 

  • Naeije M, Zorn H (1982) Relation between emg power spectrum shifts and muscle fibre action potential conduction velocity changes during local muscular fatigue in man. Eur J Appl Physiol 50:23–33

    Article  Google Scholar 

  • Pei Z, Wang H, Bezerianos A, Li J (2021) Eeg-based multiclass workload identification using feature fusion and selection. IEEE Trans Instru Measure 70:4001108

    Article  Google Scholar 

  • Ratti E, Waninger S, Berka C, Ruffini G, Verma A (2017) Comparison of medical and consumer wireless eeg systems for use in clinical trials. Front Human Neurosci 11

  • Rogado E, García JL, Barea R, Bergasa LM, López E (2009) Driver fatigue detection system. In: 2008 IEEE International Conference on Robotics and Biomimetics, ROBIO 2008, pp 1105–1110

  • Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. Proc Adv Neural Inf Process Syst 3856–3866

  • Sikander G, Anwar S (2019) Driver fatigue detection systems: a review. IEEE Trans Intell Transp Syst 20:2339–2352

    Article  Google Scholar 

  • Trutschel U, Sirois B, Sommer D, Golz M, Edwards D (2011) Perclos: an alertness measure of the past. In: Driving Assessment: International Driving Symposium on Human Factors in Driver Assessment

  • Tuncer T, Dogan S, Ertam F, Subasi A (2021) A dynamic center and multi threshold point based stable feature extraction network for driver fatigue detection utilizing eeg signals. Cogn Neurodyn 15:223–237

    Article  PubMed  Google Scholar 

  • Vaswani A, Brain G, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need

  • Wei W, JonathanWu QM, Sun W, Yang Y, Yuan X, Zheng WL, Lu BL (2021) A regression method with subnetwork neurons for vigilance estimation using eog and eeg. IEEE Trans Cogn Develop Syst 13:209–222

    Article  Google Scholar 

  • Wu W, Sun W, Jonathan Wu QM, Yang Y, Zhang H, Zheng WL, Lu BL (2022) Multimodal vigilance estimation using deep learning. IEEE Trans Cybern 52:3097–3110

    Article  PubMed  Google Scholar 

  • Yue W, Ji Q (2019) Facial landmark detection: a literature survey. Int J Comput Vision 127:115–142

    Article  Google Scholar 

  • Zeng H, Yang C, Dai G, Qin F, Zhang J, Kong W (2018) Eeg classification of driver mental states by deep learning. Cogn Neurodyn 12:597–606

    Article  PubMed  PubMed Central  Google Scholar 

  • Zhang Y-F, Gao X-Y, Zhu J-Y, Zheng W-L, Lu B-L (2015) A novel approach to driving fatigue detection using forehead eog. In 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER), pp 707–710

  • Zhang G, Etemad A (2021) Capsule attention for multimodal eeg-eog representation learning with application to driver vigilance estimation. IEEE Trans Neural Syst Rehabil Eng 29:1138–1149

    Article  CAS  PubMed  Google Scholar 

  • Zhang C, Sun L, Cong F, Kujala T, Ristaniemi T, Parviainen T (2020) Optimal imaging of multi-channel eeg features based on a novel clustering technique for driver fatigue detection. Biomed Signal Process Control 62:102103

    Article  Google Scholar 

  • Zhang Y, Guo R, Peng Y, Kong W, Nie F, Bao-Liang L (2022) An auto-weighting incremental random vector functional link network for eeg-based driving fatigue detection. IEEE Trans Instrum Meas 71:1–14

    Google Scholar 

  • Zhang Y, Guo H, Zhou Y, Chengji X, Liao Y (2023) Recognising drivers’ mental fatigue based on eeg multi-dimensional feature selection and fusion. Biomed Signal Process Control 79:104237

    Article  Google Scholar 

  • Zhao L, Li M, He Z, Ye S, Qin H, Zhu X, Dai Z (2022) Data-driven learning fatigue detection system: a multimodal fusion approach of ecg (electrocardiogram) and video signals. Measurement 201:111648

    Article  Google Scholar 

  • Zheng WL, Lu BL (2017) A multimodal approach to estimating vigilance using eeg and forehead eog. J Neural Eng 14

Download references

Funding

This work was funded by the National Key R &D Program of China (grant number 2017YFD0600904).

Author information

Authors and Affiliations

  1. College of Information Science and Technology, Nanjing Forestry University, Nanjing, 210037, China

    Huizhou Yang, Jingwen Huang & Yunfei Liu

  2. CR/RIX1-AP, Bosch (China) Investment Ltd., Shanghai, 200335, China

    Yifei Yu, Zhigang Sun, Shouyi Zhang & Lijuan Xia

  3. College of Letters and Science, University of Wisconsin-Madison, Madison, WI, 53706, USA

    Han Liu

Authors
  1. Huizhou Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jingwen Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yifei Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhigang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shouyi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yunfei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Han Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Lijuan Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yunfei Liu.

Ethics declarations

Conflict of interest

The authors declare no Conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, H., Huang, J., Yu, Y. et al. An improved CapsNet based on data augmentation for driver vigilance estimation with forehead single-channel EEG. Cogn Neurodyn (2024). https://doi.org/10.1007/s11571-024-10105-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11571-024-10105-0

Keywords

Search

Navigation