北京生物醫(yī)學工程

基于腦電信號深度學習的情緒識別研究現(xiàn)狀

A review of EEG emotion recognition based on deep learning

作者：李穎潔李玉玲楊幫華

單位：上海大學通信與信息工程學院生物醫(yī)學工程研究所（上海 200444）

關(guān)鍵詞：腦電信號; 深度學習; 情緒; 情緒識別; 特征提取

分類號：R318.04; B841.1; B842.6

出版年·卷·期（頁碼）：2020·39·6（634-642）

摘要：

基于腦電信號的情緒識別方法與傳統(tǒng)的人臉識別、語音識別等方法相比，表現(xiàn)出更高的可靠性。然而，由于腦電信號具有低信噪比、非平穩(wěn)性以及被試間差異大等特點，傳統(tǒng)機器學習方法很難進一步提高情緒分類準確性。近年來，隨著深度學習在圖像分類、語音識別等領(lǐng)域的成功應用，許多研究者將其應用于腦電情緒識別。本文在Web of Science和Google Scholar中利用deep learning、EEG及emotion recognition等關(guān)鍵詞檢索到154篇相關(guān)文獻，并基于PRISMA準則篩選出了31篇近幾年內(nèi)將深度學習應用于腦電情緒分類的文獻。文中從腦電信號的預處理、特征提取和深度網(wǎng)絡輸入形式、深度網(wǎng)絡架構(gòu)選擇及參數(shù)設置等方面，介紹了基于腦電深度學習的情緒識別研究進展。同時，以某情緒腦電公共數(shù)據(jù)庫（a Dataset for Emotion Analysis using EEG, Physiological, and video signals, DEAP）相關(guān)研究為例進行各種深度網(wǎng)絡架構(gòu)的比較。本文進一步將文獻分析結(jié)果提煉，為有興趣將深度學習技術(shù)應用于情緒腦電數(shù)據(jù)的研究人員，提供一些處理過程中方法選擇與參數(shù)設置的建議。

Emotion recognition based on EEG signals exhibits higher reliability than traditional approaches such as behavioral, facial and voice. However, due to the characteristics of EEG signals with low signal-to-noise ratio, non-stationary, and large individual differences, it is difficult to improve the classification accuracy by using traditional machine learning methods. In recent years, deep learning has successfully achieved impressive performance in tasks such as image classification and speech recognition, it also proposed to achieved competitive accuracy in EEG-based recognition task. In this paper, we formalized keywords such as deep learning, EEG and emotion recognition search on Web of Science and Google Scholar. The PRISMA criteria was used to identify studies and narrow down the collection, which led this review from an original count of 154 studies to a final count of 31 studies. This paper introduced the research progress on EEG-based emotion recognition using deep learning, from the aspects of EEG signal preprocessing, feature extraction, the choice of input form, architecture and parameter setting of deep networks. Meanwhile, the studies using a public database DEAP (a Dataset for Emotion Analysis using EEG, Physiological, and video signals) are summarized to compare the performance of different deep network architecture. This paper further refines the results of literature analysis which can provide some suggestions for researchers interested in applying deep learning in EEG-based emotion recognition in the selection of EEG signal preprocessing methods and parameters setting.

參考文獻：

[1] Picard RW. Affective computing: challenges[J]. International Journal of Human-Computer Studies, 2003, 59(1-2): 55-64.
[2] Bal E, Harden E, Lamb D, et al. Emotion recognition in children with autism spectrum disorders: Relations to eye gaze and autonomic state[J]. Journal of Autism and Developmental Disorders, 2010, 40: 358-370.
[3] Harms MB, Martin A, Wallace GL. Facial emotion recognition in autism spectrum disorders: a review of behavioral and neuroimaging studies[J]. Neuropsychology Review, 2010, 20(3): 290-322.
[4] Kessous L, Castellano G, Caridakis G. Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis[J]. Journal on Multimodal User Interfaces, 2010, 3: 33-48.
[5] Busso C, Deng Z, Yildirim S, et al. Analysis of emotion recognition using facial expressions, speech and multimodal information[C]// the 6th International Conference on Multimodal Interfaces. PA, USA: Association for Computing Machinery, 2004: 205-211.
[6] Kim KH, Bang SW, Kim SR. Emotion recognition system using short-term monitoring of physiological signals[J]. Medical and Biological Engineering and Computing, 2004, 42: 419-427.
[7] Black MH, Chen NTM, Iyer KK, et al. Mechanisms of facial emotion recognition in autism spectrum disorders: Insights from eye tracking and electroencephalography[J]. Neuroscience & Biobehavioral Reviews, 2017, 80: 488-515.
[8] Alarc?o SM, Fonseca MJ. Emotions recognition using EEG signals: a survey[J]. IEEE Transactions on Affective Computing, 2019, 10(3): 374-393.
[9] Zheng WL, Zhu JY, Lu BL. Identifying stable patterns over time for emotion recognition from EEG[J]. IEEE Transactions on Affective Computing, 2019, 10(3): 417-429.
[10] Cacioppo JT, Berntson GG, Larsen JT, et al. The psychophysiology of emotion[M]//The handbook of emotions. 2nd ed. New York, USA: Guilford Press, 2000: 173-191.
[11] 聶聃, 王曉韡, 段若男, 等. 基于腦電的情緒識別研究綜述[J]. 中國生物醫(yī)學工程學報, 2012, 31(4): 595-606.  Nie D, Wang XW, Duan RN, et al. A survey on EEG based emotion recognition[J]. Chinese Journal of Biomedical Engineering, 2012, 31 (4): 595-606.
[12] Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement[J]. Systematic Reviews, 2015, 4: 1.
[13] Koelstra S, Mühl C, Soleymani M, et al. DEAP: a database for emotion analysis; using physiological signals[J]. IEEE Transactions on Affective Computing, 2011, 3: 18-31.
[14] Wang X, Zhang T, Xu X, et al. EEG emotion recognition using dynamical graph convolutional neural networks and broad learning system[C]//2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). Madrid, Spain: IEEE Press, 2018,1: 1240-1244.
[15] Qiao R, Qing C, Zhang T, et al. A novel deep-learning based framework for multi-subject emotion recognition[C]//2017 4th International Conference on Information, Cybernetics and Computational Social Systems (ICCSS). Dalian, China: IEEE Press, 2017: 181-185.
[16] Li Y, Huang J, Zhou H, et al. Human emotion recognition with electroencephalographic multidimensional features by hybrid deep neural networks[J]. Applied Sciences, 2017, 7(10): 1060.
[17] Xing X, Li Z, Xu T, et al. SAE+ LSTM: A New Framework for Emotion Recognition from Multi-Channel EEG[J]. Frontiers in Neurorobotics, 2019, 13: 37.
[18] Chen T, Ju S, Yuan X, et al. Emotion recognition using empirical mode decomposition and approximation entropy[J]. Computers & Electrical Engineering, 2018, 72: 383-392.
[19] Salama ES, El-Khoribi RA, Shoman ME, et al. EEG-based emotion recognition using 3D convolutional neural networks[J]. International Journal of Advanced Computer Science and Applications, 2018, 9(8): 329-337.
[20] Jung TP, Makeig S, Humphries C, et al. Removing electroencephalographic artifacts by blind source separation[J]. Psychophysiology, 2010, 37: 163-178.
[21] Nathan K, Contreras-Vidal JL. Negligible motion artifacts in scalp electroencephalography (EEG) during treadmill walking[J]. Frontiers in Human Neuroscience, 2015, 9: 708.
[22] Delorme A, Sejnowski T, Makeig S. Enhanced detection of artifacts in EEG data using higher-order statistics and independent component analysis[J]. Neuroimage, 2007, 34(4): 1443-1449.
[23] Akhtar MT, Mitsuhashi W, James CJ. Employing spatially constrained ICA and wavelet denoising, for automatic removal of artifacts from multichannel EEG data[J]. Signal Processing, 2012, 92(2): 401-416.
[24] Islam MK, Rastegarnia A, Yang Z. Methods for artifact detection and removal from scalp EEG: A review[J]. Neurophysiologie Clinique, 2016, 46(4-5): 287-305.
[25] Daly I, Nicolaou N, Nasuto SJ, et al. Automated artifact removal from the electroencephalogram: a comparative study[J]. Clinical EEG and Neuroscience, 2013, 44(4): 291-306.
[26] Mannan MMN, Kamran MA, Jeong MY. Identification and removal of physiological artifacts from electroencephalogram signals: a review[J]. IEEE Access, 2018, 6: 30630-30652.
[27] Zheng WL, Lu BL. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks[J]. IEEE Transactions on Autonomous Mental Development, 2020, 7(3): 162-175.
[28] Yanagimoto M, Sugimoto C. Recognition of persisting emotional valence from EEG using convolutional neural networks[C]//2016 IEEE 9th International Workshop on Computational Intelligence and Applications (IWCIA). Hiroshima, Japan: IEEE Press, 2016: 27-32.
[29] Alhagry S, Fahmy AA, El-Khoribi RA. Emotion recognition based on EEG using LSTM recurrent neural network[J]. International Journal of Advanced Computer Science and Applications, 2017, 8(10): 355-358.
[30] Yang Y, Wu Q, Qiu M, et al. Emotion recognition from multi-channel EEG through parallel convolutional recurrent neural network[C]//2018 International Joint Conference on Neural Networks (IJCNN). Rio de Janeiro, Brazil: IEEE Press, 2018: 1-7.
[31] Das BB, Kumar P, Kar D, et al. A spatio-temporal model for EEG-based person identification[J]. Multimedia Tools and Applications, 2019, 78(19): 28157-28177.
[32] Kandel ER, Schwartz JH, Jessell TM, et al. Principles of neural science[M]. 5th ed. New York, USA: McGraw-hill, 2000, 4: 1227-1246.
[33] Zheng WL, Zhu JY, Peng Y, et al. EEG-based emotion classification using deep belief networks[C]//2014 IEEE International Conference on Multimedia and Expo (ICME). Chengdu, China: IEEE Press, 2014: 1-6.
[34] Li H, Qing C, Xu X, et al. A novel DE-PCCM feature for EEG-based emotion recognition[C]//2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC). Shenzhen, China: IEEE Press, 2017: 389-393.
[35] Li J, Zhang Z, He H. Hierarchical convolutional neural networks for EEG-based emotion recognition[J]. Cognitive Computation, 2018, 10(2): 368-380.
[36] Huang J, Xu X, Zhang T. Emotion classification using deep neural networks and emotional patches[C]//2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). Kansas City, MO, USA: IEEE Press, 2017: 958-962.
[37] Wang F, Zhong S, Peng J, et al. Data augmentation for EEG-based emotion recognition with deep convolutional neural networks[C]//International Conference on Multimedia Modeling. Bangkok, Thailand: Springer, Cham, 2018: 82-93.
[38] Zhang T, Zheng W, Cui Z, et al. Spatial–temporal recurrent neural network for emotion recognition[J]. IEEE Transactions on Cybernetics, 2019, 49(3): 839-847.
[39] Luo Y. EEG data augmentation for emotion recognition using a conditional wasserstein GAN[C]//2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Honolulu, HI, USA: IEEE Press, 2018: 2535-2538.
[40] Xu H, Plataniotis KN. Affective states classification using EEG and semi-supervised deep learning approaches[C]//2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP). Montreal, QC, Canada: IEEE Press: 2016: 1-6.
[41] Al-Nafjan A, Hosny M, Al-Wabil A, et al. Classification of human emotions from electroencephalogram (EEG) signal using deep neural network[J]. International Journal of Advanced Computer Science and Applications (IJACSA), 2017, 8(9): 419-425.
[42] Zhang W, Liu Q. Using the center loss function to improve deep learning performance for EEG signal classification[C]//2018 Tenth International Conference on Advanced Computational Intelligence (ICACI). Xiamen, China: IEEE Press, 2018: 578-582.
[43] Rayatdoost S, Soleymani M. Cross-corpus EEG-based emotion recognition[C]//2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP). Aalborg, Denmark: IEEE Press, 2018: 1-6.
[44] Moon SE, Jang S, Lee JS. Convolutional neural network approach for EEG-based emotion recognition using brain connectivity and its spatial information[C]//2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Seoul, South Korea: IEEE Press, 2018: 2556-2560.
[45] Chen JX, Zhang PW, Mao ZJ, et al. Accurate EEG-based emotion recognition on combined features using deep convolutional neural networks[J]. IEEE Access, 2019, 7: 44317-44328.
[46] Chao H, Dong L, Liu Y, et al. Emotion recognition from multiband EEG signals using CapsNet[J]. Sensors, 2019, 19(9): 2212-2246.
[47] Chao H, Zhi H, Dong L, et al. Recognition of emotions using multichannel EEG data and DBN-GC-based ensemble deep learning framework[J]. Computational Intelligence and Neuroscience, 2018, 2018(2): 1-11.
[48] Lu Y, Wang M, Wu W, et al. Identification of emotional valences via memory-informed deep neural network with entropy features[C]// 2019 4th International Conference on Multimedia Systems and Signal Processing. Guangzhou, China: ACM, 2019: 31-35.
[49] Liu N, Fang Y, Li L, et al. Multiple feature fusion for automatic emotion recognition using EEG signals[C]//2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Seoul, Korea: IEEE Press, 2018: 896-900.
[50] Li X, Song D, Zhang P, et al. Emotion recognition from multi-channel EEG data through convolutional recurrent neural network[C]//2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). Shenzhen, China: IEEE Press, 2016: 352-359.
[51] Li X, Song D, Zhang P, et al. Deep fusion of multi-channel neurophysiological signal for emotion recognition and monitoring[J]. International Journal of Data Mining and Bioinformatics, 2017, 18(1): 1-27.

服務與反饋：

【文章下載】【加入收藏】

提示：您還未登錄，請登錄！點此登錄

51黑料吃瓜在线观看,51黑料官网|51黑料捷克街头搭讪_51黑料入口最新视频