Research and Experiment with Some Activation Functions in Sleep Stage Classification Using Deep Learning Methods
Nghiên cứu và thử nghiệm một số hàm kích hoạt trong phân lớp giai đoạn giấc ngủ sử dụng các phương pháp học sâu
Abstract
Sleep disorders are becoming increasingly complex in today’s society. Analyzing sleep structure provides valuable information about sleep types, sleep cycles, and variations in different stages. However, manual sleep stage classification is challenging and time-consuming, limiting its clinical use. Deep learning methods have demonstrated promise for automated sleep stage classification, but they face challenges such as class imbalance and the need for interoperability. This paper investigates three deep learning methods, TinySleepNet, TS-TCC, and CA-TCC, using the Expanded Sleep-EDF dataset. The results indicate that TinySleepNet performs well overall, achieving an accuracy of around 80%, while TS-TCC and CA-TCC also perform relatively well with accuracies of around 70%. All three methods classify the Wake stage effectively, with accuracies reaching up to 90%, but struggle with the N1 stage, where accuracy falls below 60%. We propose methods to address these challenges, including the use of GELU, PReLU, LeakyReLU, and SinLU activation functions, which can handle negative values as alternatives to the original ReLU function, and the result can increase from 0.2-1%. Additionally, we have developed a sleep stage classification tool based on these methods.
References
A. J. Scott, T. L. Webb, M. Martyn-St James, G. Rowse, and S. Weich, “Improving sleep quality leads to better mental health: A meta-analysis of randomised controlled trials,” Sleep medicine reviews, vol. 60, p. 101556, 2021.
F. Mendonc¸a, S. S. Mostafa, F. Morgado-Dias, A. G. RaveloGarcia, and T. Penzel, “A review of approaches for sleep quality analysis,” Ieee Access, vol. 7, pp. 24 527–24 546, 2019.
I. Feinberg and T. Floyd, “Systematic trends across the night in human sleep cycles,” Psychophysiology, vol. 16, no. 3, pp. 283–291, 1979.
A. K. Patel, V. Reddy, and J. F. Araujo, “Physiology, sleep stages,” in StatPearls [Internet]. StatPearls Publishing, 2022.
W. Al-salman, Y. Li, and P. Wen, “Detection of eeg kcomplexes using fractal dimension of time frequency images technique coupled with undirected graph features,” Frontiers in Neuroinformatics, vol. 13, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:192519080
J.-S. Wang, G.-R. Shih, and W.-C. Chiang, “Sleep stage classification of sleep apnea patients using decision-tree-based support vector machines based on ecg parameters,” in Proceedings of 2012 IEEE-EMBS International Conference on Biomedical and Health Informatics. IEEE, 2012, pp. 285–288.
E. Rajbhandari, A. Alsadoon, P. Prasad, I. Seher, T. Q. V. Nguyen, and D. T. H. Pham, “A novel solution of enhanced loss function using deep learning in sleep stage classification: predict and diagnose patients with sleep disorders,” Multimedia Tools and Applications, vol. 80, pp. 11 607–11 630, 2021.
Y. Li, Z. Xu, Y. Zhang, Z. Cao, and H. Chen, “Automatic sleep stage classification based on a two-channel electrooculogram and one-channel electromyogram,” Physiological Measurement, vol. 43, no. 7, p. 07NT02, 2022.
J. Fan, C. Sun, M. Long, C. Chen, and W. Chen, “Eognet: A novel deep learning model for sleep stage classification based on single-channel eog signal,” Frontiers in Neuroscience, vol. 15, p. 573194, 2021.
A. Supratak, H. Dong, C. Wu, and Y. Guo, “Deepsleepnet:
A model for automatic sleep stage scoring based on raw single-channel eeg,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 11, pp. 1998–2008, 2017.
S. Mousavi, F. Afghah, and U. R. Acharya, “Sleepeegnet: Automated sleep stage scoring with sequence to sequence deep learning approach,” PloS one, vol. 14, no. 5, p. e0216456, 2019.
E. Eldele, Z. Chen, C. Liu, M. Wu, C.-K. Kwoh, X. Li, and C. Guan, “An attention-based deep learning approach for sleep stage classification with single-channel eeg,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 29, pp. 809–818, 2021.
Y.-L. Hsu, Y.-T. Yang, J.-S. Wang, and C.-Y. Hsu, “Automatic sleep stage recurrent neural classifier using energy features of eeg signals,” Neurocomputing, vol. 104, pp. 105–114, 2013.
A. Supratak and Y. Guo, “Tinysleepnet: An efficient deep learning model for sleep stage scoring based on raw singlechannel eeg,” in 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, 2020, pp. 641–644.
E. Eldele, M. Ragab, Z. Chen, M. Wu, C. K. Kwoh, X. Li, and C. Guan, “Time-series representation learning via temporal and contextual contrasting,” arXiv preprint arXiv:2106.14112, 2021.
E. Eldele, M. Ragab, Z. Chen, M. Wu, C.-K. Kwoh, X. Li, and C. Guan, “Self-supervised contrastive representation learning for semi-supervised time-series classification,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 12, pp. 15 604–15 618, 2023.
B. Kemp, A. Zwinderman, B. Tuk, H. Kamphuisen, and J. Oberyé, “Sleep-edf database expanded,” physionet. org, 2018.
O. Tsinalis, P. M. Matthews, and Y. Guo, “Automatic sleep stage scoring using time-frequency analysis and stacked sparse autoencoders,” Annals of biomedical engineering,
vol. 44, pp. 1587–1597, 2016.
R. B. Berry, R. Brooks, C. Gamaldo, S. M. Harding, R. M. Lloyd, S. F. Quan, M. T. Troester, and B. V. Vaughn, “Aasm scoring manual updates for 2017 (version 2.4),” pp. 665–666, 2017.
J. Xu, Z. Li, B. Du, M. Zhang, and J. Liu, “Reluplex made more practical: Leaky relu,” in 2020 IEEE Symposium on Computers and communications (ISCC). IEEE, 2020, pp. 1–7.
K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1026–1034.
M. Lee, “Gelu activation function in deep learning: A comprehensive mathematical analysis and performance,” arXiv preprint arXiv:2305.12073, 2023.
A. Paul, R. Bandyopadhyay, J. H. Yoon, Z. W. Geem, and R. Sarkar, “Sinlu: Sinu-sigmoidal linear unit,” Mathematics, vol. 10, no. 3, p. 337, 2022.
