Гибридная бионическая система управления протезами: обзор

A M Samandari; A N Afonin

doi:10.22213/2410-9304-2024-3-23-30

Authors

A. M. Samandari Belgorod State National Research University
A. N. Afonin Belgorod State National Research University

DOI:

https://doi.org/10.22213/2410-9304-2024-3-23-30

Keywords:

prosthetics, functional near- infrared spectroscopy (fNIRS), electromyography (EMG), electroencephalography (EEG), neural interfaces, hybrid brain-computer interface systems HBCI, hybrid bionic control system (HBCS)

Abstract

The legs and hands are the most susceptible to loss and this is due to the fact that they are prominent external organs in the human body. With increasing disasters, accidents, wars and diseases the loss of limbs is increasing, making the individual restricted in freedom and movement and pursuing to find alternatives to improve the individual's life is extremely important. Modern bionic prostheses are the best alternatives to replace amputated ones to perform both aesthetic and functional tasks. On this basis and by analyzing the most common and used methods in prosthetics management as the electroencephalography (EEG), electromyography (EMG), and functional near- infrared spectroscopy (fNIRS) methods, knowing their advantages and disadvantages, comparing them and documenting their results from the outputs of literature and previous experimental studies, whether in individual use or hybrid use.In the light of these data, this article highlights the most common technologies and considers their superiority and insuperiority, which can be suitable for the formation of a hybrid bionic control system for prostheses or rehabilitation and restoration of lost functions. Based on the most important studies that have dealt with these technologies whereas individually or in their hybrid state. In addition, this article provides an encouraging outlook for those interested in scientific research to research, compare, identify and characterise superior hybrid systems related to exoskeletal control systems and in particular prostheses.

Author Biographies

A. M. Samandari, Belgorod State National Research University

Postgraduate Student

A. N. Afonin, Belgorod State National Research University

Doctor of Technical Sciences, Professor, Professor of the Department of Information and Robotic Systems

References

Nsugbe E., Phillips C., Fraser M.F., McIntosh J. Gesture recognition for transhumeral prosthesis control using EMG and NIR. IET Cyber-Syst. Robot., 2020 vol. 2, no. 3, pp. 122131.

He L. et al. Subject-Independent Estimation of Continuous Movements Using CNN-LSTM for a Home-Based Upper Limb Rehabilitation System. IEEE ROBOTICS AND AUTOMATION LETTERS, OCTOBER 2023.VOL. 8, NO. 10.

NEELUM Y.S. et al. Enhancing Classification Accuracy of Transhumeral Prosthesis: A Hybrid sEMG and fNIRS Approach Digital Object Identifier 10.1109/ACCESS. 2021.VOLUME 9, 3099973.

Guo W.C., Zhang X., Liu H., Zhu X. Toward an enhanced human machine interface for upper-limb prosthesis control with combined EMG and NIRS signals,'' IEEE Trans. Human-Mach. Syst., 2017, vol. 47, no. 4, pp. 564575.

Bin Abdul Ghaffar M.S., Khan U.S., Naseer N., Rashid N., and Tiwana M.I. Improved Classification Accuracy of Four Class FNIRS-BCI," 2020 12th International Conference on Electronics, Computers and Artificial Intelligence (ECAI), Bucharest, Romania, 2020, pp. 1-5, doi: 10.1109/ ECAI50035.2020.9223258.

Samandari A.M.N. fNIRS as a hybrid system with EEG and with sEMG for controlling prostheses. In: Nauchnye issledovaniya molodykh uchenykh: sbornik statei XXV Mezhdunarodnoi nauchno-prakticheskoi konferentsii, 10 November 2023, Penza, Russia. Penza: Nauka i Prosveshchenie (IP Gulyaev G.Yu.); 2023. P. 31-34. (In Russ.).

Asadullaev R.R., Afonin A.N., Shchetinina E.S. Recognition of patterns of motor activity by a neural network based on continuous optical tomography fNIRS data. Экономика. Информатика, 2021, vol.48, no.4,pp. 735-746.

Kwak Y., Song W.J., Kim S.E. FGANet: FNIRS-Guided Attention Network for Hybrid EEG-fNIRS Brain-Computer Interfaces. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2022, vol. 30, pp. 329-339.

Aydin E.A. Subject-specific feature selection for near infrared spectroscopy based brain-computer interfaces.Comput. Methods Programs Biomed., 2020, vol. 195, Oct. Art. no. 105535.

Arif A. et al. Hemodynamic response detection using integrated EEG- fNIRS-VPA for BCI, Computers, Materials and Continua, 2021, vol.70, no. 1, pp. 535-555.

Milanes D. H., et al. Shallow Convolutional Network Excel for Classifying Motor Imagery EEG in BCI Applications," 2021.IEEE Access, vol. 9, pp. 98275-98286.

NA L., RUI Z., BHARATH K., ASHIRBAD P., HYOWON L., JIAYUAN H., NING J. Non-invasive Techniques for Muscle Fatigue Monitoring: A Comprehensive Survey. ACM 0360-0300/2024/02-ART http://dx.doi.org/10.1145/3648679.

Daniel N., Sybilski K., Kaczmarek W., Siemiaszko D., Małachowski J. Relationship between EMG and fNIRS during Dynamic Movements. Sensors. 2023;23(11). DOI: 10.3390/s23115004.

Giminiani R.D., Marco C., Marco F., Valentina Q. Validation of fabric-based thigh-wearable EMG sensors and oximetry for monitoring quadricep activity during strength and endurance exercises. Sensors. 2020;17:1-13. DOI: 10.3390/s20174664.

Kimoto H.F., Machida M. A wireless multi-layered EMG/MMG/NIRS sensor for muscular activity evaluation. Sensors. 2023;23(3). DOI: 10.3390/s23031539.

Cheng X., Sie E.J., Boas D.A., Marsili F. Choosing an optimal wavelength to detect brain activity in functional near-infrared spectroscopy. Opt Lett. 2021;46(4):924. DOI: 10.1364/ol.418284.

Samandari Аli М. Functional near-infrared spectroscopy (fNIRS) as a hybrid system: a review. Modelirovanie, optimizatsiya i informatsionnye tekhnologii = Modeling, Optimization and Information Technology. 2024;12(1). https://doi.org/10.26102/2310-6018/2024.44.1.005 [Accessed 20th January 2024].

Taborri J., Keogh J., Kos A. et al. Sport biomechanics applications using inertial, force, and EMG sensors: a literature overview. Appl Bionics Biomech. 2020;2020:2041549. DOI: 10.1155/2020/2041549.

Liu Z., Shore J., Wang M., Yuan F., Buss A., Zhao X. A systematic review on hybrid EEG/fNIRS in brain-computer interface. Biomed Signal Process Control. 2021;68 102595.

Asmaa M., Saeed M.Q., Salankar N., Feng J., Ryszard T., Paweł P., Ahmed A., Mohamed H. Hybrid EEG-fNIRS brain-computer interface based on the non-linear features extraction and stacking ensemble learning. Biocybern Biomed Eng. 2023;43(2):463-475. DOI: 10.1016/j.bbe.2023.05.001.

Lin J.F.L. Dual-MEG interbrain synchronization during turn-taking verbal interactions between mothers and children. Cerebral Cortex. 2023;33(7):4116-4134. DOI: 10.1093/cercor/ bhac330.

Samandari А.М. Functional near-infrared spectroscopy (fNIRS) as a hybrid system: a review. Modeling, Optimization and Information Technology. 2024;12(1). URL: https://moitvivt.ru/ru/journal/pdf?id=1459 DOI: 10.26102/2310-6018/2024.44.1.005 (In Russ.).

Brian F. S., Charles P., Christopher H., Matthew T., Rishishankar E. S., Jordon G., Mark G., Steven A.K., Nathan C.R. The evolution of neuromodulation for chronic stroke: From neuroplasticity mechanisms to brain-computer interfacesAuthor links open overlay panel. Neurotherapeutics Volume 21, Issue 3, April 2024, e00337. https://doi.org/10.1016/j.neurot.2024.e00337.

McManus L., Giuseppe D.V., Madeleine M.L. Analysis and Biophysics of Surface EMG for Physiotherapists and Kinesiologists: Toward a Common Language With Rehabilitation Engineers. Front. Neurol. 11:576729 (2020). doi: 10.3389/fneur.2020.576729.

Marinelli A., Canepa M., Domenico D.D., Gruppioni E., Laffranchi M., Michieli L.D., Chiappalone M., Semprini M., Boccardo N. A comparative optimization procedure to evaluate pattern recognition algorithms on hannes prosthesis. Neurocomputing V 569, 7 2024, 127123.https://doi.org/10.1016/j.neucom.2023.127123.

Cha H., An S., Choi S., Yang S., Park S., Park S. Study on Intention Recognition and Sensory Feedback: Control of Robotic Prosthetic Hand Through EMG Classificationand Proprioceptive Feedback Using Rule-based Haptic Device, IEEE Transactionson Haptics 15 (3) (2022) 560-571.

Tchimino J., Markovic M., Dideriksen J.L., Dosen S. The effect of calibrationparameters on the control of a myoelectric hand prosthesis using EMG feedback,",J. Neural Eng. vol. 18 (4) (2021), 046091.

Li X., Chen S., Zhang H., Samuel O., Wang H. Towards reducing the impacts of unwanted movements on identification of motion intentions," Journal of Electromyography and Kinesiology, vol. 28, pp. 90-98, 2016.

Lobo-Prat J., Kooren P.N., Stienen A.H., Herder J.L., Koopman B.F., Veltink P.H. Non-invasive control interfaces for intention detection in active movement-assistive devices," Journal of neuroengineering and rehabilitation, vol. 11, no. 1, pp. 1-22, 2014.

Song T., Yan Z., Guo S., Li Y., LiX., Xi F. Review of sEMG for robot control: Techniques and applications. Applied Sciences (Switzerland). 2023;13(17). DOI: 10.3390/ app13179546.

Chunfu L., Ruite G., Zhichuan T., Xiaoyun F., Lekai Z., Keshuai Y., Xuan X. Multi-channel FES gait rehabilitation assistance system based on adaptive sEMG modulation. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2023;31:3652-3663. DOI: 10.1109/tnsre. 2023.3313617.

Radek M., Martina L., Michaela S., Rene J., Khosrow B., Radana K., Aleksandra K.S. Advanced bioelectrical signal processing methods: Past, present, and future approach-part iii: Other biosignals. Sensors. 2021;21(18). DOI: 10.3390/s21186064.

Shah A.K., Mittal S. Invasive electroencephalography monitoring: Indications and presurgical planning, Ann. Indian Acad. Neurol. 17 (Suppl S1) (2014) 89-94, http://dx.doi.org/10.4103/0972-2327.128668.

Huda M.R. et al. Recognition of Upper Limb Movements Based on Hybrid EEG and EMG Signals for Human-Robot Interaction Iraqi Journal of Computers, Communications, Control & Systems Engineering (IJCCCE), Vol. 23, No. 2, June 2023 DOI: https://doi.org/10.33103/uot.ijccce.23.2.14.

Lubo F., Haoyang L., Hongfei J. EEG-EMG ANALYSIS METHODIN HYBRID BRAIN COMPUTER INTERFACEFOR HAND REHABILITATION TRAINING. 2023, 741-761, doi: 10.31577/cai 2023 3 741.

Janis P., Dmytro M. State-of-the-art on brain-computer interface technology. Sensors. 2023;23(13). Multidisciplinary Digital Publishing Institute (MDPI). DOI: 10.3390/s23136001.

Kun L., Zhaochu Y., Dong T., Libo Z., Nuno P., Carlos A.D., Bjørn T.S., Lars E.R., Wen L., Zhuangde J. Exploring the Intersection of Brain-Computer Interfaces and Quantum Sensing: A Review of Research Progress and Future Trends. Adv. Quantum Technol. 2023, 2300185. DOI: 10.1002/qute.202300185.

Hramov A.E., Maksimenko V.A., Pisarchik A.N. Physical principles of brain-computer interfaces and their applications for rehabilitation, robotics and control of human brain states. Physics Reports. 2021;918:1-133. DOI: 10.1016/j.physrep.2021.03.002.

Becerra-Fajardo L., Minguillon J., Krob M.O. et al. First-in-human demonstration of floating EMG sensors and stimulators wirelessly powered and operated by volume conduction. J NeuroEngineering Rehabil 21, 4 (2024). https://doi.org/10.1186/s12984-023-01295-5.

Dario F., Ning J., Hubertus R., Aleš H., Bernhard G., Hans D., Oskar C.A. The extraction of neural information from the surface EMG for the control of upper-limb prostheses: Emerging avenues and challenges," IEEE Trans. Neural Syst. Rehabil. Eng., vol. 22, no. 4, pp. 797-809, Jul. 2014.

Afonin, A.N., Asadullaev, R.G., Sitnikova, M.A., 2018. Data analysis of the fNIRS tomograph for the management of limb prostheses using the brain-computer interface. Scientific and Technical Bulletin of the Volga region, 11: 182 - 185. (in Russian).

Usama A.S., Zareena K., Neelum Y.S. Control of a Prosthetic Arm using fNIRS, A Neural-Machine Interface. 2020. DOI: 10.5772/intechopen.93565.

Mondini V., Sburlea A.I., Müller-Putz G.R. Towards unlocking motor control in spinal cord injured by applying an online EEG-based framework to decode motor intention, trajectory and error processing. Sci Rep 14, 4714 (2024). https://doi.org/10.1038/s41598-024-55413-x.

Khajuria A., Sharma R., Joshi D. EEG Dynamics of Locomotion and Balancing: Solution to Neuro-Rehabilitation. Clinical EEG and Neuroscience. 2024;55(1):143-163. doi:10.1177/15500594221123690.

Asanza V., Pelaez E., Loayza F., Lorente-Leyva L.L. Peluffo-Ordonez D.H. Identification of lower-limb motor tasks via brain-computer interfaces: a topical overview. Sensors. 2022;22(5). DOI: 10.3390/s22052028

Deligani R.J., Borgheai S.B., McLinden J., Shahriari Y. Multimodal fusion of EEG-fNIRS: a mutual information-based hybrid classification framework. Biomed Opt Express. 2021;12(3):1635. DOI: 10.1364/boe.413666.

Klein F.S., Debener K.W., Kranczioch C. fMRI-based validation of continuous-wave fNIRS of supplementary motor area activation during motor execution and motor imagery. Sci Rep. 2022;12(1). DOI: 10.1038/s41598-022-06519-7.

Sergio L.N., Alex C.C., Forti R.M., Fernado C., Clarissa L.Y., Rickson C.M. Revealing the spatiotemporal requirements for accurate subject identification with resting-state functional connectivity: a simultaneous fNIRS-fMRI study. Neurophotonics. 2023;10(1). DOI: 10.1117

Teng M. et al. The hybrid BCI system for movement control by combining motor imagery and moving onset visual evoked potential, J. Neural Eng. 14 (2) (2017) 26015, http://dx.doi.org/10.1088/1741-2552/aa5d5f.

Vidal J.J. Toward direct brain-computer communication, Annu Rev. Biophys. Bioeng. 2 (1) (1973) 157-180, http://dx.doi.org/10.1146/annurev.bb.02.060173.001105. [23] J.J. Vidal, Real-time detection of brain events in EEG, Pros. IEEE 65 (5) (1977) 633-641, http://dx.doi.org/10.1109/ PROC.1977.10542.

Nerlich A.G., Zink A., Szeimies U., Hagedorn H.G. Ancient Egyptian prosthesis of the big toe. Lancet 2000, Vol. 356, 2176-2179.