Применение методов снижения размерности данных к построению нечетких нейронных сетей

V. A. Tenenev; A. S. Shaura

doi:10.22213/2410-9304-2020-4-109-116

Authors

V. A. Tenenev Kalashnikov ISTU, Udmurt Federal Research Center of the Ural Branch of the Russian Academy of Sciences, Izhevsk
A. S. Shaura Kalashnikov ISTU, Udmurt Federal Research Center of the Ural Branch of the Russian Academy of Sciences, Izhevsk

DOI:

https://doi.org/10.22213/2410-9304-2020-4-109-116

Keywords:

data reduction, principal component analysis, fuzzy neural network, decision tree, autoencoder, restricted Boltzmann machine

Abstract

Many data mining problems can be reduced to classification and regression problems; modern approaches to their solving are based on the use of neural networks, decision trees, fuzzy logic, and classical statistical methods. The solution to complex practical problems consists of several stages, such as feature detection and feature selection, reducing the initial space's dimension, searching for relationships, and building a mathematical model.

The paper considers data dimension reduction when constructing a fuzzy neural network to reduce the number of initial features and increase their informativeness. We consider several approaches to solve it: application of the principal component analysis (PCA), a neural network with an autoencoder architecture, and a restricted Boltzmann machine. In contrast to the method of principal components, the use of an autoencoder and a limited Boltzmann machine makes it possible to take into account the nonlinear connections between the existing features. An important feature of the proposed fuzzy neural network based on fuzzy decision trees is the visibility of the presentation of relationships in the data system in the form of logical rules to assess the importance of each rule. Using the presented methods for reducing the input data dimension allowed us to reduce the approximation error significantly. According to the test results, the best ability to generalize data among the considered methods is possessed by the restricted Boltzmann machine: its application for constructing rules in a fuzzy neural network reduces the approximation error several times compared with the principal component analysis. The results obtained in this work can be used to construct universal fuzzy approximators for solving machine learning and data analysis problems.

References

Antonio Rico-Sulayes. Reducing Vector Space Dimensionality in Automatic Classiﬁcation for Authorship Attribution // Revista Ingeniería Electrónica, Automática y Comunicaciones. — 2017. — Т. 38, № 3.

Sakshi Indolia, Anil Kumar Goswami, S.P . Mishra, Pooja Asopa, Conceptual Understanding of Convolutional Neural Network - A Deep Learning Approach, Procedia Computer Science, Volume 132, 2018, Pages 679-688, ISSN 1877-0509, https://doi.org/10.1016/j.procs.2018.05.069.

Rawat, Waseem & Wang, Zenghui. (2017). Deep Convolutional Neural Networks for Image Classiﬁcation: A Comprehensive Review. Neural Computation. 29. 1-98. 10.1162/ NECO_a_00990.

Mostaar A, Sattari MR, Hosseini S, Deevband MR. Use of Artiﬁcial Neural Networks and PCA to Predict Results of Infertility Treatment in the ICSI Method. J Biomed Phys Eng. 2019;9(6): 679-686. Published 2019 Dec 1. doi:10.31661/jbpe.v0i0.1187

M. Seuret, M. Alberti, M. Liwicki and R. Ingold, "PCA-Initialized Deep Neural Networks Applied to Document Image Analysis," 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Kyoto, 2017, pp. 877-882. doi: 10.1109/ICDAR.2017.148

Kruger U., Zhang J., Xie L. (2008) Developments and Applications of Nonlinear Principal Component Analysis – a Review. In: Gorban A.N., Kégl B., Wunsch D.C., Zinovyev A.Y. (eds) Principal Manifolds for Data Visualization and Dimension Reduction. Lecture Notes in Computational Science and Enginee, vol 58. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73750-6_1

Classification and regression trees // L. Breiman,

J. H. Friedman, R. A. Olshen, C. J. Stone. – California: Wadsworth & Brooks, 1984. – 368 p.

Antonio Hernández-Blanco, Boris Herrera-Flores, David Tomás, Borja Navarro-Colorado, "A Systematic Review of Deep Learning Approaches to Educational Data Mining", Complexity, vol. 2019, Article ID 1306039, 22 pages, 2019.https://doi.org/10.1155/ 2019/1306039

Emmert-Streib F , Yang Z, Feng H, Tripathi S and Dehmer M (2020) An Introductory Review of Deep Learning for Prediction Models With Big Data. Front. Artif. Intell. 3:4. doi: 10.3389/frai. 2020.00004

Созыкин А. В. Обзор методов обучения глубоких нейронных сетей // Вестник ЮУрГУ. Серия: Вы-числительная математика и информатика. 2017. Т. 6, № 3. С. 28–59. DOI: 10.14529/cmse170303.

Salakhutdinov, R. & Hinton, G. (2009). Deep Boltzmann Machines. Proceedings of the Twelth International Conference on Artiﬁcial Intelligence and Statis-tics, in PMLR5:448-455

Hinton G.E., Osindero S., Teh Y.-W. A Fast Learning Algorithm for Deep Belief Nets. Neural Com-puting. 2006. vol. 18, no. 7. pp. 1527–1554. DOI: 10.1162/neco.2006.18.7.1527.

Осовский, Станислав. Нейронные сети для обработки информации / Станислав Осовский ; пер. с пол. И. Д. Рудинского. М. : Финансы и статистика, 2004 (Великолук. гор. тип.). 43 с. : ил.; 24 см. ISBN 5-279-02567-4 (в обл.)

Тененев В. А., Ворончак В. И. Решение задач классификации и аппроксимации с применением нечетких деревьев решений // Интеллектуальные системы в производстве. 2005. № 2. С. 46–54.

Тененев В. А., Тененева А. В. Обучение нечетких нейронных сетей генетическим алгоритмом // Интеллектуальные системы в производстве. 2010. № 1. С. 76–85.