SYNTHESIS AND USAGE OF NEURAL NETWORK MODELS WITH PROBABILISTIC STRUCTURE CODING
DOI:
https://doi.org/10.15588/1607-3274-2021-2-10Keywords:
neuroevolution, coding, probabilistic data structures, neural networks, genetic algorithm.Abstract
Context. The problem of encoding information of models based on artificial neural networks for further transmission and use of such models is considered. The object of research is the process of coding artificial neural networks using probabilistic data structures.
Objective of this work is to develop a method for coding neural networks to reduce the resource intensity of the process of neuroevolutionary model synthesis.
Method. A method for encoding neural networks based on probabilistic data structures is proposed. At the beginning, the method uses the basic principles of the approach of direct encoding of network information and, based on sequencing, encodes a matrix of interneuronal connections in the form of biopolymers. Then, probabilistic data structures are used to represent the original matrix more compactly. For this purpose, hash functions are used, the initial matrix goes through the hashing process, which significantly reduces the requirements for memory resources. The method allows to reduce memory costs when sending artificial neural networks, which significantly expands the practical use of such models, preventing a sharp decrease in the accuracy of their operation.
Results. The developed method is implemented and investigated in solving the problem of classification of the state of South German creditors. The use of the developed method allowed increasing the rate of neuromodel synthesis by 15–17.6%, depending on the computing resources used. The method also reduced the share of information transfers by 8%, which also indicates faster and more efficient use of resources.
Conclusions. The conducted experiments confirmed the efficiency of the proposed mathematical software and allow us to recommend it for use in practice, when encoding models based on artificial neural networks, for further solving problems of diagnostics, forecasting, evaluation and pattern recognition. Prospects for further research may consist in pre-processing data for more strict control of the encoding process in order to minimize the loss of quality of models based on neural networks.
References
Kanaan M. T-Minus AI: Humanity’s Countdown to Artificial Intelligence and the New Pursuit of Global Power. Dallas, BenBella Books, 2019, 270 p.
Taulli T. Artificial Intelligence Basics: A Non-Technical Introduction. New York, Apress, 2019, 199 p.
Rothman D. Artificial Intelligence By Example: Acquire advanced AI, machine learning, and deep learning design skills. Birmingham, Packt Publishing, 2020, 578 p.
Chang A.C. Intelligence-Based Medicine: Artificial Intelligence and Human Cognition in Clinical Medicine and Healthcare. Cambridge, Academic Press, 2020, 534 p.
Ponteves de P. AI Crash Course: A fun and hands-on introduction to machine learning, reinforcement learning, deep learning, and artificial intelligence with Python. Birmingham, Packt Publishing, 2019, 360 p.
Artasanchez A., Joshi P. Artificial Intelligence with Python: Your complete guide to building intelligent apps using Python 3.x. Birmingham, Packt Publishing, 2020, 618 p.
Oliinyk A., Subbotin S., Leoshchenko S., Ilyashenko M., Myronova N., Mastinovsky Y. Аdditional training of neurofuzzy diagnostic models, Radio Electronics, Computer Science, Control, 2018, № 3, pp. 113–119. DOI: 10.15588/1607-3274-2018-3-12.
Leoshchenko S., Oliinyk A., Subbotin S., Zaiko T. Using Modern Architectures of Recurrent Neural Networks for Technical Diagnosis of Complex Systems, 2018 International Scientific-Practical Conference Problems of Infocommunications. Science and Technology (PIC S&T), Kharkiv, 9–12 October 2018, proceedings. Kharkiv, IEEE, 2018, pp. 411–416. DOI: 10.1109/INFOCOMMST.2018.8632015
Leoshchenko S., Oliinyk A., Subbotin S., Shylo S., Shkarupylo V. Method of Artificial Neural Network Synthesis for Using in Integrated CAD, 15th International Conference on the Experience of Designing and Application of CAD Systems (CADSM), Polyana, 26 February – 2 March 2019, proceedings. Lviv, IEEE 2019, pp. 1–6. DOI: 10.1109/CADSM.2019.8779248
Iba H. Evolutionary Approach to Machine Learning and Deep Neural Networks: Neuro-Evolution and Gene Regulatory Networks, New York, Springer, 2018, 258 p.
Omelianenko I. Hands-On Neuroevolution with Python: Build high-performing artificial neural network architectures using neuroevolution-based algorithms, Birmingham, Packt Publishing, 2019, 368 p.
Bergel A. Agile Artificial Intelligence in Pharo: Implementing Neural Networks, Genetic Algorithms, and Neuroevolution, New York, Apress, 2020, 407 p.
Blokdyk G. Neuroevolution of augmenting topologies: Second Edition, Ohio, 5STARCooks, 2018, 128 p.
Lockett A.J. General-Purpose Optimization Through Information Maximization (Natural Computing Series). New York, Springer, 2020, 579 p.
Rouhiainen L. Artificial Intelligence: 101 Things You Must Know Today About Our Future, Scotts Valley, CreateSpace Independent Publishing Platform, 2018, 300 p.
Koul A., Ganju S., Kasam M. Practical Deep Learning for Cloud, Mobile, and Edge: Real-World AI & ComputerVision Projects Using Python, Keras & TensorFlow, Newton, O’Reilly Media, 2019, 620 p.
Singh A., Bhadani R. Mobile Deep Learning with TensorFlow Lite, ML Kit and Flutter: Build scalable real-world projects to implement end-to-end neural networks on Android and iOS, Birmingham, Packt Publishing, 2020, 380 p.
Davies J., Fortuna C. The Internet of Things: From Data to Insight, Hoboken, Wiley, 2020, 240 p.
Zheng N., Mazumder P. Learning in Energy-Efficient Neuromorphic Computing: Algorithm and Architecture CoDesign, New York, Wiley-IEEE Press, 2019, 296 p.
Bianchi F.M., Maiorino E., Kampffmeyer M.C., et. al. Recurrent Neural Networks for Short-Term Load Forecasting: An Overview and Comparative Analysis (SpringerBriefs in Computer Science), New York, Springer, 2017, 81 p.
Ozkan L. RNA Sequencing: Principles and Data Analysis, Traverse City, Independently published, 2020, 118 p.
Robinson T.R., Spock L. Genetics For Dummies, New York, For Dummies, 2020, 400 p.
Tan T.W., Lee E. Beginners Guide to Bioinformatics for High Throughput Sequencing, Singapore, World Scientific Publishing Co Pte Ltd, 2018, 300 p.
Reagen B., Gupta U., Adolf R., Mitzenmacher M., et al. Weightless: Lossy Weight Encoding For Deep Neural Network Compression, International Conference on Machine Learning (ICML 2018), Stockholmsmässan, 10–15 July, proceesings. Stockholmsmässan, PMLR, 2018, pp. 1–10.
Gakhov A. Probabilistic Data Structures and Algorithms for Big Data Applications, Madison : Books on Demand, 2019, 220 p.
Knebl H. Algorithms and Data Structures: Foundations and Probabilistic Methods for Design and Analysis, New York, Springer, 2020, 360 p.
Cormen T.H., Leiserson C.E., Rivest R.L., Stein C. Introduction to Algorithms, Cambridge, The MIT Press, 2009, 1292 p.
Leoshchenko S. D., Oliinyk A. O., Subbotin S. A., Lytvyn V. A., Shkarupylo V.V. Modification and parallelization of genetic algorithm for synthesis of artificial neural networks, Radio Electronics, Computer Science, Control, 2019, № 4, pp. 68–82. DOI: 10.15588/1607-3274-2018-312.
Leoshchenko S., Oliinyk A., Subbotin S., Gorobii N., Zaiko T. Synthesis of artificial neural networks using a modified genetic algorithm, 1st International Workshop on Informatics & Data-Driven Medicine (IDDM 2018), Lviv, 28–30 October, 2018 : proceedings. Lviv, CEUR WS, 2018, pp. 1–13.
Leoshchenko S., Oliinyk A., Subbotin S., Gorobii N., Zaiko T.Implementation of Selective Pressure Mechanism to Optimize Memory Consumption in the Synthesis of Neuromodels for Medical Diagnostics, 2nd International Workshop on Informatics and Data-Driven Medicine (IDDM 2019), Lviv, 11–13 November, 2019. proceedings. Lviv, CEUR WS, 2019, pp. 109–120.
Leoshchenko S., Oliinyk A., Subbotin S.Adaptive Mechanisms for Parallelization of the Genetic Method of Neural Network Synthesis, 10th International Conference on Advanced Computer Information Technologies (ACIT 2020), Deggendorf, 16–18 November, proceedings. Ternopil, IEEE, 2020, pp. 446–450, DOI: 10.1109/ACIT49673.2020.9208905.
South German Credit Data Set [Electronic resource]. Access mode: https://archive.ics.uci.edu/ml/datasets/South+German+Credit
Grömping U. South German Credit Data: Correcting a Widely Used Data Set, Reports in Mathematics, Physics and Chemistry, Department II, Berlin, Beuth University of Applied Sciences Berlin, 2019, 14 p.
Oliinyk A., Subbotin S., Lovkin V., Leoshchenko S., Zaiko T. Feature Selection Based on Parallel Stochastic Computing, 13th International Scientific and Technical Conference on Computer Sciences and Information Technologies (CSIT’2018), Lviv, 11–14 September 2018, proceedings. Lviv, IEEE, 2018, P. 347–351. DOI: 10.1109/STCCSIT.2018.8526729
Oliinyk A., Subbotin S., Lovkin V., Leoshchenko S., Zaiko T. Development of the indicator set of the features informativeness estimation for recognition and diagnostic model synthesis, 14th International Conference on Advanced Trends in Radioelectronics, Telecommunications and Computer Engineering (TCSET 2018) Slavsko, 20–24 February 2018, proceedings. Lviv, IEEE, 2018, P. 903–908. DOI: 10.1109/TCSET.2018.8336342.
Leoshchenko S., Oliinyk A., Subbotin S., Zaiko Methods of semantic proximity extraction between the lexical units in infocommunication systems, 2017 International ScientificPractical Conference Problems of Infocommunications. Science and Technology (PIC S&T) Kharkiv, 10–13 October 2017, proceedings. Kharkiv, IEEE, 2017, pp. 7–12. DOI: 10.1109/INFOCOMMST.2017.8246137.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 С. Д. Леощенко , А. О. Олійник , С. О. Субботін , Є. О. Гофман , М. Б. Ільяшенко
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Creative Commons Licensing Notifications in the Copyright Notices
The journal allows the authors to hold the copyright without restrictions and to retain publishing rights without restrictions.
The journal allows readers to read, download, copy, distribute, print, search, or link to the full texts of its articles.
The journal allows to reuse and remixing of its content, in accordance with a Creative Commons license СС BY -SA.
Authors who publish with this journal agree to the following terms:
-
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License CC BY-SA that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
-
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
-
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
