DOI: https://doi.org/10.15588/1607-3274-2017-4-8

ADVANCED TECHNOLOGIES OF BIG DATA RESEARCH IN DISTRIBUTED INFORMATION SYSTEMS

N. І. Boyko

Abstract


Context. Considered question correct interpretation information flow in distributed information systems. The object of study methods are promotion “big data” on cluster system.

Objective. Is the study promising areas and technology for the analysis of structures data in distributed information systems.

Method. The big data tendency prospects as well as timeliness of the problem are studied in this paper. The principles of work with them are addressed. Big data processing technologies are provided. The analysis of each one is performed. An example of “MapReduce” paradigm application, uploading of big volumes of data, processing and analyzing of unstructured information and its distribution into the clustered database is provided. The article summarizes the concept of “big data”. Examples of methods for working with arrays of unstructured data. Dedicated scientific guidance for analyzing big data. The principles of unstructured data in distributed information systems. Driven work platform “Hadoop MapReduce” and “Apache Spark”. Analyzed their properties and given the differences. An analysis of comparative performance against both platforms – the performance of the number of iterations. Consider ways to create RDD: parallelization  transmitted collection program and a link to an external file system in “Hadoop”. There is an example rozparalelenoyi system RDD. Proposed work lone class for basic database operations: database connection, create a table, a table, get in line id, returning all elements of the database, update, delete and create the line.

Results. The analysis Models Spark and Hadoop MapReduce for phased construction distributed information system. built up SparkConf object, containing information about applique and is the final version of the experiment.

Conclusions. Conducted experiment confirmed efficiency the proposed method, are capable process horizontal data arrays, that parallelization by defective presentation of information. These  promising areas of analyze structure data for the purpose of forecast results and create algorithms advanced correlation, contributing new understanding activity distributed information systems further research can consist in wide use information systems, that would provide a full range technological process adaptation information flows in clusters.

Keywords


System; technology; big data; information; technique; database; Web application; modeling; processing; analytics.

References


What is Big Data [Electronic resource]. Access mode: http://datascience.berkeley.edu/what-is-big-data/

Shaw J. Why “Big Data” Is a Big Deal [Electronic resource]. Rezhym dostupu: http://harvardmag.com/pdf/2014/03-pdfs/0314-HarvardMag.pdf

Schutt P. What is Big Data? [Electronic resource]. Rezhym dostupu: https://blogs.oracle.com/bigdata/big-data-and-analytic-top-10-trends-for-2014

Boyko N., Pobereyko P. Basic concepts of dynamic recurrent neural networks development, ECONTECHMOD : an international quarterly journal on economics of technology and modelling processes. Lublin, Polish Academy of Sciences, 2016, Vol. 5, No. 2, pp. 63–68.

Leskovec J., Rajaraman A., Ullman J. D. Mining of massive datasets. Massachusetts, Cambridge University Press, 2014, 470 р.

Mayer-Schoenberger V., Cukier K. A revolution that will transform how we live, work, and think. Boston New York, 2013, 230 р.

Boyko N. A look trough methods of intellectual data analysis and their applying in informational systems, Komp”yuterni nauky ta informatsiyni tekhnolohiyi CSIT 2016 : Materialy XI Mizhnarodnoyi naukovo-tekhnichnoyi konferentsiyi CSIT 2016 : proceedings. L’viv, Vydavnytstvo L’vivs’koyi politekhniky, 2016, pp. 183–185.

Benderskaia E. N., Zhukova S. V. Ostsilliatornye neironnye seti s khaoticheskoi dinamikoi v zadachakh klasternogo analiza, Neirokomp”iutery: razrabotka, primenenie; Radiotekhnika : proceedings. Moscow, Radyotekhnyka, 2011, No. 7, pp. 74–86.

Benderskaia E. N., Nikitin K. V. Modelirovanie neironnoi aktivnosti mozga i bionspirirovannye vychisleniia, Nauchno-tekhnicheskie vedomosti SPbGPU. Informatika. Telecommunicatcii. Upravlenie : proceedings. St.-Petersburg, Izd-vo Politehn. un-ta, 2011, No. 6–2(138), pp. 34–40.

Benderskaia E. N. Vozmozhnosti primeneniia nekotorykh kharakteristik sinkhronizatsii dlia vyiavleniia samoorganizuiushchikhsia klasterov v ostsilliatornoi neironnoi seti s khaoticheskoi dinamikoi, Neirokomp”iutery: razrabotka, primenenie: nauchno-tekhnicheskii zhurnal : proceedings. Moscow, Nauchnyi tsentr neirokomp”iuterov, 2012, No. 11, pp. 69–73.

Feng J., Brown D. Fixed-point attractor analysis for a class of neurodynamics, Neural Computation : proceedings. Massachusetts, MIT Press Cambridge, 1998, Vol. 10, pp. 189–213.

Kaneko K. Life: an introduction to complex systems biology. Berlin, Springer-Verlag, 2006, 369 p.

Maass W., Natschger T., Markram H. Real-time computing without stable states: a new framework for neural computations based on perturbations, Neural Computation : proceedings. Switzerland, Institute for Theoretical Computer Science, 2002, Vol. 11, pp. 2531–2560.

Schrauwen B., Verstraeten D., Campenhout J. V. An overview of reservoir computing theory, applications and implementations, Proc. of the 15th European Symp. on Artificial Neural Networks : proceedings. Belgium, Bruges, 2007, pp. 471–482.

Coombes S. Waves, bumps, and patterns in neural field theories, Biological Cybernetics : proceedings. Nottingham, University of Nottingham, 2005, Vol. 93, No. 2, pp. 91–108.


GOST Style Citations


1. What is Big Data [Electronic resource] – Access mode: http://datascience.berkeley.edu/what-is-big-data/

2. Shaw J. Why “Big Data” Is a Big Deal [Electronic resource] / J. Shaw. – Режим доступу: http://harvardmag.com/pdf/2014/03-pdfs/0314-HarvardMag.pdf

3. Schutt P. What is Big Data? [Electronic resource] / Р. Schutt. – Режим доступу: https://blogs.oracle.com/bigdata/big-data-and-analytic-top-10-trends-for-2014

4. Boyko N. Basic concepts of dynamic recurrent neural networks development / N. Boyko, P. Pobereyko // ECONTECHMOD : an international quarterly journal on economics of technology and modelling processes. – Lublin : Polish Academy of Sciences, 2016. – Vol . 5 , № 2. – P. 63–68.

5. Leskovec J. Mining of massive datasets / J. Leskovec, A. Rajaraman, J. D. Ullman. – Massachusetts : Cambridge University Press, 2014. – 470 р.

6. Mayer-Schoenberger V. A revolution that will transform how we live, work, and think / V. Mayer-Schoenberger, K. Cukier. – Boston New York, 2013. – 230 р.

7. Boyko N. A look trough methods of intellectual data analysis and their applying in informational systems / N. Boyko // Komp”yuterni nauky ta informatsiyni tekhnolohiyi CSIT 2016 : Materialy XI Mizhnarodnoyi naukovo-tekhnichnoyi konferentsiyi CSIT 2016 : proceedings. – L’viv : Vydavnytstvo L’vivs’koyi politekhniky, 2016. – P. 183–185.

8. Benderskaia E. N. Ostsilliatornye neironnye seti s khaoticheskoi dinamikoi v zadachakh klasternogo analiza / E. N. Benderskaia, S. V. Zhukova // Neirokomp’iutery: razrabotka, primenenie; Radiotekhnika : proceedings. – Moscow : Radyotekhnyka, 2011. – № 7. – P. 74–86.

9. Benderskaia E. N. Modelirovanie neironnoi aktivnosti mozga i bionspirirovannye vychisleniia / E. N. Benderskaia, K. V. Nikitin // Nauchno-tekhnicheskie vedomosti SPbGPU. Informatika. Telecommunicatcii. Upravlenie : proceedings. – St.-Petersburg : Izd-vo Politehn. un-ta. – 2011. – № 6–2(138). – P. 34–40.

10. Benderskaia E. N. Vozmozhnosti primeneniia nekotorykh kharakteristik sinkhronizatsii dlia vyiavleniia samoorganizuiushchikhsia klasterov v ostsilliatornoi neironnoi seti s khaoticheskoi dinamikoi / E.N. Benderskaia // Neirokomp’iutery: razrabotka, primenenie: nauchno-tekhnicheskii zhurnal : proceedings. – Moscow : Nauchnyi tsentr neirokomp’iuterov, 2012. – № 11. – P. 69–73.

11. Feng J. Fixed-point attractor analysis for a class of neurodynamics / J. Feng, D. Brown // Neural Computation : proceedings. – Massachusetts : MIT Press Cambridge, 1998. – Vol. 10. – P. 189–213.

12. Kaneko K. Life: an introduction to complex systems biology / K. Kaneko. – Berlin : Springer-Verlag, 2006. – 369 p.

13. Maass W. Real-time computing without stable states: a new framework for neural computations based on perturbations / W. Maass, T. Natschger, H. Markram // Neural Computation : proceedings. – Switzerland:  Institute for Theoretical Computer Science, 2002. – Vol. 11. – P. 2531–2560.

14. Schrauwen B. An overview of reservoir computing theory, applications and implementations / B. Schrauwen, D. Verstraeten, J. V. Campenhout // Proc. of the 15th European Symp. on Artificial Neural Networks : proceedings. – Belgium : Bruges, 2007. – P. 471–482.

15. Coombes S. Waves, bumps, and patterns in neural field theories / S. Coombes // Biological Cybernetics : proceedings. – Nottingham : University of Nottingham, 2005. – Vol. 93, № 2. – P. 91–108.






Copyright (c) 2017 N. І. Boyko

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Address of the journal editorial office:
Editorial office of the journal «Radio Electronics, Computer Science, Control»,
Zaporizhzhya National Technical University, 
Zhukovskiy street, 64, Zaporizhzhya, 69063, Ukraine. 
Telephone: +38-061-769-82-96 – the Editing and Publishing Department.
E-mail: rvv@zntu.edu.ua

The reference to the journal is obligatory in the cases of complete or partial use of its materials.