THE SYSTEM OF CRITERIA FOR FEATURE INFORMATIVENESS ESTIMATION IN PATTERN RECOGNITION

A. Oliinyk, S. Subbotin, V. Lovkin, O. Blagodariov, T. Zaiko

Abstract


Context. The task of automation of feature informativeness estimation process in diagnostics and pattern recognition problems i solved. The object of the research is the process of informative feature selection. The subject of the research are the criteria of feature informativeness estimation.

Objective. The research objective is to develop the system of criteria for feature informativeness estimation which enables to comput informativeness of interdependent feature sets.

Method. The system of criteria for feature informativeness estimation is proposed. The proposed system is based on the idea tha feature significance is computed according to spatial location of observations of different classes (size of changing of output parameter) The developed criteria system enables to estimate individual and group feature informativeness in classification and regression problems in situations when initial data samples contain redundant and interdependent features as well as observations with missing values. The proposed criteria don’t require to construct models based on the estimated feature combinations, in such a way considerably reducing time and computing costs for informative feature selection. Application of the proposed criteria for estimation and selection of informative feature allows to reduce structural complexity of synthesized diagnosis and recognition models, to raise its interpretability and generalization ability due to removing of insignificant, interdependent and redundant features in diagnostics and pattern recognition problems.

Results. The software which implements the proposed system of criteria for feature informativeness estimation and allows to selec informative features for synthesis of recognition models based on the given data samples has been developed.

Conclusions. The conducted experiments have confirmed operability of the proposed system of criteria for feature informativenes estimation and allow to recommend it for processing of data sets for pattern recognition in practice. The prospects for further researche may include the modification of the known feature selection methods and the development of new ones based on the proposed system o criteria for individual and group feature informativeness estimation.

Keywords


Data sample; pattern recognition; feature selection; informativeness criterion; individual informativeness; group informativeness.

Full Text:

PDF

References


Jensen R., Shen Q. Computational intelligence and feature selection: rough and fuzzy approaches. Hoboken, John Wiley & Sons, 2008, 339 p. DOI: 10.1002/9780470377888.

Mulaik S. A. Foundations of Factor Analysis. Boca Raton, Florida, CRC Press, 2009, 548 p.

Lee J. A., Verleysen M. Nonlinear dimensionality reduction. New York, Springer, 2007, 308 p. DOI: 10.1007/978-0-387-39351-3.

Bezdek J. C. Pattern Recognition with Fuzzy Objective Function Algorithms. N.Y., Plenum Press, 1981, 272 p. DOI: 10.1007/978-1-4757-0450-1.

Hyvarinen A., Karhunen J., Oja E. Independent component analysis. New York, John Wiley & Sons, 2001, 481 p. DOI: 10.1002/0471221317.

Fedotov N. G. Teorija priznakov raspoznavanija obrazov na osnove stohasticheskoj geometrii i funkcional’nogo analiza. Moscow, Fizmatlit, 2010, 304 p. (In Russian).

Guyon I., Elisseeff A. An introduction to variable and feature selection, Journal of machine learning research, 2003, No. 3, pp. 1157–1182.

McLachlan G. Discriminant Analysis and Statistical Pattern Recognition. New Jersey, John Wiley & Sons, 2004, 526 p. DOI: 10.1002/0471725293.

Oliinyk A. A., Skrupsky S. Yu., Shkarupylo V. V., Blagodariov O. Parallel multiagent method of big data reduction for pattern recognition, Radio Electronics, Computer Science, Control, No. 2. 2017, pp. 82–92.

Oliinyk A. Production rules extraction based on negative selection, Radio Electronics, Computer Science, Control, 2016, Vol. 1, pp. 40–49. DOI: 10.15588/1607-3274-2016-1-5.

Oliinyk A., Skrupsky S., Subbotin S., Blagodariov O., Gofman Ye. Parallel computing system resources planning for neuro-fuzzy models synthesis and big data processing, Radio Electronics, Computer Science, Control, 2016, Vol. 4, pp. 61–69. DOI: 10.15588/1607-3274-2016-4-8.

Oliinyk A. A., Skrupsky S. Yu., Shkarupylo V. V., Subbotin S. A. The model for estimation of computer system used resources while extracting production rules based on parallel computations, Radio Electronics, Computer Science, Control, 2017, No. 1, pp. 142–152. DOI: 10.15588/1607-3274-2017-1-16.

Subbotin S., Oliinyk A. The Sample and Instance Selection for Data Dimensionality Reduction, Recent Advances in Systems, Control and Information Technology. Advances in Intelligent Systems and Computing, 2017, Vol. 543, pp. 97–103. DOI: 10.1007/978-3-319-48923-0_13.

Shitikova O. V., Tabunshchyk G. V. Method of Managing Uncertainty in Resource-Limited Settings, Radio Electronics, Computer Science, Control, 2015, No. 2, pp. 87–95. DOI: 10.15588/1607-3274-2015-2-11.

Tabunshchyk G. V., Kaplienko T. I., Shitikova O. V. Verification model of systems with limited resources, Radio Electronics, Computer Science, Control, 2017, No. 4.

Bodyanskiy Ye., Vynokurova O. Hybrid adaptive wavelet-neuro-fuzzy system for chaotic time series identification, Information Sciences, 2013, Vol. 220, pp. 170–179. DOI: 10.1016/j.ins.2012.07.044.

Kononenko I. Estimating Attributes: Analysis And Extensions Of Relief, Machine Learning : European Conference on Machine Learning ECML-94, Catania, 6–8 April 1994 : proceedings of the conference. Berlin, Springer, 1994, pp. 171–182. DOI:10.1007/3-540-57868-4_57.

Kira K., Rendell L. A practical approach to feature selection, Machine Learning : International Conference on Machine Learning ML92, Aberdeen, 1–3 July 1992 : proceedings of the conference. New York, Morgan Kaufmann, 1992, pp. 249–256. DOI: 10.1016/B978-1-55860-247-2.50037-1.

Salfner F., Lenk M., Malek M. A survey of online failure prediction methods, ACM computing surveys, 2010, Vol. 42, Issue 3, pp. 1–42. DOI: 10.1145/1670679.1670680.

Shin Y. C. Intelligent systems : modeling, optimization, and control / C. Y. Shin, C. Xu. – .Boca Raton, CRC Press, 2009, 456 p. DOI: 10.1201/9781420051773.

Oliinyk A. A., Subbotin S. A., Skrupsky S. Yu., Lovkin V. M., Zaiko T. A. Information Technology of Diagnosis Model Synthesis Based on Parallel Computing, Radio Electronics Computer Science Control, 2017, No. 3, pp. 139–151.

Subbotin S., Oliinyk A., Oliinyk O. Noniterative, evolutionary and multi-agent methods of fuzzy and neural network models synthesis : monograph. Zaporizhzhya, ZNTU, 2009, 375 p. (In Ukrainian).


GOST Style Citations


1. Jensen R. Computational intelligence and feature selection: rough and fuzzy approaches / R. Jensen, Q. Shen. – Hoboken : John Wiley & Sons, 2008. – 339 p. DOI: 10.1002/9780470377888.

2. Mulaik S. A. Foundations of Factor Analysis / S. A. Mulaik. – Boca Raton, Florida: CRC Press. – 2009. – 548 p.

3. Lee J. A. Nonlinear dimensionality reduction / J. A. Lee, M. Verleysen. – New York : Springer, 2007. – 308 p.  DOI: 10.1007/978-0-387-39351-3.

4. Bezdek J. C. Pattern Recognition with Fuzzy Objective Function Algorithms / J. C. Bezdek. – N.Y. : Plenum Press, 1981. – 272 p. DOI: 10.1007/978-1-4757-0450-1.

5. Hyvarinen A. Independent component analysis / A. Hyvarinen, J. Karhunen, E. Oja. – New York : John Wiley & Sons, 2001. – 481 p. DOI: 10.1002/0471221317.

6. Федотов Н. Г. Теория признаков  распознавания образов на основе стохастической геометрии и функционального анализа / Н. Г. Федотов. – М. : Физматлит, 2010. – 304 с.

7. Guyon I. An introduction to variable and feature selection / I. Guyon, A. Elisseeff // Journal of machine learning research. – 2003. – № 3. – P. 1157–1182.

8. McLachlan G. Discriminant Analysis and Statistical Pattern Recognition / G. McLachlan. – New Jersey : John Wiley & Sons. – 2004. – 526 p. DOI:  10.1002/0471725293.

9. Parallel multiagent method of big data reduction for pattern recognition / [A. A. Oliinyk, S. Yu. Skrupsky, V. V. Shkarupylo, O. Blagodariov] // Радіоелектроніка, інформатика, управління. – 2017. – № 2. – С. 82–92.

10. Oliinyk A. Production rules extraction based on negative selection / A. Oliinyk // Радіоелектроніка,  інформатика, управління. – 2016. – №. 1. – С. 40–49. DOI: 10.15588/1607-3274-2016-1-5.

11. Oliinyk A. Parallel computing system resources planning for neuro-fuzzy models synthesis and big data processing / [A. Oliinyk, S. Skrupsky, S. Subbotin et al] // Радіоелектроніка, інформатика, управління. – 2016. – №. 4. – С. 61–69. DOI: 10.15588/1607-3274-2016-4-8.

12. The model for estimation of computer system used resources while extracting production rules based on parallel computations / [A. A. Oliinyk, S. Yu. Skrupsky, V. V. Shkarupylo, S. A. Subbotin] // Радіоелектроніка, інформатика, управління. – 2017. – № 1. – С. 142–152. DOI: 10.15588/1607-3274-2017-1-16.

13. Subbotin S. The Sample and Instance Selection for Data Dimensionality Reduction / S. Subbotin, A. Oliinyk // Recent Advances in Systems, Control and Information Technology. Advances in Intelligent Systems and Computing. –  2017. – Vol. 543. – P. 97–103. DOI: 10.1007/978-3-319-48923-0_13.

14. Shitikova O. V. Method of Managing Uncertainty in Resource-Limited Settings / O. V. Shitikova, G. V. Tabunshchyk // Радіоелектроніка, інформатика, управління. – 2015. – № 2. – P. 87– 95. DOI: 10.15588/1607-3274-2015-2-11.

15. Tabunshchyk G. V. Verification model of systems with limited resources / G. V. Tabunshchyk, T. I. Kaplienko, O. V. Shitikova // Радіоелектроніка, інформатика, управління. – 2017. – № 4.

16. Bodyanskiy Ye. Hybrid adaptive wavelet-neuro-fuzzy system for chaotic time series identification / Ye. Bodyanskiy, O. Vynokurova // Information Sciences. – 2013. – Vol. 220. – P. 170–179. DOI: 10.1016/j.ins.2012.07.044.

17. Kononenko I. Estimating Attributes: Analysis And Extensions Of Relief / I. Kononenko // Machine Learning : European Conference on Machine Learning ECML-94, Catania,  6–8 April 1994 : proceedings of the conference. – Berlin : Springer, 1994. – P. 171– 182. DOI: 10.1007/3-540-57868-4_57.

18. Kira K. A practical approach to feature selection / K. Kira, L. Rendell // Machine Learning : International Conference on Machine Learning ML92, Aberdeen, 1–3 July 1992 : proceedings of the conference. – New York: Morgan Kaufmann, 1992. – P. 249–256. DOI: 10.1016/B978-1-55860-247-2.50037-1.

19. Salfner F. A survey of online failure prediction methods / F. Salfner, M. Lenk, M. Malek // ACM computing surveys. – 2010. – Vol. 42, Issue 3. – P. 1–42. DOI: 10.1145/1670679.1670680.

20. Shin Y. C. Intelligent systems : modeling, optimization, and control / C. Y. Shin, C. Xu. – Boca Raton : CRC Press, 2009. – 456 p. DOI: 10.1201/9781420051773.

21. Oliinyk A. A. Information Technology of Diagnosis Model Synthesis Based on Parallel Computing / [A. A. Oliinyk, S. A. Subbotin, S. Yu. Skrupsky et al] // Радіоелектроніка, інформатика, управління. – 2017. – № 3. – С. 139–151.

22. Субботін С. О. Неітеративні, еволюційні та мультиагентні методи синтезу нечіткологічних і нейромережних моделей: монографія / С. О. Субботін, А. О. Олійник, О. О. Олійник ; під заг. ред. С.О. Субботіна. – Запоріжжя : ЗНТУ , 2009. – 375 с.




DOI: https://doi.org/10.15588/1607-3274-2017-4-10



Copyright (c) 2017 A. Oliinyk, S. Subbotin, V. Lovkin, O. Blagodariov, T. Zaiko

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Address of the journal editorial office:
Editorial office of the journal «Radio Electronics, Computer Science, Control»,
Zaporizhzhya National Technical University, 
Zhukovskiy street, 64, Zaporizhzhya, 69063, Ukraine. 
Telephone: +38-061-769-82-96 – the Editing and Publishing Department.
E-mail: rvv@zntu.edu.ua

The reference to the journal is obligatory in the cases of complete or partial use of its materials.