AN INTELLIGENT MODEL BASED ON DEEP TRANSFER LEARNING FOR DETECTING ANOMALIES IN CYBER-PHYSICAL SYSTEMS
DOI:
https://doi.org/10.15588/1607-3274-2021-3-11Keywords:
anomaly detection, acoustic signal, transfer learning, spectrogram, scalogram, cyber-physical system.Abstract
Context. The problem of detecting anomalies from signals of cyber-physical systems based on spectrogram and scalogram images is considered. The object of the research is complex industrial equipment with heterogeneous sensory systems of different nature.
Objective. The goal of the work is the development of a method for signal anomalies detection based on transfer learning with the extreme gradient boosting algorithm.
Method. An approach based on transfer learning and the extreme gradient boosting algorithm, developed for detecting anomalies in acoustic signals of cyber-physical systems, is proposed. Little research has been done in this area, and therefore various pre-trained deep neural model architectures have been studied to improve anomaly detection. Transfer learning uses weights from a deep neural model, pre-trained on a large dataset, and can be applied to a small dataset to provide convergence without overfitting. The classic approach to this problem usually involves signal processing techniques that extract valuable information from sensor data. This paper performs an anomaly detection task using a deep learning architecture to work with acoustic signals that are preprocessed to produce a spectrogram and scalogram. The SPOCU activation function was considered to improve the accuracy of the proposed approach. The extreme gradient boosting algorithm was used because it has high performance and requires little computational resources during the training phase. This algorithm can significantly improve the detection of anomalies in industrial equipment signals.
Results. The developed approach is implemented in software and evaluated for the anomaly detection task in acoustic signals of cyber-physical systems on the MIMII dataset.
Conclusions. The conducted experiments have confirmed the efficiency of the proposed approach and allow recommending it for practical use in diagnosing the state of industrial equipment. Prospects for further research may lie in the application of ensemble approaches based on transfer learning to various real datasets to improve the performance and fault-tolerance of cyber-physical systems.
References
Langner R. Stuxnet: dissecting a cyberwarfare weapon, IEEE Security & Privacy, 2011, Vol. 9, No. 3, pp. 49–51. DOI: 10.1109/MSP.2011.67
Ahmed C. M., Zhou J. Challenges and opportunities in CPS security: a physics-based perspective, IEEE Security & Privacy, 2020, Vol. 18, No. 6, pp. 14–22. DOI: 10.1109/MSEC.2020.3002851
Ahmed C. M., Murguia C., Ruths J. Model-based attack detection scheme for smart water distribution networks, Computer and Communications Security: 17th ACM ASIA Conference, Abu Dhabi, 2–6 April 2017, proceedings. 2017, pp. 101–113. DOI: 10.1145/3052973.3053011
Canizo M., Triguero I., Conde A., Onieva E. Multi-head CNN–RNN for multi-time series anomaly detection: an industrial case study, Neurocomputing, 2019, Vol. 363, pp. 246–260. DOI: 10.1016/j.neucom.2019.07.034
Kiseľák J., Lu Y., Švihra J., Szépe P., Stehlík M. “SPOCU”: scaled polynomial constant unit activation function, Neural Computing and Applications, 2021, No. 33, pp. 3385–3401. DOI: 10.1007/s00521-020-05182-1
M. R. G. R., Ahmed C. M., Mathur A. Machine learning for intrusion detection in industrial control systems: challenges and lessons from experimental evaluation, Cybersecurity, 2021, Vol. 4, No. 27, pp. 1–12. DOI: 10.1186/s42400-02100095-5
Morita K., Yano T., Tran K. Q. Anomalous sound detection by using local outlier factor and Gaussian mixture model, 2020 [Electronic resource]. Access mode: http://dcase.community/documents/challenge2020/technical _reports/DCASE2020_Morita_51_t2.pdf
Grollmisch S., Johnson D., Abeßer J., Lukashevich H. IAEO3 – combining OpenL3 embeddings and interpolation autoencoder for anomalous sound detection, 2020 [Electronic resource]. Access mode: http://dcase.community/documents/challenge2020/technical _reports/DCASE2020_Grollmisch_15_t2.pdf
Michau G., Fink O. Fully learnable deep wavelet transform for unsupervised monitoring of high-frequency time series, 2021 [Electronic resource]. Access mode: https://arxiv.org/pdf/2105.00899.pdf
Coelho G., Pereira P., Matos L., Ribeiro A., Nunes E., Ferreira A., Cortez P., Pilastri A. Deep dense and convolutional autoencoders for machine acoustic anomaly detection, Artificial Intelligence Applications and Innovations: 17th IFIP International Conference, Hersonissos, 25–27 June 2021: proceedings. Cham, Springer, 2021, pp. 337–348. DOI: 10.1007/978-3-030-79150-6_27
Tiwari P., Jain Y., Avila A., Monteiro J., Kshirsagar S., Gaballah A., Falk T. H. Modulation spectral signal representation and i-vectors for anomalous sound detection, 2020 [Electronic resource]. Access mode: http://dcase.community/documents/challenge2020/technical _reports/DCASE2020_Tiwari_84_t2.pdf
Abbasi S., Famouri M., Shafiee M. J., Wong A. OutlierNets: highly compact deep autoencoder network architectures for on-device acoustic anomaly detection, Sensors, 2021, Vol. 21, No. 14, pp. 1–12. DOI: 10.3390/s21144805
Müller R., Ritz F., Illium S., Linnhoff-Popien C. Acoustic anomaly detection for machine sounds based on image transfer learning, Agents and Artificial Intelligence, 13th International Conference, 4–6 February 2021, proceedings. pp. 49–56. DOI: 10.5220/0010185800490056
Lin J., Qu L. Feature Extraction based on Morlet wavelet and its application for mechanical fault diagnosis, Journal of Sound and Vibration, 2000, Vol. 234, No. 1, pp. 48–135. DOI: 10.1006/jsvi.2000.2864
Chollet F. Xception: deep learning with depthwise separable convolutions, Computer Vision and Pattern Recognition: 30th IEEE Conference. Honolulu, 21–26 July 2017, proceedings, pp. 1800–1807. DOI: 10.1109/CVPR.2017.195
Sandler M., Howard A., Zhu M., Zhmoginov A., Chen L. C. MobileNetV2: inverted residuals and linear bottlenecks, Computer Vision and Pattern Recognition: IEEE/CVF Conference, Salt Lake City, 18–23 June 2018: proceedings. pp. 4510–4520. DOI: 10.1109/CVPR.2018.00474
Huang G., Liu Z., van der Maaten L., Weinberger K. Q., Densely connected convolutional networks, Computer Vision and Pattern Recognition, 30th IEEE Conference. Honolulu, 21–26 July 2017, proceedings, pp. 2261–2269. DOI: 10.1109/CVPR.2017.243
Szegedy C., Vanhoucke V., Ioffe S., Shlens J., Wojna Z. Rethinking the inception architecture for computer vision, Computer Vision and Pattern Recognition, IEEE Conference. Las Vegas, 27–30 June 2016, proceedings. pp. 2818– 2826. DOI: 10.1109/CVPR.2016.308
Szegedy C., Liu W., Jia Y., Sermanet P., Reed S., Anguelov D., Erhan D., Vanhoucke V., Rabinovich A. Going deeper with convolutions, Computer Vision and Pattern Recognition, IEEE Conference. Boston, 7–12 June 2015, proceedings, pp. 1–9. DOI: 10.1109/CVPR.2015.7298594
Kassani S. H., Kassani P. H., Khazaeinezhad R., Wesolowski M. J., Schneider K. A., Deters R. Diabetic retinopathy classification using a modified Xception architecture, Signal Processing and Information Technology, IEEE International Symposium, Ajman, 10–12 December 2019, proceedings, pp. 1–6. DOI: 10.1109/ISSPIT47144.2019.9001846
He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition, Computer Vision and Pattern Recognition: IEEE Conference, Las Vegas, 27–30 June 2016: proceedings. pp. 770–778. DOI: 10.1109/CVPR.2016.90
Chen T., Guestrin C. XGBoost: a scalable tree boosting system, Knowledge Discovery and Data Mining, 22nd ACM SIGKDD International Conference. San Francisco, 13–16 August 2016, proceedings, pp. 785–794. DOI: 10.1145/2939672.2939785
Purohit H., Tanabe R., Ichige K., Endo T., Nikaido Y., Suefusa K., Kawaguchi Y. MIMII dataset: sound dataset for malfunctioning industrial machine investigation and inspection, Detection and Classification of Acoustic Scenes and Events, 4th Workshop. New York, 25–26 October 2019, proceedings, pp. 209–213. DOI: 10.33682/m76f-d618
Ferri C., Hernández-Orallo J., Modroiu R. An experimental comparison of performance measures for classification, Pattern Recognition Letters, 2009, Vol. 30, No. 1, pp. 27–38. DOI: 10.1016/j.patrec.2008.08.010
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 L. V. Sukhostat
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Creative Commons Licensing Notifications in the Copyright Notices
The journal allows the authors to hold the copyright without restrictions and to retain publishing rights without restrictions.
The journal allows readers to read, download, copy, distribute, print, search, or link to the full texts of its articles.
The journal allows to reuse and remixing of its content, in accordance with a Creative Commons license СС BY -SA.
Authors who publish with this journal agree to the following terms:
-
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License CC BY-SA that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
-
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
-
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
