METHOD FOR NUMERICAL DIFFERENTIATION OF NOISY DATA WITH OUTLIERS
DOI:
https://doi.org/10.15588/1607-3274-2017-3-5Keywords:
Numerical differentiation, smooth approximation, minimum-extent criteria.Abstract
Context. Using of conventional methods of numerical differentiation to the noisy data with outliers leads to significant errors. The objectof this study is the process of numerical differentiation of such data.
Objective. The goal of this work is the development of a method of numerical differentiation of the noisy data with outliers to obtain a smooth approximation of the first derivative of original data as well as a smooth approximation of the original data themselves.
Method. The proposed method of numerical differentiation is based on solving the problem of minimizing the smoothing functional, which is built on the criteria of a minimum of extent of the solution residual and of an energy constraint of the first derivative of solution. The minimum-extent criterion defines the main part of functional and ensures its stable behavior with respect to the additive noise and outliers. The energy constraint defines the stabilizing part of the functional and provides a smooth solution of the problem. The contribution of these parts is controlled by a regularization parameter. Since the main part of smoothing functional is not convex, then the minimization problem is the non-convex nonlinear programming problem. For the numerical solution of this problem the conjugate gradient method is used. In this method the step size along the descent direction is defined on the set of test steps. These steps minimize the individual components of the main and stabilizing parts of the smoothing functional that allows to move from the one local minimum of the functional to another deeper local minimum.
Results. Simulation of the problem of numerical differentiation of noisy data with outliers and processing of the experimental data, which are photoluminescence spectra with narrow line components in their compositions, confirmed the performance of the proposed method.
Conclusions. The proposed method can be used for numerical differentiation of noisy data with outliers. It provides a smooth approximation of the first derivative of the original data, as well as a smooth approximation of the original data themselves. This method can be generalized to the case of non-smooth solutions by constructing a stabilizing part of the functional based on the criterion of minimum total variation.
References
Davies A. R., Anderssen R. S., de Hoog F. R., Goulding N. J. Derivative spectroscopy and the continuous relaxation spectrum, Journal of Non-Newtonian Fluid Mechanics, 2016, Vol. 233, pp. 107–118. DOI: 10.1016/j.jnnfm.2016.01.018.
Lindner R. R., Vera-Ciro C., Murray C. E., Stanimirovic S., Babler B. L., Heiles C., Hennebelle P., Goss W. M., Dickey J. Autonomous Gaussian decomposition, The Astronomical Journal, 2015, Vol. 149, No. 4, pp. 138-149. DOI: 10.1088/0004-6256/149/4/138.
Maitra I. K., Bandhyopadhyaay S. K. Adaptive Edge Detection Method towards Features Extraction from Diverse Medical Imaging Technologies, Intelligent Multidimensional Data Clustering and Analysis, 2016, Chapter 7, pp. 159–192. DOI:10.4018/978-1-5225-1776-4
Miкkina A., Wagner J., Mazurek P., Morawski R. Z. Selected algorithms for measurement data processing in impulse-radarbased system for monitoring of human movements, Journal of Physics, 2016, Vol. Conference Series 772, pp. 1–6. DOI:10.1088/1742-6596/772/1/012057.
Knowles I., Renka R. J. Methods for numerical differentiation of noisy data, Electronic Journal of Differential Equations, 2014, Vol. Conference 21, pp. 235–246.
Chartrand R. Numerical differentiation of noisy, nonsmooth data, ISRN Applied Mathematics, 2011, Vol. 2011, pp. 1–11. DOI: 10.5402/2011/164564.
Liao H., Li Y., Brooks G. Outlier Impact and Accommodation Methods: Multiple Comparisons of Type I Error Rates, Journal of Modern Applied Statistical Methods, 2016, Vol. 15, No. 1, pp. 452–471.
Borulko V. F., Vovk S. M. Principle of minimum extent in spatial spectrum extrapolation problems of complex-valued sources, Telecommunications and Radio Engineering, 2013, Vol. 72, No. 7, pp. 581–592. DOI:10.1615/TelecomRadEng.v72.i7.30.
Vovk S. M., Borulko V. F. Dvoystvennyiy metod minimuma prostranstvennoy protyazhennosti dlya robastnogo otsenivaniya parametrov dipolnyih istochnikov izlucheniya, Radio Electronics, Computer Science, Control, 2014, No. 2, pp. 8–17. DOI:10.15588/1607-3274-2014-2-1.
Vovk S. M. General approach to building the methods of filtering based on the minimum duration principle, Radioelectronics and Communications Systems, 2016, Vol. 59, No. 7, pp. 281–292. DOI:10.3103/S0735272716070013.
Gonzalez J. G., Arce G. R. Optimality of the myriad filter in practical impulsive-noise environments, IEEE Trans. on Signal Processing, 2001, Vol. 49, No. 2, pp. 438–441. DOI: 10.1109/78.902126.
Aysal T. C., Barner K. E. Meridian filtering for robust signal processing, IEEE Trans. on Signal Processing, 2007, Vol. 55, No. 8, pp. 3949–3962. DOI: 10.1109/TSP.2007.894383
Downloads
How to Cite
Issue
Section
License
Copyright (c) 2017 S. M. Vovk
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Creative Commons Licensing Notifications in the Copyright Notices
The journal allows the authors to hold the copyright without restrictions and to retain publishing rights without restrictions.
The journal allows readers to read, download, copy, distribute, print, search, or link to the full texts of its articles.
The journal allows to reuse and remixing of its content, in accordance with a Creative Commons license СС BY -SA.
Authors who publish with this journal agree to the following terms:
-
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License CC BY-SA that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
-
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
-
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
