METHOD AND SOFTWARE COMPONENT MODEL FOR SKIN DISEASE DIAGNOSIS

Context. The problem of skin disease diagnosis was investigated in the paper. Its actuality is caused by the necessity of automation of at least advisory medical decision making. Such decisions are made in telemedicine, for instance, when skin disease diagnostics is performed under specific conditions. These conditions are specified by situations when data for analysis are collected but a qualified doctor has no possibility to process the data and to make a diagnosis decision based on it. The object of the study is a process of skin disease diagnosis. Objective. The objective of the study is to develop a skin disease diagnosis method to automate making of advisory medical diagnosis decisions and to increase efficiency of such decisions. Method. The skin disease diagnosis method was proposed in the work. This method applies the modified ResNet50 model. It was proposed to add layers to the ResNet50 model and to train it using transfer learning and fine-tuning techniques. The method also defines image processing in particular through the change of its resolution and uses oversampling technique to prepare a dataset for model training. Results. Experimental investigation of the proposed method was performed using the HAM10000 dataset which contains images of skin diseases. The images were collected using dermatoscopy method. The dataset contains observations for 7 different skin diseases. The proposed method is characterized by the accuracy of 96.31% on this dataset. It is improved accuracy in comparison with the existing neural network models. Software component model was created to give a possibility to integrate the proposed method into a medical diagnosis system. Conclusions. The obtained results of the investigation suggest application of the proposed skin disease method in medical diagnostic system to make advisory decisions by the system and to support making final decisions by a doctor.


ABBREVIATIONS
XGBoost is eXtreme Gradient Boosting.

NOMENCLATURE  is a standard deviation of pixels;
A is an accurancy; C is a number of correctly classified observations; D is a set of diagnoses; d A is a skin disease diagnosis for an image of skin area which was not diagnosed by a dermatologist as it was classified by a model; d i is an actual skin disease diagnosis for the i-th observation in a dataset; d i C is a skin disease diagnosis for the i-th observation in a dataset as it was classified by a model; d i is a class of the i-th observation in a dataset; f is a functional dependence; M is a number of misclassified observations; N is a number of observations in a dataset; p i is the i-th image in a dataset; p i av is an average pixel intensity value calculated through all values of the i-th image; p i q,k is a pixel intensity in all channels for pixel in the q-th row and k-th column of the i-th image; pn i q,k is a normalized pixel intensity in all channels for a pixel in the q-th row and k-th column of the i-th image; S is a dataset; S tr is a training dataset; S t is a test dataset; S v is a validation dataset; s A is an image of skin area which was not diagnosed by a dermatologist; s i is an image of the i-th observation in a dataset.

INTRODUCTION
Despite active current progress of medicine, skin diseases are one of the most dangerous and widespread types of diseases.At the same time cases of skin diseases increase in number.Such trends are caused by either unfavorable external factors (for instance, environmental degradation, climate change, influence of ultraviolet radiation) or individual factors (chronic stress, heredity, unhealthy lifestyle, bad quality of diagnostics or untimely diagnostics).This year the trends are additionally reinforced in Ukraine by the war.Access of displaced persons or Ukrainian citizens in general to quality medicine (specialists with experience in a specified area) is complicated in some territories.
According to the data presented by WHO, the number of cases with detected nonmelanoma skin cancer increased to the level of 1.2 million and the number of melanoma cases reached more than 0.3 million in 2020 [1].The percentage of fatal cases in these cases is the following: 5% for nonmelanoma skin cancer and 17% for melanoma skin cancer [1].
Official statistics in Ukraine confirm skin cancer as the cancer type with the biggest number of cases: there were more than 180 thousand of active cases in 2020 [2].
Skin diseases are more easily defined in comparison with the other types of cancer in particular.But people do not always consider its symptoms, paying necessary attention in visiting a doctor.Such a way may not only be the reason for serious health problems but also for a death of a human.
The source [3] states that only 40% of nonmelanoma skin tumors are detected in the early stages (the first and the second stages) in Ukraine.It should be noted that early detection guarantees effective treatment of a patient.At the same time melanoma is detected unfrequently at the initial stage in Ukraine.But in this case a disease is best treated.
The low quality of diagnostics is caused either by low level of patient information awareness regarding these diseases or by low level of procedure quality.The last one is connected with insufficient qualifications of a doctor, lack of necessary equipment, similarity of disease symptoms, analyses collected with procedural violations, human factor.
Diagnostic process includes gathering of data on patient's condition and drawing a conclusion on health issues.The process is realized through the following stages: -detection of health issues by a patient; -patient's referring to a medical institution for medical aid; -collection, integration and interpretation of patient's medical data; -generation of hypothesis on illness; -confirmation/update of hypothesis via additional obtained data; -making final diagnosis [4].
Gathering of patient's medical data for skin diseases requires physical examination of a patient.The examination is traditionally organized using dermatoscopy method.It requires to capture an image of an examined skin area and to estimate the following criteria: -skin pigmentation change; -skin color uniformity; -structure characteristics; -skin contours and pattern change; -skin peeling and inflammation; -skin tightening; -skin cracks and ulcer areas [5].
Estimations of the presented criteria results in making a diagnosis and determining a treatment.
The object of the study is a process of skin disease diagnosis.
Because these diseases have a great impact on the quality of human life and can be dangerous for human health, timely detection of skin neoplasms with increasing accuracy of classification without doctor participation is actual and important problem which should be solved.This classification is necessary in telemedicine support systems, in systems where advisory decisions are made to decrease working time of a doctor with the necessary qualification.
Dermatologist makes his job in a way similar to image recognition by neural network: it demands input image (skin lesions) processed through neural network (skills and knowledge of dermatologist who analyses an image and synthesizes data) to classify it (to make a diagnosis) [6].In addition, early diagnostics of a disease is one of necessary conditions for successful treatment of a disease, so modern approaches and methods of diagnosis support based on AI (in particular CNN models) should be applied to increase classification accuracy.
In general, neural network models are widely used in the field of dermatology presently taking into account the described similarity between real process and such models.ANN models are assumed to be efficient diagnosis tool [6,7].It can be applied to improve recognition results and to decrease number of misdiagnosed cases.
The subject of the study are ANN models which are applied for diagnostics of skin diseases based on classification of images which present skin state observations collected using dermatoscopy method.
The objective of the study is to develop a skin disease diagnosis method to automate making of advisory medical diagnosis decisions and to increase efficiency of such decisions.

PROBLEM STATEMENT
Skin disease diagnosis problem lies in a recognition of skin area images using appropriate mathematical model.In this work the problem should be specified using CNN as a mathematical model.
Let S be a dataset presenting skin disease cases with known diagnoses.
The dataset S = {s i }, i = 1, 2, ..., N consists of N observations s i .Each i-th observation consists of a pair s i = <p i , d i >, where p i is an object which presents image of skin area and d i is a diagnosis which was made by dermatologist based on an image p i .
Object p i is an array {p i q,k } of elements.Each element p i q,k is a record of intensity for pixel in the q-th row and k-th column.This record presents pixel intensity for each channel of an image.
Let D be a set of diagnoses.A set D contains the following elements: - In this study the diagnoses from d 0 to d 6 are used in a set D. This set could be extended if it is necessary in the medical diagnostic system and there is appropriate dataset.
The skin disease diagnosis problem is to determine a functional dependence f This functional dependence should define a model f minimizing accuracy A.
Accuracy A is calculated as a ratio of correctly classified C observations to misclassified M: A = C / M • 100.Number C is calculated as a total number of correctly classified observations between N observations of a dataset.Number M is calculated as a total number of misclassified observations between N observations of a dataset.
The defined model f should be used to classify new diagnosis case s A into a class d A ϵ D.

REVIEW OF THE LITERATURE
AI-based approach to medical diagnostics in general and to diagnostics of skin diseases in particular became widespread recently [8][9].In a few years a lot of AIbased diagnosis methods were investigated.To develop an own method to support one of the stages of medical diagnostic system according to the objective of the study it is necessary to analyze existing solutions in the beginning.
In the paper [10] different classifiers, including SVM, LR, RF, AdaBoost, BB, BRF, were investigated for melanoma detection using images collected by dermatoscopy method.The best results were obtained by BRF method.SVM, LR and RF were unstable.AdaBoost was a stable classifier but with the lowest AUC value between all the considered methods.AUC was used because the problem in the paper was considered as a binary classification problem.Classification was performed for two classes: melanoma and benign tumor.The main shortcoming of the study is in the binary classification.At the same time this study allowed to compare different classifiers.
The methods investigated in the paper [10] are still applied for images recognition in medical diagnosis but ANN became the most actively used model for solving these problems.ANN popularity is conditioned on its higher productivity, adaptability to different types of objects and its ability to increase classification accuracy [11].
In the paper [12] the method for diagnostics based on the proposed model of ANN was applied for skin disease image recognition.Machine learning methods, including RF, SVM and XGBoost methods, VGG16 and ResNet models, were investigated for the problem solving.As a result of this investigation ANN had significantly better recognition results in comparison with the other considered machine learning methods.The main conclusion of this investigation is in the efficiency and applicability of ANN for the problem solving.The model proposed in the paper [12] was characterized by less estimation results in comparison with VGG16 and ResNet models.The ResNet model had the highest accuracy (90.5%) in the investigation.
The paper [13] was aimed at investigation of different models of CNN, including VGG16, VGG19, MobileNet and InceptionV3 with transfer learning, for recognition of skin disease images.Main feature of the study is in the modification of these models by extended convolution technique, where filter is applied to an area which is bigger than filter length by excluding input values with a step, which is equivalent to a convolution with big filter, created from initial filter by placing zeros in it [14].The highest value of recognition accuracy was provided by InceptionV3 (90.95%) model.The obtained accuracy value should be increased for practical application in medical diagnostic system.
In the paper [15] efficiency of ANN application for skin disease recognition was investigated in comparison with diagnostics by general practitioners.The dataset was processed for usage of two variants of images: images with high and low resolution.VGG16, VGG19, ResNet34, ResNet50, ResNet101, SEResNet50, EfficientNetB5, MobileNet models were investigated.The highest accuracy (83.88% for images with high resolution and 82.47% for images with low resolution) was obtained using MobileNet model.The main result of this investigation is in the higher recognition accuracy of the worst investigated model in comparison with general practitioners.It means that only decisions made by experienced specialists are valuable.At the same time the following decisions could be made using computer systems: -advisory decisions; -decisions in situations when there is no possibility of access to a specialist; -preliminary decisions on what specialist is needed and if this specialist is needed.
The investigated works have such shortcomings as application of outdated methods, classification into less number of classes (for example, the work [10]) and besides low recognition accuracy which is a major indicator for the subject area.These drawbacks are a basis for the next study of skin disease diagnosis methods and models to improve the known results.

MATERIALS AND METHODS
To achieve the objective of the study the skin disease diagnosis method was proposed.It consists of the following stages.
At the first stage of the method data presenting images of skin diseases from a set D collected using dermatoscopy method should be processed to prepare it for a dataset S creation.This preparation procedure includes change of image resolution to guarantee higher speed of model training because of the less size of images in dataset (investigated and confirmed in the study [16]).Higher training speed is important for practical application of the method in medical diagnosis system because it decreases time necessary for update when new data presenting new cases are included to the dataset and model retraining is performed.
At the second stage a dataset S ready for training should be created.
During this stage dataset S should be separated into the following subdatasets: training dataset S tr and validation dataset S v .During this study of the method test dataset S t should be separated from dataset S. Then these subdatasets should be normalized using a procedure which calculates a value of each pixel.Each pixel p i q,k should be transformed into pn i q,k by subtracting average value p i av calculated through all values and by dividing this result by standard pixel deviation : pn i q,k = (p i q,k -p i av ) / .
A dataset is possibly unbalanced, because a number of observations referred to one class may significantly exceed a number of observations referred to another classes.Such an issue notably impacts on model recognition accuracy, because classes, which observations are in minority, may be ignored by a model.It results in misdiagnosing.Oversampling technique should be used to solve this issue.But oversampling has its own practical drawback.After its application images will be duplicated in training dataset for classes, causing model overtraining.It is explained by image learning without its analysis and detection of specified details of each disease.As a result this model will not be able to highlight structural patterns of tumors and could not be applied for new data observations, which were not used for training.Such methods of image transformation as rotation, horizontal and vertical shift, wheeling and scaling should be used to solve the issue.
At the third stage a model, which defines functional dependence f, should be created by modification of pretrained ResNet50 model.This model was chosen, because it fits the problem solved in the study, using connections with quick access, missing one layer or some layers avoiding model overtraining and increasing diagnosis accuracy [17].
The structure of the full modified model is shown in Fig. 1.The structure of identity block of ResNet50 model is shown in Fig. 2.

EXPERIMENTS
Experimental investigation was held using the HAM10000 dataset [20].The dataset contains 10015 images of skin diseases.The images were collected using dermatoscopy method.The dataset contains observations for 7 different skin diseases (AKIEC, BCC, BKL, DF, NV, MEL, VASC).
Histopathology (53% cases), further examination of a patient (37% cases), consensus of experts (9%) and microscopy (1%) were used to detect diagnosis for every case in the dataset.Localization of observations of skin diseases is presented in Fig. 4.
This dataset was separated into subdatasets in the following manner: 60% of data were used as training dataset S tr , 20% were used as validation dataset S v and 20% were used as test dataset S t .
Preliminary analysis of the HAM10000 dataset defined that the biggest number of cases in the dataset represents NV (6705).It is 6 times more than number of MEL (1113) and BKL (1099) cases.Four other diagnoses are represented by even less number of cases.Fraction of every diagnosis in the initial dataset is visualized in Fig. 5. Oversampling technique was applied to balance number of cases in the processed dataset.Number of cases for each diagnosis is presented in Table 1.Experimental investigation was performed using the processed dataset.The following models were investigated: − VGG16 in modifications used in the papers [12], [15], [21]; − VGG19 [15]; − ResNet in modifications used in the papers [12], [21]; -ResNet34 [15]; -ResNet101 [15]; -SEResNet50 [15]; -ResNeXt101 [22]; -AlexNet [21]; -DenseNet [21]; -MobileNet in modifications in the papers [15], [21]; -EfficientNetB5 [15]; -Ensemble [22]; -the proposed model using appropriate method.The proposed model was trained during 60 epochs with Adam optimizer, which is used instead of stochastic gradient descent algorithm and updates weights in network iteratively, making possible to change learning rate for each parameter and to adapt weight values according to the learning process [23].Dropout value was set to 0.5.Last 22 layers of the model were used for training, because fine-tuning technique was applied in the method, providing only last layers training.
Experimental investigation was performed using software which was developed to support method practical application.It was developed using the following tools: -TensorFlow and Keras libraries for model creation and training; -NumPy, Pandas, scikit learn libraries for data processing; -Python programming language; -Matplotlib library for data visualization.
Software component model was created to give a possibility to integrate the proposed method into a medical diagnosis system.Software component model provides software interface to communicate with class model.The interface includes methods for the following tasks: -to create modified model; -to upload a dataset; -to preprocess uploaded dataset; -to train a model; -to save trained model into a file; -to upload saved model from a file; -to upload new case; -to preprocess new diagnosis case; -to classify preprocessed diagnosis case using trained model.
This decision allows to integrate this component model into medical diagnosis system in the future.Medical diagnosis system makes decisions for different tasks using accumulated patient data.This approach allows to add necessary components after this system is developed and to create system after separate component models are created if these component models use the same approach.

RESULTS
During training of the proposed model values of recognition accuracy (Fig. 6) and error function (Fig. 7) presented by categorical crossentropy were visualized in dynamics.Confusion matrix (Table 2) was created to assess recognition quality for every skin disease from the set D. This matrix represents number of images which were true classified and misclassified by the proposed model.A row in the matrix represents actual values, and a column represents results of classification by the proposed model.All observations, which actual skin disease diagnosis d i from the dataset S t is not equal to a value d i C classified by the model, are considered as misclassified.
Accuracy of the proposed and other investigated models was estimated using the test dataset.Results of accuracy comparison are presented in Table 3. 6 DISCUSSION A wide investigation (Table 3) of different CNN models was conducted to clarify its practical applicability for skin disease diagnosis problem solving.For the purpose HAM10000 dataset was used.The dataset includes cases for 7 skin diseases.Since other investigations were mainly conducted for a smaller number of skin disease such an investigation has practical significance.
The results of experimental investigation confirm that propositions for modification of mathematical model based on ResNet50, presented in the appropriate method, allowed to increase classification accuracy.The modified model has accuracy of 96.31% on test dataset, which is 5.81% higher than base ResNet model and which is 3.11% higher than the best model in a group of other investigated models (ResNeXt101).The best results were obtained by ResNet (90.5%),Ensemble (92.83%),ResNeXt101 (93.2%) and the modified ResNet50 model.Other models had accuracy lower than 90%.
Accuracy of models with less number of layers was lower: VGG16, VGG19, ResNet34, MobileNet models all had accuracy of less than 90% and closer to 80%.It approves that such a number of layers is not enough to learn all patterns necessary to recognize all diseases which were analyzed.
The proposed model allowed to get the number of misclassified observations which was not higher than 5% for all diagnoses (Fig. 10).All AKIEC and DF cases were true classified (Table 2).NV had 1349 observations in test dataset.The proposed model misclassified 63 observations: 4 cases were diagnosed as BCC, 22 cases as BKL, 23 cases as DF, 13 cases as MEL and 1 case as VASC.This diagnosis is characterized by the lowest accuracy of 95.33%.
The modified ResNet50 model has additional layers, in such a way having large demands on dataset necessary for model training.Oversampling technique together with transformation methods, applied to dataset images, allowed to get a wider training dataset with sufficient number of observations in it.Images used as input for the model had size of 100x75 pixels.Learning curves (Fig. 6 and Fig. 7) together with model training results (Fig. 8) approve this statement, demonstrating that there was no under-or overtraining.The modified model has accuracy of 97.53% on validation dataset These experimental results allow to recommend to integrate the proposed method into medical diagnosis system for practical usage as an advisory block [24].It might have significant effect in countries or territories, where equal and wide access to quality medicine and qualified specialists is impossible or temporarily restricted.

CONCLUSIONS
The problem of skin disease diagnosis was investigated in the paper.It was considered as classification problem.The existing methods for the problem solving are not perfect and may be improved.These methods classify observations into less number of classes and have lower accuracy.The skin disease diagnosis method, based on the modified ResNet50 model, was proposed.
To increase model recognition accuracy it was proposed to perform preliminary data processing to balance dataset, to reduce image size, to normalize values, to transform images inside dataset using different methods to avoid oversampling.
ResNet50 model was modified adding new layers.Dropout layer, layer with 128 neurons and ReLu activation function, dropout layer, and layer with 7 neurons and SoftMax activation function were added to the model.The extended model was trained using transfer learning and fine-tuning techniques Main learning settings were defined.Separate stages were defined in the method to apply the trained model to diagnose new cases.
Experimental investigation was performed using the HAM10000 dataset, which consists of 10015 images of 7 skin diseases (AKIEC, BCC, BKL, DF, NV, MEL, VASC).Accuracy of the proposed model and the existing models was compared during experimental investigation.The proposed method with the modified ResNet50 model applied for classification had an accuracy 96.31 %, which was the highest result between all investigated models.This accuracy level of the method together with learning results confirms possibility of its practical implementation.
Software component model was created to provide an interface for integration of the proposed method into a medical diagnosis system.This interface provides means to manage classification model and to use it for decision making.
The scientific novelty of the obtained results is in the proposed method of skin disease diagnosis.
The practical significance of the obtained results is in classification accuracy of the proposed method and in software, which was developed to support the proposed method, as well as software component model, which allows to integrate the developed software into medical diagnosis system in the future.
Prospects for the further research are to integrate the proposed method and software, which was created based on appropriate software component, for skin disease diagnosis into medical diagnosis system.

Figure 1 -
Figure 1 -Structure of the modified model

Figure 2 -
Figure 2 -Structure of identity block of ResNet50 model The structure of convolutional block of ResNet50 model is shown in Fig. 3.At the fourth stage the created model is trained.Transfer learning [18] and fine-tuning [19] should be used for training.Transfer learning allows to reuse pre-trained model for new problem.Fine-tuning allows to fix weights of first layers, changing only last layers (aiming at more specific forms and objects on an image) of the model during training.At the next stages the trained model is applied for new cases when skin diseases should be diagnosed.Each new case should be investigated through the following stages:

Figure 3 -
Figure 3 -Structure of convolutional block of ResNet50 model

4 Figure 4 -Figure 5 -
Figure 4 -Graph of localization of observations in the dataset

Figure 6 -Figure 7 -
Figure 6 -Graph of accuracy change during model training for training and validation datasets

Figure 8 -
Figure 8 -Model training results classified images for the proposed model is presented in Fig.9.The results are presented for each diagnosis separately.

Figure 9 -
Figure 9 -Chart displaying fraction of true classified images for every diagnosisAppropriate fraction of misclassified images for the proposed model is presented in Fig.10.

Table 1 -
Number of images in the dataset used for investigation after oversampling

Table 2 -
Confusion matrix for skin disease recognition using the proposed method

Table 3 -
Results of classification by different CNN models