Evaluation of Machine Learning Hyperparameters Performance for Mice Protein Expression Data in Different Situations

Abstract

In this study, the aim was to assess the effect and significance of hyperparameters in four different datasets containing different values for observation numbers and variable counts with the machine-learning methods of support vector machines and artificial neural networks. With this aim, a dataset comprising 15 repeats of 77 protein levels from 38 healthy and 34 down syndrome mice was used. A total of 138 different models and model classification performance criteria were obtained from the datasets in the study comprising combinations of hyperparameters in machine-learning methods. Comparison of the models used criteria like accurate classification percentage, kappa statistic, mean absolute error and square root of mean error squares. According to performance criteria, the first dataset with 1080 observations x 77 variables had 71.30% accurate classification percentage for assumed parameters with the support vector machines polynomial kernel function, while changing the hyperparameter variables increased this rate to 99.44%. Similarly, the second dataset had 50.65% accurate classification percentage with the artificial neural network single hidden layer 2 neuron model, while changing the hyperparameter values increased this rate to 90.46%. In conclusion, in situations with low variable and observation numbers, the machine learning methods were determined to display lower performance. However, in datasets, it is very important for classification performance in artificial neural networks and support vector machines, especially polynomial and radial basis function kernel functions, to set hyperparameters according to the dataset. In situations with low variable numbers, especially, the effect of hyperparameters was determined to gain importance.