Learning rate optimization in convolutional neural networks for medical images classification

This research proposes a novel learning rate optimization algorithm for Adaptive Moment Estimation (Adam), which is a common optimizer in convolutional neural networks (CNNs). Optimizers are used to control the training efficiency and prediction accuracy by controlling the convergence progress. However, optimizers are merely hyperparameter-free and very sensitive to the hyperparameters. For example, the learning rate is one of the hyperparameters which represents the step size in the calculation process and has the most significant influence on prediction accuracy. Thus, it is necessary to optimize the learning rate to pursue good results. In this research, a Gaussian Process Regression (GPR)-based learning rate optimization algorithm is proposed to increase the classification accuracy. To be specific, the relationship between the learning rate and corresponding accuracy is studied and the potential learning rate is obtained from the GPR model which developed with the learning rates and accuracies of previous iterations. To evaluate the proposed algorithm, one of the state-of-art CNNs called AlexNet is applied as the CNNs framework. Recently, with the development of healthcare, medical image classification becomes a common domain of applying of the CNNs technique. Thus, the Stimulated Raman scattering (SRS) images on human brain tumors are used to classify the cells and non-cells. The proposed GPR-based learning rate optimization algorithm will be compared to the constant learning rate algorithm on SRS image classification in terms of accuracy, sensitivity, specificity, and precision. The experimental results illustrate that the proposed GPR-based algorithm has a better performance by showing a 95% classification accuracy.