PRINCIPLE OF LONG-AND-CLOSE-RANGE ACTION IN STRUCTURIZATION PROBLEMS AND TRAINING OF ARTIFICIAL NEURAL NETWORKS

Abstract

In this work, a comparative analysis of three clustering algorithms is performed according to the criteria of accuracy and speed. The well-known k-means algorithm was chosen as the basic algorithm for comparison, which, according to certain criteria, is compared with the algorithm of artificial neural networks, which are built according to the principle of long-range-near-range (LRNR-networks) and a hybrid algorithm that includes operations of LRNR -networks, as well as operations of k- means.

The main idea of ​​ LRNR networks is to replace hyperparameters with network parameters. Namely, instead of training a large number of elements of transition matrices of connections between neighboring layers of the network, a much smaller number of neurons, or more precisely, the activation functions of neurons, are trained. Such activation functions must have at least one learnable parameter. In this work, it is proposed to use a parametric sigmoid with only one free parameter - the nonlinearity coefficient - as such functions.

The hybrid algorithm involves the use of a LRNR network for the nonlinear projection of primary samples from a large-dimensional factor space into a factor space with a smaller number of coordinates. Next, in a new, compressed space, the problem of clustering is solved using the well-known k-means algorithm.

Algorithms were compared on the example of 50,000 samples from the well-known MNIST set of handwritten digits. The number of clusters in all cases was fixed: 1000 clusters.

As a result of testing the algorithms with different values of hyperparameters, it was established that the LRNR -network algorithms always exceed other algorithms according to the speed criterion and, in the best cases, give very acceptable results according to the accuracy criteria. However, the best results were shown by the hybrid algorithm, which is not far behind LRNR networks in terms of speed, but outperforms all other algorithms in terms of accuracy. Thus, in the best case, the result of the validation of this algorithm gives 98.8% correct solutions of the clustering problem.