ALGORITHM FOR OPTIMIZATION OF LCR-NETWORK ACTIVATION FUNCTIONS PARAMERERS

Abstract

This paper continues research aimed at developing efficient algorithms for solving clustering and classification problems. The previously considered hybrid algorithms involve processing input data using artificial neural networks based on the Long-and-Close-Range (LCR) principle. According to this principle, the transition matrices between network layers are defined as nearly static, while the primary network optimization effort focuses on training individual neurons.

The paper proposes and substantiates an algorithm for optimizing activation functions in neurons of LCR-network layers. As nonlinear activation functions, we introduce a parametric modification of the well-known ReLU function, named Relu2. This function includes a single scalar parameter, the cutoff coefficient. It is demonstrated that applying Relu2 functions enhances the discriminative capability (resolution) of artificial neural networks.

To evaluate the accuracy and computational efficiency of the proposed method, we selected the classical K-nearest neighbors (KNN) algorithm as a baseline. Specifically, we adopted its simplest and fastest variant—one nearest neighbor (1-NN).

In this study, the LCR-network implementation utilizes a two-layer architecture consisting of an input and an output layer. The activation functions of the output layer are optimized to maximize the average distance between transformed samples from different classes. These transformed samples are subsequently divided equally among subclasses (clusters) across all classes. Classification of the test set is performed by minimizing distances to cluster centroids, thus employing the K-means clustering method.

A comparative analysis demonstrates that the hybrid algorithm significantly outperforms the nearest neighbor algorithm in terms of computational speed while achieving comparable accuracy.