INCREASING THE STABILITY OF LINEAR REGRESSION

Abstract

The paper considers a method for increasing the stability of a linear regression equation, the coefficients of which are the solution of a system of normal equations. The instability of the equation is determined by the presence of a linear relationship between the experimental data; the quantitative characteristics of the dependence are determined from the correlation matrix, which can also serve as a matrix of the system of equations. To reduce the correlation coefficients, Ridge (ridge) regression is traditionally used, which involves an increase in the diagonal members of the matrix by the same positive number. As a result, the matrix condition number decreases and the regression equation becomes more stable: a small change in the input results in a small change in the solution. The number by which the diagonal terms of the matrix increase is called the penalty imposed in ridge regression on all regression coefficients. In the proposed method, penalties, and different ones, are imposed only on those coefficients that correspond to data with high correlation. This leads to an increase in the stability of the equation due to a decrease in the values of the coefficients corresponding to correlated data. The choice of elements to be increased is based on the analysis of the correlation matrix of the original data set by decomposing it into diagonal matrices using the square root method. In addition to increasing the stability using the proposed method, a reduction in the dimension of the regression model can be achieved - a decrease in the number of terms of the corresponding equation, for which the LASSO and LARS algorithms are usually used. The effectiveness of the method is tested on a known data set, and a comparison is made not only with Ridge regression, but also with the results of known dimensionality reduction algorithms.