On the Approximation of the Cut Function by Smooth Sigmoid Functions

Abstract

We discuss some computational, modelling and approximation issues related to several classes of sigmoid functions.В Sigmoid functions find numerous applications in various fields related to life sciences, chemistry, physics, artificial intelligence, etc. In fields such as signal processes, pattern recognition, machine learning, neural networks, sigmoid functions are also known as ``activation functions''. A practically important class of sigmoid functions is the class of cut functions including the Heaviside step function as a limiting case. Cut functions are continuous but they are not smooth (differentiable) at the two endpoints of the interval where they increase; however cut functions are Hausdorff continuous (H-continuous). In some applications smooth sigmoid functions are preferred.В  Two familiar classes of smooth sigmoid functions are the logistic and squashing functions. There are situations when one needs to pass from a cut function to smooth sigmoid functions, and vice versa. Such a necessity rises in a natural way the issue of approximating nonsmooth sigmoid functions by smooth sigmoid functions.\medskipВ  We prove that the uniform distance between a cut function and theВ В  logistic function of best uniform approximation is an absolute constant. By contrast,В В  the Hausdorff distance (H-distance) depends on the slope k and tends to zero with ....