Activation Function
A nonlinear operation applied on top of a linear transformation in neural networks, converting linear relationships into nonlinear ones to significantly enhance the network's expressive capacity. Common activation functions include Sigmoid, Tanh, and ReLU, with the field having evolved from early S-shaped functions to the ReLU family.
Timeline (last 90 days)
A neural network without an activation function can only represent linear functions no matter how deep it is.
An artificial neuron receives multiple input signals, multiplies each input by a weight, adds a bias, and then passes the result through an activation function to determine whether to fire an output.
All Facts (2)
A neural network without an activation function can only represent linear functions no matter how deep it is.
50%UnverifiedAn artificial neuron receives multiple input signals, multiplies each input by a weight, adds a bias, and then passes the result through an activation function to determine whether to fire an output.
50%