ELU
A classical activation function that is linear on the positive interval and exponential on the negative interval with a lower bound of -α; its smoothness on the negative interval helps mitigate vanishing gradients.
Timeline (last 90 days)
PreCap is an activation function whose core building block, CapExp, is an exponential unit that switches to a linear extension once the input exceeds a threshold, equivalent to a shifted and scaled ELU.
Plain ELU performs comparably to SPreCap overall, suggesting that the gain from CapExp may not come entirely from its special design but is related to the overall characteristics of the ELU family.
All Facts (2)
PreCap is an activation function whose core building block, CapExp, is an exponential unit that switches to a linear extension once the input exceeds a threshold, equivalent to a shifted and scaled ELU.
50%UnverifiedPlain ELU performs comparably to SPreCap overall, suggesting that the gain from CapExp may not come entirely from its special design but is related to the overall characteristics of the ELU family.
50%