Skip to content

Added Scaled Exponential Linear Unit Activation Function #9027

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Sep 6, 2023
Merged
Prev Previous commit
Next Next commit
Update scaled_exponential_linear_unit.py
  • Loading branch information
AdarshAcharya5 authored Sep 2, 2023
commit b10622916bd88adcac517701b98a0124d72cb9fc
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
"""
Implements the Scaled Exponential Linear Unit or SELU function.

The function takes a vector of K real numbers and two real numbers alpha(default = 1.6732) and lambda (default = 1.0507)
as input and then applies the SELU function to each element of the vector. SELU is a self-normalizing activation function.
It is a variant of the ELU. The main advantage of SELU is that we can be sure that the output will always be standardized
due to its self-normalizing behavior. That means there is no need to include Batch-Normalization layers.

The function takes a vector of K real numbers and two real numbers
alpha(default = 1.6732) & lambda (default = 1.0507) as input and
then applies the SELU function to each element of the vector.
SELU is a self-normalizing activation function. It is a variant
of the ELU. The main advantage of SELU is that we can be sure
that the output will always be standardized due to its
self-normalizing behavior. That means there is no need to
include Batch-Normalization layers.
References :
https://iq.opengenus.org/scaled-exponential-linear-unit/
"""
Expand Down