103.json

{
  "id": "103",
  "title": "Implement the SELU Activation Function",
  "difficulty": "easy",
  "category": "Deep Learning",
  "video": "",
  "likes": "0",
  "dislikes": "0",
  "contributor": [
    {
      "profile_link": "https://github.com/Haleshot",
      "name": "Haleshot"
    }
  ],
  "description": "Implement the SELU (Scaled Exponential Linear Unit) activation function, a self-normalizing variant of ELU. Your task is to compute the SELU value for a given input while ensuring numerical stability.",
  "learn_section": "## Understanding the SELU Activation Function\n\nThe SELU (Scaled Exponential Linear Unit) activation function is a self-normalizing variant of the ELU activation function, introduced in 2017. It's particularly useful in deep neural networks as it automatically ensures normalized outputs with zero mean and unit variance.\n\n### Mathematical Definition\n\nThe SELU function is defined as:\n\n$$\nSELU(x) = \\lambda \\begin{cases} \nx & \\text{if } x > 0 \\\\\n\\alpha(e^x - 1) & \\text{if } x \\leq 0\n\\end{cases}\n$$\n\nWhere:\n- $\\lambda \\approx 1.0507$ is the scale parameter\n- $\\alpha \\approx 1.6733$ is the alpha parameter\n\n### Characteristics\n\n- **Output Range:** The function maps inputs to $(-\\lambda\\alpha, \\infty)$\n- **Self-Normalizing:** Automatically maintains mean close to 0 and variance close to 1\n- **Continuous:** The function is continuous and differentiable everywhere\n- **Non-Linear:** Provides non-linearity while preserving gradients for negative values\n- **Parameters:** Uses carefully chosen values for $\\lambda$ and $\\alpha$ to ensure self-normalization\n\n### Advantages\n\n1. **Self-Normalization:** Eliminates the need for batch normalization in many cases\n2. **Robust Learning:** Helps prevent vanishing and exploding gradients\n3. **Better Performance:** Often leads to faster training in deep neural networks\n4. **Internal Normalization:** Maintains normalized activations throughout the network\n\n### Use Cases\n\nSELU is particularly effective in:\n- Deep neural networks where maintaining normalized activations is crucial\n- Networks that require self-normalizing properties\n- Scenarios where batch normalization might be problematic or expensive",
  "starter_code": "def selu(x: float) -> float:\n\t\"\"\"\n\tImplements the SELU (Scaled Exponential Linear Unit) activation function.\n\n\tArgs:\n\t\tx: Input value\n\n\tReturns:\n\t\tSELU activation value\n\t\"\"\"\n\talpha = 1.6732632423543772\n\tscale = 1.0507009873554804\n\t# Your code here\n\tpass",
  "solution": "import math\n\ndef selu(x: float) -> float:\n    \"\"\"\n    Implements the SELU (Scaled Exponential Linear Unit) activation function.\n\n    Args:\n        x: Input value\n\n    Returns:\n        SELU activation value\n    \"\"\"\n    alpha = 1.6732632423543772\n    scale = 1.0507009873554804\n    return round(scale * x if x > 0 else scale * alpha * (math.exp(x) - 1), 4)",
  "example": {
    "input": "selu(-1.0)",
    "output": "-1.1113",
    "reasoning": "For x = -1.0, the SELU activation is calculated using the formula $SELU(x) = \\lambda \\alpha (e^x - 1)$. Substituting the values of $\\lambda$ and $\\alpha$, we get $SELU(-1.0) = 1.0507 \\times 1.6733 \\times (e^{-1.0} - 1) = -1.1113$."
  },
  "test_cases": [
    {
      "test": "print(round(selu(1.0), 4))",
      "expected_output": "1.0507"
    },
    {
      "test": "print(round(selu(0.0), 4))",
      "expected_output": "0.0"
    },
    {
      "test": "print(round(selu(-1.0), 4))",
      "expected_output": "-1.1113"
    },
    {
      "test": "print(round(selu(5.0), 4))",
      "expected_output": "5.2535"
    },
    {
      "test": "print(round(selu(-5.0), 4))",
      "expected_output": "-1.7463"
    }
  ]
}