

``mx.nd.LeakyReLU``
======================================

Description
----------------------

Applies Leaky rectified linear unit activation element-wise to the input.

Leaky ReLUs attempt to fix the "dying ReLU" problem by allowing a small `slope`
when the input is negative and has a slope of one when input is positive.

The following modified ReLU Activation functions are supported:

	- *elu*: Exponential Linear Unit. `y = x > 0 ? x : slope * (exp(x)-1)`
	- *selu*: Scaled Exponential Linear Unit. `y = lambda * (x > 0 ? x : alpha * (exp(x) - 1))` where
  *lambda = 1.0507009873554804934193349852946* and *alpha = 1.6732632423543772848170429916717*.
- *leaky*: Leaky ReLU. `y = x > 0 ? x : slope * x`
- *prelu*: Parametric ReLU. This is same as *leaky* except that `slope` is learnt during training.
- *rrelu*: Randomized ReLU. same as *leaky* but the `slope` is uniformly and randomly chosen from
  *[lower_bound, upper_bound)* for training, while fixed to be
  *(lower_bound+upper_bound)/2* for inference.


Arguments
------------------

+----------------------------------------+------------------------------------------------------------+
| Argument                               | Description                                                |
+========================================+============================================================+
| ``data``                               | NDArray-or-Symbol.                                         |
|                                        |                                                            |
|                                        | Input data to activation function.                         |
+----------------------------------------+------------------------------------------------------------+
| ``gamma``                              | NDArray-or-Symbol.                                         |
|                                        |                                                            |
|                                        | Input data to activation function.                         |
+----------------------------------------+------------------------------------------------------------+
| ``act.type``                           | {'elu', 'gelu', 'leaky', 'prelu', 'rrelu',                 |
|                                        | 'selu'},optional,                                          |
|                                        | default='leaky'.                                           |
|                                        |                                                            |
|                                        | Activation function to be applied.                         |
+----------------------------------------+------------------------------------------------------------+
| ``slope``                              | float, optional, default=0.25.                             |
|                                        |                                                            |
|                                        | Init slope for the activation. (For leaky and elu only)    |
+----------------------------------------+------------------------------------------------------------+
| ``lower.bound``                        | float, optional, default=0.125.                            |
|                                        |                                                            |
|                                        | Lower bound of random slope. (For rrelu only)              |
+----------------------------------------+------------------------------------------------------------+
| ``upper.bound``                        | float, optional, default=0.333999991.                      |
|                                        |                                                            |
|                                        | Upper bound of random slope. (For rrelu only)              |
+----------------------------------------+------------------------------------------------------------+

Value
----------

``out`` The result mx.ndarray


Link to Source Code: http://github.com/apache/incubator-mxnet/blob/1.6.0/src/operator/leaky_relu.cc#L161