| |
| |
| ``mx.opt.adadelta`` |
| ====================================== |
| |
| Description |
| ---------------------- |
| |
| Create an AdaDelta optimizer with respective parameters. |
| |
| AdaDelta optimizer as described in Zeiler, M. D. (2012). |
| *ADADELTA: An adaptive learning rate method.* |
| http://arxiv.org/abs/1212.5701 |
| |
| Usage |
| ---------- |
| |
| .. code:: r |
| |
| mx.opt.adadelta( |
| |
| rho = 0.9, |
| |
| epsilon = 1e-05, |
| |
| wd = 0, |
| |
| rescale.grad = 1, |
| |
| clip_gradient = -1 |
| |
| ) |
| |
| Arguments |
| ------------------ |
| |
| +----------------------------------------+------------------------------------------------------------+ |
| | Argument | Description | |
| +========================================+============================================================+ |
| | ``rho`` | float, default=0.90. | |
| | | | |
| | | Decay rate for both squared gradients and delta x. | |
| +----------------------------------------+------------------------------------------------------------+ |
| | ``epsilon`` | float, default=1e-5. | |
| | | | |
| | | The constant as described in the thesis. | |
| +----------------------------------------+------------------------------------------------------------+ |
| | ``wd`` | float, default=0.0. | |
| | | | |
| | | L2 regularization coefficient add to all the weights. | |
| +----------------------------------------+------------------------------------------------------------+ |
| | ``rescale.grad`` | float, default=1. | |
| | | | |
| | | rescaling factor of gradient. | |
| +----------------------------------------+------------------------------------------------------------+ |
| | ``clip_gradient`` | float, default=-1 (no clipping if < 0). | |
| | | | |
| | | clip gradient in range [-clip_gradient, clip_gradient]. | |
| +----------------------------------------+------------------------------------------------------------+ |
| |
| |
| |
