Lamb optimizer warning in pytorch 1.6

[Laksh1997]

Hi I'm getting this deprecated warning in pytorch 1.6 for Lamb:

  | 2020-06-25T01:58:41.682+01:00 | add_(Tensor other, *, Number alpha) (Triggered internally at /opt/conda/conda-bld/pytorch_1592982553767/work/torch/csrc/utils/python_arg_parser.cpp:766.)
-- | -- | --
  | 2020-06-25T01:58:41.682+01:00 | exp_avg.mul_(beta1).add_(1 - beta1, grad)
  | 2020-06-25T01:58:41.682+01:00 | 2020-06-25T00:58:41 - WARNING - /opt/conda/envs/py36/lib/python3.6/site-packages/torch_optimizer/lamb.py:120: UserWarning: This overload of add_ is deprecated:
  | 2020-06-25T01:58:41.682+01:00 | add_(Number alpha, Tensor other)
  | 2020-06-25T01:58:41.682+01:00 | Consider using one of the following signatures instead:
  | 2020-06-25T01:58:41.682+01:00 | add_(Tensor other, *, Number alpha) (Triggered internally at /opt/conda/conda-bld/pytorch_1592982553767/work/torch/csrc/utils/python_arg_parser.cpp:766.)
  | 2020-06-25T01:58:41.682+01:00 | exp_avg.mul_(beta1).add_(1 - beta1, grad)

[ghost]

Hi. @Laksh1997

I would like to know if the LAMB optimizer works properly with Pytorch 1.6.

Many thanks for your help in advance !!!

[jettify]

Just executed test suite with pytorch 1.6, had to fix lookahead optimizer but all other tests are green.

[jettify]

As for warning it is easy to fix and should not have any influence over results. Gonna fix it soon also happy to merge PRs with fixes.

[ghost]

@jettify Hi~
Many thanks for your feedback!
And thanks for your amazing job!

[jettify]

Fixed in #163 going to do PyPI release relatively soon.

[jettify]

New release on PyPI https://pypi.org/project/torch-optimizer/0.0.1a15/