I tried my best to make an optimization function, but it didn't work.
reference
https://qiita.com/tokkuman/items/1944c00415d129ca0ee9
Execution source
https://github.com/akidukin/optimize_test/blob/master/benchmark_optimizer.py
Target
--Self-satisfaction
What did you do ??
Based on the URL that I referred to
- SGD
- MomentumSGD
- AdaGrad
- RMSprop Try to write in python
In this article (https://qiita.com/Akidukin14/items/4cf3fd1a065726450e4d) --sphere function --losenbrock function --styblinskitang function I wanted to see how it works by applying it to
Since I solved each partial differential formula myself, it may be bad.
result
Summary
Momentum SGD is especially terrible, where are all? I'm tired so I thought it was okay