PyTorch implementation of Neural Optimizer Search's Optimizer_1
Import Optimizer_1 like any torch.optim Optimizer:
from optimizer_1 import Optimizer_1
optimizer = Optimizer_1(model.parameters(), lr=1e-3, momentum=0.99)
loss.backward()
optimizer.step()
Optimizer_1 Test set: Average loss: 0.0525, Accuracy: 9820/10000 (98%)
SGD with Momentum Test set: Average loss: 0.0542, Accuracy: 9825/10000 (98%)
Optimizer_1 Test set: Average loss: 111.2389
Adam Test set: Average loss: 105.5995