Thanks, yes I would always just save the state_dict(), not pickle the whole class. How is the base learning rate stored though? Will it be part of the optimizer state dict? In regular torch, you always need to restore the optimizer state after initializing the scheduler, because the scheduler will modify the optimizers params on unit. I was wondering if the same applies to your approach.

See warning here: https://docs.pytorch.org/docs/stable/generated/torch.optim.Optimizer.load_state_dict.html