LR hyperparameters tuning method

[Cadene]

Hi,

Thanks again for your code. Unfortunately, I ran into a little issue. I can't reproduce some of your results because I am obliged to reduce my batch size (from 512 (yours) to 75). Thus I need to change the hyperparameters related to the learning rate.

Finding the right learning rate can be easily done by a little grid search. However, I would like to know how did you tune the hyperparameters related to the scheduler.
Especially:

• __C.training_parameters.wu_factor = 0.2
• __C.training_parameters.wu_iters = 1000
• __C.training_parameters.lr_steps = [5000, 7000, 9000, 11000]
• __C.training_parameters.lr_ratio = 0.1

Sharing your method would be awesome :)

Thanks for your help!
Remi

[YuJiang01]

we roughly adjusted based on the number of epochs when changing batch size for wu_iters and lr_steps