Launch a mock training to find a good learning rate and return suggestions based on
suggest_funcs as a named tuple. Use
restore_state to reset dataloaders and random state after running.
lr_find advances both the random state and dataloaders and behaves the same way as fast.ai's
lr_find. Which means the following two code blocks:
with no_random(): dls = get_dls() learn = Learner(dls, xresnet18(n_out=dls.c)) with no_random(): learn.lr_find(restore_state=False) learn.fit_one_cycle(2, 3e-3)
with no_random(): dls = get_dls() learn = Learner(dls, xresnet18(n_out=dls.c)) with no_random(): learn.fit_one_cycle(2, 3e-3)
will result with different training output.
While the default of
restore_state=True prevents this from occuring, it has the potential downside of showing less variance in learning rate results. As every call to
lr_find will be over the same first
n_iter items using the same random state. Without
no_random set, most of the variation appears to be from cuda not being set in deterministic mode.