Like `fit_flat_cos`, but allows changing to multiple new learning rates immediately or via any fastai schedule.


Learner.fit_flat_varied(n_epoch, start_lr=None, div_final=100000.0, pct_start=0.75, wd=None, next_lr=None, change_by=None, change_time=1, change_sched=None, cbs=None, reset_opt=False)

Fit self.model for n_epoch at flat start_lr, then change to flat next_lr at change_by, optionally with cosine annealing or custom change_sched over change_time. Final cosine annealing at pct_start.

n_epoch, start_lr, div_final, pct_start, wd, cbs, & reset_opt are all same as fit_flat_cos from

next_lr single or list of learning rates to switch to at change_by. Must be same length as change_by.

change_by single or list of epochs or percent of steps to switch to next_lr by. Must be same length as next_lr.

change_time if greater than 0 (percent of steps or epochs), how long to cosine anneal to next_lr. Can be single or list of same length as next_lr.

change_sched optional single or list of schedules. If None defaults to SchedCos. Must be same length as next_lr. SchedPoly must be passed as partial: partial(SchedPoly, power=0.5).

Example Schedules

Discriminative Linear Warmup:

learn.fit_flat_varied(4, slice(3e-5, 3e-3), next_lr=3e-3, change_by=1, change_time=1, change_sched=SchedLin)

discriminative linear warmup

Multiple Cosine Annealing:

learn.fit_flat_varied(15, 8e-3, next_lr=[6e-3, 4e-3], change_by=[4, 8], change_time=2)

multiple cosine annealing

Immediate Change:

learn.fit_flat_varied(10, 8e-3, next_lr=[6e-3, 4e-3], change_by=[0.25, 0.5], change_time=0)

immediate change