Loss Functions

Additional loss functions

source

BCEWithLogitsLoss

 BCEWithLogitsLoss (weight:Tensor|None=None, reduction:str='mean',
                    pos_weight:Tensor|None=None, thresh:float=0.5)

Like nn.BCEWithLogitsLoss, but with ‘batchmean’ reduction from MosiacML. batchmean scales loss by the batch size which results in larger loss values more similar to nn.CrossEntropy then mean reduction.

Type Default Details
weight Tensor | None None Rescaling weight for each class
reduction str mean Pytorch reduction to apply to loss output. Also supports ‘batchmean’.
pos_weight Tensor | None None Weight of positive examples
thresh float 0.5 Threshold for decodes

source

ClassBalancedCrossEntropyLoss

 ClassBalancedCrossEntropyLoss (samples_per_class:Tensor|Listy[int],
                                beta:float=0.99, ignore_index:int=-100,
                                reduction:str='mean',
                                label_smoothing:float=0.0, axis:int=-1)

Class Balanced Cross Entropy Loss, from https://arxiv.org/abs/1901.05555.

Type Default Details
samples_per_class Tensor | Listy[int] Number of samples per class
beta float 0.99 Rebalance factor, usually between [0.9, 0.9999]
ignore_index int -100 Target value which is ignored and doesn’t contribute to gradient
reduction str mean Pytorch reduction to apply to loss output
label_smoothing float 0.0 Convert hard targets to soft targets, defaults to no smoothing
axis int -1 ArgMax axis for fastai `decodes``

source

ClassBalancedBCEWithLogitsLoss

 ClassBalancedBCEWithLogitsLoss (samples_per_class:Tensor|Listy[int],
                                 beta:float=0.99, reduction:str='mean',
                                 pos_weight:Tensor|None=None,
                                 thresh:float=0.5)

Class Balanced BCE With Logits Loss, from https://arxiv.org/abs/1901.05555 with ‘batchmean’ reduction

Type Default Details
samples_per_class Tensor | Listy[int] Number of samples per class
beta float 0.99 Rebalance factor, usually between [0.9, 0.9999]
reduction str mean Pytorch reduction to apply to loss output. Also supports ‘batchmean’.
pos_weight Tensor | None None BCE Weight of positive examples
thresh float 0.5 Threshold for fastai decodes