Huber Loss

#loss-in-ml #machine-learning #deep-learning #interview

Hybrid of Mean Squared Error (MSE) and Mean Absolute Error (MAE)
- Squared error for small differences
- Absolute error for large differences
It has a hyperparameter $δ$

Huber Loss Formula

$L = {\begin{cases} \frac{1}{2} (y - \hat{y})^{2}, & if | y - \hat{y} | \leq δ \\ δ | y - \hat{y} | - \frac{1}{2} δ^{2}, & otherwise \end{cases}$

Pros

Differentiable at 0
good at Handling Outliers
The hyperparameter $δ$ can be tuned to maximize model accuracy

Cons

Additional conditions make it computationally expensive
Differentiable only once

Related Notes