Entropy

#machine-learning #interview #loss-in-ml

\begin{aligned} E (s u r p r i s e) & = \sum l o g_{2} \frac{1}{P (x)} P (x) \\ = \sum P (x) (l o g_{2} 1 - l o g_{2} P (x)) \\ = \sum P (x) (0 - l o g_{2} P (x)) \\ = \sum - P (x) l o g_{2} P (x) \\ = E n t r o p y \end{aligned}

Entropy Formula

$E n t r o p y = \sum - P (x) l o g_{2} P (x)$

How entropy can be used?

Entropy is used to quantify similarity or differentiate in the number of classes

Related Notes