Perplexity

#nlp #deep-learning #interview #evaluation

Perplexity is an Intrinsic Evaluation
Using Extrinsic Evaluation or downstream tasks is slow and hard
So people use Perplexity to compare different models and with other research
Perplexity tells us how good the model is generating sentences
Less Perplexity is better

Perplexity from Likelihood

$\begin{aligned} P P L & = P (w_{1} w_{2} . . . w_{N})^{- \frac{1}{N}} \\ = \sqrt[N]{\frac{1}{P (w_{1} w_{2} . . . w_{N})}} \\ = \sqrt[N]{\frac{1}{\prod_{i} P (w_{i} | w_{1} w_{2} . . . w_{i - 1})}} \end{aligned}$

Perplexity from Cross Entropy

$P P L = e^{C E}$

Related Notes

ROUGE-N Score
Contrastive Learning
Root Mean Squared Logarithmic Error (RMSLE)
Grid Search Hyperparameter Finding
CNN