TF-IDF

• TF-IDF is used to find the importance of a word in multiple documents
• TF = Term Frequency
  • number of times the word is in a document
• IDF = Inverse Document Frequency
  • how relevant that term is across all documents
• TF-IDF is the product of TF and IDF

Pros

1. Simple
2. Efficient
3. Effective in document retrieval

Limitations:

1. All limitations of Count based Word Embeddings
2. Bias towards rare tokens
3. Sparse representation (high dimensional and Density Sparse Data)

• TF-IDF can be used as Word Embeddings also, by replacing $1$ in One Hot Vector by the TF-IDF score.