🧠of Dipta
Search
CTRL + K
🧠of Dipta
Search
CTRL + K
Home
Recent Notes
Literature Notes
Advanced NLP with Scipy
Deep Learning by Ian Goodfellow
DS & Algo Interview
How To 100M Learning Text Video
How to Read a Paper
How To Write a Paper
ML Interview
Papers
$Ï„$-bench - A Benchmark for Tool-Agent-User Interaction in Real-World Domains
COIN
Compressed Chain of Thought - Efficient Reasoning Through Dense Representations
DeepSeek-R1
Deliberative Alignment - Reasoning Enables Safer Language Models
G-Eval - NLG Evaluation using GPT-4 with Better Human Alignment
Investigating Continual Pretraining in Large Language Models - Insights and Implications
Is a Question Decomposition Unit All We Need
Large Language Models are Zero-Shot Rankers for Recommender Systems
MM-LLMs
Molmo and PixMo
MultiVENT
OpenPI-C
Piecing It All Together - Verifying Multi-Hop Multimodal Claims
PubMedQA - A Dataset for Biomedical Research Question Answering
Scientific Fact-Checking - A Survey of Resources and Approaches
Semantic Product Search for Matching Structured Product Catalogs in E-Commerce
Token Assorted - Mixing Latent and Text Tokens for Improved Language Model Reasoning
Vision Language Model-based Caption Evaluation Method Leveraging Visual Context Extraction
What is More Likely to Happen Next
Templates
Paper Template 1
Paper Template
Permanent Notes
Topic Template
Zotero Template
Topics
activation-function
algorithm
behavioral
deep-learning
evaluation
interview
llm
loss-in-ml
machine-learning
math
nlp
paper
probability
retrieval
sci-fact
statistics
vision
Zettelkasten
3 key question in data visualization
Accuracy
Activation Function
Active Learning
AdaBoost vs. Gradient Boosting vs. XGBoost
Adaboost
AdaDelta
AdaGrad
Adam
Adaptive Softmax
ADASYN
Additive Attention
Adjusted R-squared Value
Alternative Hypothesis
Amazon Leadership Principles
Ancestral Sampling
Area Under Precision Recall Curve (AUPRC)
Attention
AUC Score
Auto Regressive Model
Autoencoder for Denoising Images
Autoencoder
Averaging in Ensemble Learning
Back Propagation
Backward Feature Elimination
Bag of Words
Bagging
Balanced Accuracy
Basics of Kubernetes
Batch Normalization
Bayes Theorem
Bayesian Optimization Hyperparameter Finding
Beam Search
Behavioral Interview
BERT Embeddings
BERT
Best Match 25 (BM25)
Bias & Variance
Bidirectional RNN or LSTM
Binary Cross Entropy
Binning or Bucketing
Binomial Distribution
bisect_left vs. bisect_right
bitsandbytes
BLEU Score
Boosting
Box Plot
Byte Level BPE
Byte Pair Encoding (BPE)
Cache Augmented Generation (CAG)
Causal Language Modeling
Causality vs. Correlation
Central Limit Theorem
Chain Rule
Challenges of NLP (2022)
Character Tokenizer
Claim Verification Datasets
CNN
Co-occurrence based Word Embeddings
Co-Variance
Collinearity
Combination
Conditional Probability
Conditional Random Field
conditionally-independent-joint-distribution
Confusion Matrix
Connections - Log Likelihood, Cross Entropy, KL Divergence, Logistic Regression, and Neural Networks
Contextualized Word Embeddings
Continuous Bag of Words
Continuous Batching
Continuous Random Variable
Contrastive Learning
Contrastive Loss
Convex vs Nonconvex Function
Cosine Similarity
Count based Word Embeddings
Cross Entropy
Cross Validation
Cross-Attention
Crossed Feature
Curse of Dimensionality
DAPO
Data Augmentation
Data Imputation
Data Monitoring (DVC)
Data Visualization
DBScan Clustering
Debugging Deep Learning
Decision Boundary
Decision Tree (Classification)
Decision Tree (Regression)
Decision Tree
Decoder Only Transformer
Decoding Strategies
Density Sparse Data
Dependent Variable
Derivative
determinant
diagonal-matrix
Differentiation of Product
Differentiation
Digit Dp
Dimensionality Reduction
Discrete Random Variable
Discriminative vs. Generative Models
DistilBERT
do_sample (vllm vs hf)
doing-literature-review
Domain vs. Codomain vs. Range
Dropout
Dying ReLU
Dynamic Batching
Dynamic Programming (DP) in python
Early Stopping
Eigendecomposition
eigenvalue-eigenvector
Elastic Net Regression
ELMo Embeddings
Encoder Only Transformer
Encoder-Decoder Transformer
Ensemble Learning
Entropy and Information Gain
Entropy
Essential Visualizations
Estimated Mean
Estimated Standard Deviation
Estimated Variance
Euclidian Distance
Euclidian Norm
Exhaustive Search
Expected Value for Continuous Events
Expected Value for Discrete Events
Expected Value
Exploding Gradient
Exponential Distribution
Extrinsic Evaluation
F-Beta Score
F-Beta@K
F1 Score
Fake News Challenge
False Negative Error
False Positive Rate
FastText Embedding
Feature Engineering
Feature Extraction
Feature Hashing
Feature Preprocessing
Feature Selection
Finding Co-relation between two data or distribution
Fine Tuning Large Language Models
Floating Point Explained
Forward Feature Selection
Foundation Model
frobenius-norm
fully-independent-join-distribution
fully-joint-joint-distribution
Gaussian Distribution
GBM
Generalized Discriminant Analysis (GDA)
Genetic Algorithm Hyperparameter Finding
Gini Impurity
Global Minima
GloVe Embedding
GPT-OSS
GPU Computation for LLM
Gradient Boost (Classification)
Gradient Boost (Regression)
Gradient Boosting
Gradient Checkpointing
Gradient Clipping
Gradient Descent
Gradient
Graph Convolutional Network (GCN)
Greedy Decoding
Grid Search Hyperparameter Finding
Group Normalization
Group-Query Attention
GRPO
GRU
GSPO
Gumbel Softmax
Handling Imbalanced Dataset
Handling Missing Data
Handling Outliers
Heapq (nlargest or nsmalles)
Hidden Markov Model
Hierarchical Clustering
Hierarchical Softmax
Hinge Loss
Histogram
Homonym or Polysemy
How to Choose Kernel in SVM
How to combine in Ensemble Learning
How to create a Scientific Poster
How to do research
How to prepare for Behavioral Interview
How to Write Academic Paper (from CS Perspective)
Huber Loss
Hyperparameters
Hypothesis Testing
Hypothetical Document Embedding (HyDE)
identity-matrix
Implement Linear Regression using Numpy
Independent Component Analysis (ICA)
Independent Variable
InfoNCE Loss
Instruction Fine Tuning
Instructional Websites
Integration by Parts or Integration of Product
Internal Covariate Shift
Interquartile Range (IQR)
Interview Resources
Interview Scheduling
Intrinsic Evaluation
Jaccard Distance
Jaccard Similarity
Jacobian Matrix
Joint Probability
joint-distribuition
jupyter-notebook-on-server
K Fold Cross Validation
K-means Clustering
K-means vs. Hierarchical
K-nearest Neighbor (KNN)
Kernel in SVM
Kernel Regression
Kernel Trick
KL Divergence
KTO
KV Cache
L1 or Lasso Regression
L1 vs. L2 Regression
L2 or Ridge Regression
Label Encoding
Label Smoothing
Layer Normalization
Leaky ReLU
Learning Rate Scheduler
Learning Rate
Leave one out Cross Validation
Lemmatization
LightGBM
Likelihood
Line Equation
Linear Discriminant Analysis (LDA)
Linear Regression with Normal Equation
Linear Regression
LLM GPU Calculate
Local Attention
Local Minima
Log (Odds Ratio)
Log (Odds)
Log Normalization
Log Scale
Log-cosh Loss
logarithm
Logistic Regression vs. Neural Network
Logistic Regression
lp-norm
LSTM
Machine Learning Algorithm Selection
Machine Learning vs. Deep Learning
Majority vote in Ensemble Learning
Mamba Architecture
Manhattan Distance
Margin in SVM
Marginal Probability
Masked Language Modeling
Masked Self-Attention
Math Dataset
matplotlib functions
matplotlib legend
Matrices
Matrix Factorization
max-norm
Maximal Margin Classifier
Maximum Likelihood
MCP.md 2
Mean Absolute Error (MAE)
Mean Absolute Percentage Error (MAPE)
Mean Reciprocal Rank (MRR)
Mean Squared Error (MSE)
Mean Squared Logarithmic Error (MSLE)
Mean
Median
Merge K-sorted List
Merge Overlapping Intervals
Meteor Score
Min Max Normalization
Mini Batch SGD
Mixed Precision
Mixture of Experts
ML Case Study or ML Design
ML System Design
Mode
Model Based vs. Instance Based Learning
Multi Class Cross Entropy
Multi Label Cross Entropy
Multi Layer Perceptron
Multi-Head Attention
Multi-Head Latent Attention
Multi-Query Attention
Multicollinearity
Multiset
Multivariable Linear Regression
Multivariate Linear Regression
Multivariate Normal Distribution
Mutual Information
N-gram Method
Naive Bayes
Named Entity Recognition (NER)
Negative Log Likelihood
Negative Sampling
Nesterov Accelerated Gradient (NAG)
Neural Network Normalization
Neural Network
Next Sentence Prediction
norm
Normal Distribution
Normalization
Null Hypothesis
Odds Ratio
Odds
One Class Classification
One Class Gaussian
One Hot Vector
One vs One Multi Class Classification
One vs Rest or One vs All Multi Class Classification
Optimizers
Optimizing Transformer
orthogonal-matrix
orthonormal-vector
Overcomplete Autoencoder
Overfitting
Oversampling
p-value
Padding in CNN
Paged KV Cache
Papers Must Read
Parallelism in LLM
Parameter vs. Hyperparameter
PCA vs. Autoencoder
Pearson Correlation
Perceptron
Permutation
Perplexity
Personal Claude Code Recommendations
Plots Compared
Polynomial Kernel
Polynomial Regression
Pooling
Population
Positional Encoding in Transformer
Posterior Probability
Pre-Fill in LLM
Pre-Training LLM
Precision Recall Curve (PRC)
Precision
Precision@K
Prepare for Talk
Presentation Making Tips
Principal Component Analysis (PCA)
Prior Probability
Probability Density Function
Probability Distribution
Probability Mass Function
Probability vs. Likelihood
Problem Solving Algorithm Selection
Prompt Engineering
Proximal Policy Optimization (PPO)
Pruning in Decision Tree
PyTorch Loss Functions
PyTorch Refresher
QLoRA 2
QLoRA 3
QLORA
Quantization Technique
Questions to ask in a Interview?
Quintile or Percentile
Quotient Rule or Differentiation of Division
R-squared Value
Radial Basis Kernel
Random Forest
Random Variable
Recall
Recall@K
Recommender System (RecSys)
Regularization
Reinforcement Learning from Human Feedback (RLHF)
Reinforcement Learning
Relational GCN
ReLU
Reno Talk @UMBC on Scale-2024
Research Skills Unsorted List
Retrieval Metrics
RMSProp
RNN
Robust Scaling Normalization
ROC Curve
Root Mean Squared Error (RMSE)
Root Mean Squared Logarithmic Error (RMSLE)
Rotary Position Embedding (RoPE)
ROUGE-L Score
ROUGE-LSUM Score
ROUGE-N Score
RTE (Recognizing Textual Entailment)
Saddle Points
scalar
Second Order Derivative or Hessian Matrix
Self-Attention
Self-Supervised Learning
Semi-supervised Learning
Sensitivity
SentencePiece Tokenization
Sequence-to-Sequence Model
Sigmoid Function
Sigmoid Kernel
Simple Linear Regression
Singular Value Decomposition (SVD)
Skip Gram Model
Sliding KV Cache
Sliding Window Attention
SMOTE
Soft Margin in SVM
Softmax
Softplus
Softsign
Some Common Behavioral Questions
Sources of Uncertainty
spacy-doc-object
spacy-doc-span-token
spacy-explanation-of-labels
spacy-matcher
spacy-named-entities
spacy-operator-quantifier
spacy-pattern
spacy-pipeline
spacy-pos
spacy-semantic-similarity
spacy-syntactic-dependency
Sparse Mixture of Experts
Specificity
Splitting tree in Decision Tree
Stacking or Meta Model in Ensemble Learning
Standard deviation
Standardization
State Space Model
Statistical Power
Statistical Significance
Stemming
Stepwise Selection
Stochastic Gradient Descent (SGD)
Stochastic Gradient Descent with Momentum
Stop Words
Stratified K Fold Cross Validation
Stride in CNN
Stump
Sub-sampling in Word2Vec
Sub-word Tokenizer
Supervised Learning
Support Vector Machine (SVM)
Support Vector
Surprise
SVC
Swallow vs. Deep Learning
t-SNE
Tanh
Temperature in Decoding
Text Preprocessing
TF-IDF
Three Way Partioning
Time Complexity of ML Algos
Time Complexity of ML Models
Tokenizer
Top-K in Retrieval System
Toward RL Learning
trace-operator
Training a Deep Neural Network
Transformer vs LSTM
Transformer
Triplet Loss
True Negative Rate
True Positive Rate
Two Pointer
Type 1 Error vs. Type 2 Error
Undercomplete Autoencoder
Undersampling
Uniform Distribution
Unigram Tokenization
unit-vector
Unsupervised Learning
Vanishing Gradient in Transformers
Vanishing Gradient
Variance
Variational Autoencoder
Vector Database
vector
Vision Transformer
Weakly Supervised Learning
Weight Initialization
When less data is better than more?
Why do we scale attention weights?
Why do we use Projection in QKV?
Why Trigonometric Function for Positional Encoding?
Word Embeddings
Word Error Rate
Word Tokenizer
Word2Vec Embedding
WordPiece Tokenization
XGBoost
Yet another Rope Extension (YaRN)
Z-Score Normalization
Zotero Guides
Tanh
#deep-learning
#activation-function
#interview
Tanh = Hyperbolic Tangent Function
Range:
[
−
1
,
1
]
TanH Formula
f
(
z
)
=
e
z
−
e
−
z
e
z
+
e
−
z
Compared to
Sigmoid Function
, the gradient near 0 is notably larger, leading to more substantial update
But it also saturates, which might cause
Vanishing Gradient
TODO:
Add derivative
Related Notes
ROUGE-LSUM Score
Deep Learning by Ian Goodfellow
RNN
Normalization
Hinge Loss
