Logistic Regression

#machine-learning #interview

There is no closed form solution like Linear Regression
The equation is same as Linear Regression
For N-dimensional data, the Decision Boundary will be (N-1) dimensional hyperplane
Logistic Regression is known as the unit step for Neural Network as in Neural Network each node is the Logistic Regression

Logistic Regression Formula

$y = σ (a x + b)$

Here $σ$ is the Sigmoid Function
if y >= Decision Boundary (usually 0.5) then positive
else negative

Advantages of Logistic Regression

Less training time
Good accuracy if data is linearly seperable
Less inclined to Overfitting, for high dimensional data there can be overfitting, which can be solved by L1 or Lasso Regression, L2 or Ridge Regression or Elastic Net Regression
Easier to implement, interpret and efficient to train

Disadvantages of Logistic Regression

Lot of feature engineering is needed
Can't solve Non-linear dataset
Real life there is very less to no linearly separable dataset
Bad at Handling Outliers
Bad at Handling Missing Data
If the number of observation is less than number of features (AKA Curse of Dimensionality), then it can't be used

Related Notes