Logistic Regression
- There is no closed form solution like Linear Regression
- The equation is same as Linear Regression
- For N-dimensional data, the Decision Boundary will be (N-1) dimensional hyperplane
- Logistic Regression is known as the unit step for Neural Network as in Neural Network each node is the Logistic Regression
Logistic Regression Formula
- Here
is the Sigmoid Function - if y >= Decision Boundary (usually 0.5) then positive
- else negative
Advantages of Logistic Regression
- Less training time
- Good accuracy if data is linearly seperable
- Less inclined to Overfitting, for high dimensional data there can be overfitting, which can be solved by L1 or Lasso Regression, L2 or Ridge Regression or Elastic Net Regression
- Easier to implement, interpret and efficient to train
Disadvantages of Logistic Regression
- Lot of feature engineering is needed
- Can't solve Non-linear dataset
- Real life there is very less to no linearly separable dataset
- Bad at Handling Outliers
- Bad at Handling Missing Data
- If the number of observation is less than number of features (AKA Curse of Dimensionality), then it can't be used