Sometimes more is not better, especially in the case of machine learning if we give a lot of unnecessary features, then there is a good chance then the model can overfit with the features. Also, it will take more time to converge as the model has to learn which are the important features and which are not.

Feature Selection algorithms:

Depending on Data:

1. Percent of missing value
   1. Remove the feature if most of them are missing
   2. Otherwise impute (refer to Handling Missing Data)
2. Drop variables with zero Variance (no important information)

Depending on Redundancy

1. Pairwise correlation (refer to Pearson Correlation)
2. Multicollinearity to remove multiple co-related features
3. Use Principal Component Analysis (PCA) to reduce feature using crossing among features
   1. good for introducing non-linearity
   2. bad for interpretability
4. Use Cluster analysis to find out which features are related (refer to Hierarchical Clustering)
   1. good for if the dataset has Multicollinearity

Greedy:

1. Co-relation with the target
2. Forward Feature Selection
3. Backward Feature Elimination
4. Stepwise Selection

Embedding Method

1. Random Forest Importance Features
2. Feature Selection using Decision Tree
3. L1 or Lasso Regression
4. Elastic Net Regression
5. Decision Tree

Pros:

1. Improved model performance
2. Reduced overfitting
3. Increased interpretability

Cons:

1. Increased computational complexity