• Full form of PCA = Principal Component Analysis
• PCA is the linear combination of features
• Principal Component 1 represents the most features
• PCA is used for Dimensionality Reduction and visualizing high dimensional data
• The difference between PC1 is more important than difference between PC2 and PC2 is more important that PC3 and ....
• The newly project dataset have maximum variance in the first component direction, then next, then next and so on....
  • The variance can be termed as measure of information, so the data has more information on the first component direction rather than the next ones.

Steps:

1. For each dimension, get the Mean
2. Plot that point
3. Transform that point to origin
   1. That will transfer all features with same aspect
4. Fit a line with lowest Sum of Residual
5. That is the PC1
6. For next Principal Component $P{C}_i$, draw a perpendicular on $P{C}_{1...(i-1)}$
7. Project the $D$ dimensional data to all the principal components (usually 2/3)
8. Now rotate PC1 to make it horizontal with X axis
   1. That will rotate the features too
9. That is the final PCA

• If Sum of squared distance for PC1 is 15 and for PC2 is 3
• Then 83% ($\frac{15}{15+18}$) data are represented by PC1

Practical Tips:

1. All variables are on same scale, i.e., [0, 1] (Normalization)
2. Make sure data is centered on origin (if not done by library)

References