R-squared Value

$R^{2}$ gives the percentage of variation between two dataset
It can't show positive or negative relationship like Pearson Correlation
But it can reliably show if two dataset has relationship
And $R^{2}$ doesn't have any scale influence, so same relation with different scale with get same $R^{2}$ value.
As $R^{2}$ only works with Continuous variable. For discrete variables, see Mutual Information

R^2 Formula

$\begin{aligned} R^{2} & = 1 - \frac{V a r (\hat{y})}{V a r (\bar{y})} \\ = \frac{V a r (\bar{y}) - V a r (\hat{y})}{V a r (\bar{y})} \end{aligned}$ $where V a r (z) = \sum_{i} (y_{i} - z)^{2}$ $And \bar{y} = mean and \hat{y} = predicted y$

$R^{2} = 0.81$ means 81% relation is explained by the features than mean

R-squared value always increases with the addition of features even if the feature doesn't have a high co-relation, so most of the time we rely on Adjusted R-squared Value

Related Notes

Log (Odds)
Collinearity
Normal Distribution
Estimated Mean
Median