Gradient Boost (Regression)

#machine-learning #interview

Steps:

Get initial guess( $p r e d i c t i o n$ ) by taking the average of the dataset
Get $r e s i d u a l_o u t p u t = g r o u n d_t r u t h - g u e s s$
Build a tree to fit the $r e s i d u a l_o u t p u t$
Now the prediction will be $p r e d i c t i o n = p r e d i c t i o n + l r * n e w_t r e e_p r e d i c t i o n$
1. remember this $n e w_t r e e_p r e d i c t i o n$ is predicted residual
Go to step 2 unless predetermined limit is reached

Related Notes