Forward Stagewise Additive Modeling

Definition

The forward stagewise additive modeling sequentially adds new basis functions to the model of the previous step without adjusting the parameters and coefficients of those that have already been added.

Algorithm

1. Initialize $f_0(x)=0$
2. For $m=1,2,\dots ,M$
   1. Compute $({\beta }_m,{\mathbf{\gamma }}_m)=\underset{\beta ,\mathbf{\gamma }}{\mathrm{argmin}}\sum _{i=1}^{n}L(y_i,f_{m-1}(x_i)+\beta b(x_i;\mathbf{\gamma }))$
   2. Set $f_m(x)=f_{m-1}(x)+{\beta }_{m}b(x;{\mathbf{\gamma }}_m)$ where $b(x;\mathbf{\gamma })$ is a basis function and $L(y,f(x))$ is a Loss Function