Smoothing Splines

Definition

$RSS(f,\lambda )=\sum _{i=1}^N(y_i-f(x_i){)}^2+\lambda \int f^{\prime \prime }(t{)}^{2}dt$ where $\lambda \ge 0$ is a smoothing parameter. If $\lambda =0$, the smoothing spline is any interpolating spline. If $\lambda =\infty$, the smoothing spline is Least Square line.

Smoothing splines is a Spline basis method that avoids the knot selection problem. Among all functions $f(x)\in C^2$, find one that minimizes the penalized RSS.

Facts

The unique analytic solution of smoothing spline is a Natural Cubic Splines with knots at the unique values of the $x_i$‘s