Curve fitting#
Generally, we will have some datapoint pairs \(x_i, y_i\). We will have some function \(y\) with parameters \(a\) that will be evaluated at a point \(x\): \(y(a, x)\).
Our goal is to find the set of parameters \(a\) that gives us the best fit of the data. Commonly, this implies minimizing the squared error between the prediction and the data,
where \(r\) is the residual vector. The least squares fit is formulated as finding \(a\) so as to minimize \(\| r\|\).
In general, this is an optimization problem (much more complicated than you would expect!) since \(y(a,x)\) can be complex.
In this section we will discuss extending our notions of the interpolation techniques to the best fit case using linear least squares via a concept called the pseudoinverse.
Nonlinear least squares is a more complicated approach which will be covered in the Optimization chapter.