Analysis 3
Linear Models
A linear model is a model that is linear in the parameters. Recall the example of the trajectory from the previous tutorial. In this example the data <math>d_i</math> was modeled by a quadratic function:
<math> d_i = a t_i^2 + b t_i + c </math>.
Note that this is quadratic in the sense that it is quadratic in time, not in the parameters <math>a,b,c</math>. With respect to the parameters, this model is linear. An example of a nonlinear model is <math> d_i = cos( 2\pi \nu t_i ) </math>, where you are fitting a frequency parameter <math>\nu</math>. In this section you'll learn how to fit linear models to data.
Matrix Notation
When dealing with linear systems, it's generally easier to work with matrices. Continuing with the above example of the trajectory, we introduce the parameter vector <math>\vec{p}</math>, which is a column vector with the parameters <math>a,b,c</math>. We also introduce an <math>N \times 3 </math> dimensional matrix <math> M </math>. The first, second, and third columns of <math>M</math> have the elements <math> [ t_1^2, ..., t_N^2 ] </math>, <math> [ t_1, ..., t_N ] </math>, and <math> [ 1, ..., 1 ] </math>. Finally we introduce the data vector <math>\vec{d}</math>, which contains all the data. Given these definitions we can write
<math> d_i = M\vec{p} </math>.