Analysis 3
Linear Models
A linear model is a model that is linear in the parameters. Recall the example of the trajectory from the previous tutorials. In this example the data <math>d_i</math> were modeled by a quadratic function:
<math> d_i = a t_i^2 + b t_i + c </math>.
Note that this is quadratic in the sense that it is quadratic in time, not in the parameters <math>a,b,c</math>. With respect to the parameters, this model is linear. An example of a nonlinear model is <math> d_i = cos( 2\pi \nu t_i ) </math>, where you are fitting a frequency parameter <math>\nu</math>. In this section you'll learn how to fit linear models to data.
Matrix Notation
When dealing with linear systems, it's generally easier to work with matrices. Continuing with the above example of the trajectory, we introduce the parameter vector <math>\vec{p}</math>, which is a column vector with the parameters <math>a,b,c</math>. We also introduce an <math>N \times 3 </math> dimensional matrix <math> M </math>. The first, second, and third columns of <math>M</math> have the elements <math> [ t_1^2, ..., t_N^2 ] </math>, <math> [ t_1, ..., t_N ] </math>, and <math> [ 1, ..., 1 ] </math>. Finally we introduce the data vector <math>\vec{d}</math>, which contains all the data. Given these definitions we can write
<math> \vec{d} = M\vec{p} </math>.
Fitting a Linear Model
In the previous tutorial, we the goodness of fit parameter <math>\chi^2</math>. For the trajectory example, we have
<math> \chi^2 = \sum_i^N \frac{(d_i - (a t_i^2 + b t_i + c))^2}{s_i^2} </math>,
We can continue to simplify our analysis by introducing the noise correlation matrix <math>D</math>, which for our purposes will be an <math>N \times N</math> matrix with <math>s_i^2</math> on the diagonal. With this matrix the <math>\chi^2</math> can be written as
<math> \chi^2 = (\vec{d}-M\vec{p})^T D^{-1} (\vec{d}-M\vec{p}) </math>.
The best fit parameters will minimize this function. Therefore, setting the gradient of this function with respect to the parameters <math>\vec{p}<\math> equal to zero, we get an equation for the best fit parameters
<math> = M^T D^{-1} M \vec{p} = M^T D^{-1} </math>.