Reading: Chapter 5, chapter 15.
So far we have defined:
Now suppose
.
Then
Next we compute the variance of Z. Note that
has ijth entry
So
Now suppose that
Now
has ith component
Moreover,
The last three lines need some justification. The point is that
matrix multiplication by A, whose entries are constants,
is just like multiplication by a constant -- you can pull the
constant outside of the expected value sign. Here is the justification.
The ijth entry in
is
So, .
Thus means that
The following do not use the normal assumption:
So
is unbiased.
If, also,
then
and
The fitted vector is
The residual vector is
where
M=I-X(XT X)-1 XT).
Notation:
Jargon: H is called the hat matrix. Notice that
I tried, as I was calculating things for the vectors , and to emphasize which things needed which assumptions.
So for instance we have following matrix identities which depend
only on the model equation
If we add the assumption that
for each i then we get
If we add the assumption that the errors are homoscedastic
(
for all i) and uncorrelated
(
for all )
then we can compute variances and get
NOTE: usually we assume that the are independent and identically distributed which guarantees the homoscedastic, uncorrelated assumption above.
Next we add the assumption that the errors are independent normal variables. Then we conclude that each of , and have Multivariate Normal distributions with the means and variances as just described.