Moments of X’ε/n
Problem
Given:
- a linear regression model \(y=Xβ+ε\) with a random sample
- the Gauss-Markov assumptions, \(E\left( ε \right|X)=0\) and \(Var\left( ε \right|X)=σ^2I\)
- \(E\left( x_ix'_i \right)=Σ_{xx'}\)
Show that
\[E\left( \frac{1}{n}X'ε \right)=0\]
Hint: Find \(E\left( \frac{1}{n}X'ε|X \right)\) and use law of iterated expectations
There is also a law of iterated variances. If \(U\) and \(V\) are arbitrary random vectors, then
\[Var\left( U \right)=E\left( Var\left( U \mid V \right) \right)+Var\left( E\left( U \mid V \right) \right)\]
b) Use the law of iterated variances to show that
\[Var\left( \frac{1}{n}X'ε \right)= \frac{σ^2}{n}Σ_{xx'}\]
Hint: set \(U= \frac{1}{n}X'ε\) and \(V=X\)
This demonstrates that
\[\textrm{plim } \frac{1}{n}X'ε=0\]
Solution
a)
\[E\left( \frac{1}{n}X'ε|X \right)= \frac{1}{n}X'E\left( ε|X \right)= \frac{1}{n}X'0=0\]
By LIE,
\[E\left( \frac{1}{n}X'ε \right)=E\left( E\left( \frac{1}{n}X'ε|X \right) \right)=E\left( 0 \right)=0\]
b)
\[Var\left( \frac{1}{n}X'ε \right)=E\left( Var\left( \frac{1}{n}X'ε|X \right) \right)+Var\left( E\left( \frac{1}{n}X'ε \mid X \right) \right)\]
Now
\[Var\left( \frac{1}{n}X'ε|X \right)= \frac{1}{n^2}X'Var\left( ε|X \right)X= \frac{1}{n^2}X'σ^2IX= \frac{σ^2}{n^2}X'X\]
and
\[E\left( Var\left( \frac{1}{n}X'ε|X \right) \right)=E\left( \frac{σ^2}{n^2}X'X \right)= \frac{σ^2}{n^2}E\left( X'X \right)= \frac{σ^2}{n}Σ_{xx'}\]
The second term is zero since
\[E\left( \frac{1}{n}X'ε \mid X \right)=0\]