Moments of X’X/n
Problem
Given:
- a random sample \(\left( y_i,x_i \right)\) of size \(n\) where \(y_i\) is a scalar and \(x_i={\left( x_{i,1},x_{i,2}, \ldots ,x_{i,k} \right)}'\) is \(k×1\)
- \(X\) is \(n×k\) where row \(i\) of \(X\) is \(x'_i\)
- \(E\left( x_ix'_i \right)=Σ_{xx'}\)
a) Show that
\[E\left( \frac{1}{n}X'X \right)=Σ_{xx'}\]
Hint:
\[X'X=\sum_{i}^{n}{ x_ix'_i }\]
b) Show that the variance of each of the elements in \(n^{-1}X'X\) go to zero as \(n→∞\) .
These parts taken together show that
\[\textrm{plim } \left( \frac{1}{n}X'X \right)=Σ_{xx'}\]
Solution
a)
\[E\left( \frac{1}{n}X'X \right)= \frac{1}{n}E\left( X'X \right)= \frac{1}{n}E\left( \sum_{i}^{n}{ x_ix'_i } \right)= \frac{1}{n}\sum_{i}^{n}{ E\left( x_ix'_i \right) }= \frac{1}{n}\sum_{i}^{n}{ Σ_{xx'} }= \frac{1}{n} n Σ_{xx'}=Σ_{xx'}\]
b) \(n^{-1}X'X\) is \(k×k\) and element \(j,k\) of this matrix is
\[ \frac{1}{n}\sum_{i=1}^{n}{ x_{i,j}x_{i,k} }\]
The variance of this term is (random sample)
\[ \frac{1}{n^2}\sum_{i=1}^{n}{ Var\left( x_{i,j}x_{i,k} \right) }= \frac{1}{n}Var\left( x_{i,j}x_{i,k} \right)→0\]
Since the variance of every term in \(n^{-1}X'X\) go to zero and \(E\left( n^{-1}X'X \right)=Σ_{xx'}^{-1}\) , \(\textrm{plim } n^{-1}X'X=Σ_{xx'}^{-1}\) .