Sampling Distribution of OLS Estimators

 

Sampling Distributions

The Distribution of Response variable “Y”

The assumption of the classical linear regression model is that the disturbance is independently, identically distributed normal with mean zero and fixed variance.

ϵi ~N(0, σ²)

The distribution of the response variable is dependent on the distribution of the disturbance term.

Consider the model

Y= α + βX + ϵ

The mean of the response variable is given by;

E(Y) = α + βX + E (ϵ)

E(Y) = α + βX 

The variance of the response variable is given by;

V(Y) = E[Y - E(Y)] ^2

V(Y) = E[α + βX + ϵ - α - βX] ^2

V(Y) = E[ϵ]^2

V(Y) = σ²

The shape of the sampling distribution is dependent on the disturbance term "ϵ". As the disturbance term follows normal distribution, the sampling distribution of response variable "Y" follows normal distribution with mean (α + βX) and standard deviation (σ²). 

The Sampling Distribution Regression Parameters 

Consider a linear regression model;

Y= α + βX + ϵ

Where:

ϵi ~N(0, σ²)

The response variable follows a normal distribution with a mean (α + βX) and standard deviation (σ²). The estimate of the slope of the simple linear regression model is given by:

let 

Where k follows the following properties.

a. ∑k = 0

b. ∑k X = 1

c. ∑ k² = 1/∑(X-X¯)²

The OLS estimate is expressed as:

β ^ =∑kY

Thus, the OLS estimate "β ^" is a linear function of the response variable.

The estimate of the intercept of the regression model;

α^ = Y¯ - β ^ X¯

α^=∑Y/n-β^ X−

$$\begin{gathered} \text{ \\ α^=∑Y/n-∑(X-X−) Y / ∑(X-X−)^2 X−} \end{gathered}$$

$\alpha ^=\sum \{(1/n\; -\; (X-X-)\; /$∑(X-X−)^2) X−} Y

α^=∑wY

Thus, the OLS estimate "α^" is a linear function of the response variable.

The Sampling Distribution of Mean value of Response Variable “Y” for a given value

Consider the simple linear regression model:

Y= α + βX + ϵ

The predicted value of Y is Y^0 X = X0, given by;

Y^0 = α^ + β^ X0

 The mean of response variable for a specified value of X = X₀

E(Y^0) = E(α^ + β^ X0)

The mean of Y^0 is denoted by μY.X0.

μY.X0 = α + βX0 

The variance of Y^0 is denoted by σ2Y.X0

Var (Y^0) = Var (α^ +  β^ X0)

Var (Y^0) = Var (α^) + Var (β^ X0 ) + 2X0 Cov (α^, β^ X) ....(1)

We know that

Equation (1) becomes:

The sampling distribution of Y^0 approaches a normal distribution.

Sampling Distribution of an Individual value of Response Variable

The primary objective of a regression model is to forecast an individual value of a response variable Y₀ for a specified value of X = X₀. The following estimated equation is used to predict Y^0.

Y^0 = a + b X0

The true value of Y0 of Y (response variable) is given by:

Y0 = a + b X0 + ϵ0

The above equation satisfies the classical assumptions of OLS.

 The mean of true  value Y0 is

E (Y0) = E (a) + E (b) X0 + E (ϵ0)

E (Y0) =   α + βX0 

The variance of true value Y0 is

Var (Y0) = Var (a + bX0 + ϵ0) 

• Read More: Confidence Interval Estimates for slope and Intercept

$$\begin{gathered} \text{ \\ <br>} \end{gathered}$$