Estimation of Regression Parameters by OLS Method Lecture 05

 

Estimation of Regression Parameters

by

OLS Method

In previous lectures, we discussed regression, estimating parameters from numerical facts, and goodness of fit. In this lecture, we will look at how these formulas can be produced by the OLS method, as well as their characteristics.

Ordinary Least Squares Approach

The ordinary least squares (OLS) approach is a linear regression methodology for estimating unknown parameters in a model. The method is based on minimizing the sum of squared residuals between the observed (observed values of the response variable) and predicted (model) values of the response variable.

Consider the simple linear regression model:

Y= α + βX + ϵ

The regression model in terms of sample:

If the regression model for an activity or phenomenon is successfully and correctly defined, the deviation or error component is zero. Using this assumption, the predicted model will be:

The sum of squares residual is defined as:

According to the principle of least squares, we determine the values of intercept and slope, which will minimize the sum of squares residual (Ch, 1990). 

To estimate α, differentiate eq. 1 with respect to b and equate to zero.

To estimate β, differentiate eq. 1 with respect to b and equate to zero.

Properties of OLS Estimates

Consider the simple linear Regression Model

Y= α + βX + ϵ

The estimated model is given by:

Where:

Express in deviated form:

i. The OLS estimate is a linear function of the response variable.

Hence, slope is a linear function of the response variable.

Where:

Properties of k:

Intercept is a linear function of the response variable.

Where:

Properties of Wi:

ii. The OLS estimates are unbiased estimators of regression parameters.

The OLS estimate for intercept is given by:

iii. The variance of slope is given by:

Now the variance of the OLS estimate of intercept is defined as:

iv. The covariance of intercept and slope is defined as:

• Read More: Features of OLS Estimates