Moving Average Models (MA Models) Lecture 17

 Moving Average Models 

(MA Models) 

Lecture 17

The autoregressive model in which the current value 'yt' of the dependent variable is based solely on the past error term(s) is called MA models. 

If the dependent variable "yt" is expressed as a linear combination of q-lag noise terms, the model is called MA(q). model

yt = μ + α1ϵt-1 + α2ϵt-2 + ⋯ + αqϵt-q + ϵt

yt = μ + ϵt + ∑αi ϵt-i

Where:

ϵt∼NIID(0, σ²)

If the dependent variable "yt" is expressed as a linear combination of previous noise terms, the model is called MA(1). model

yt = μ + α1ϵt-1 + ϵt

Or we can write as:

yt = μ + ϵt + α1ϵt-1 

Where:

ϵt∼NIID(0, σ²)

ACF & PACF MA (1) Model

Consider MA (1) model

yt = μ + ϵt + θϵt-1 

E (yt) = E (μ + ϵt + θϵt-1)

E (yt) = E (μ) + E (ϵt) + θE(ϵt-1)

We know that

 E (ϵt) = 0

E (yt) = μ

The covariance of MA (1) model

Cov (yt,yt-1) = E[{yt-E(yt)} {yt-1- E(yt-1)}]

Cov (yt, yt-1) = E [{yt- μ} {yt-1-  μ}]

 

Cov (yt, yt-1) = E [{μ + ϵt + θϵt-1 - μ} {μ + ϵt-1 + θϵt-2 - μ}]

Cov (yt, yt-1) = E [{ϵt + θϵt-1} {ϵt-1 + θϵt-2}]

Cov (yt, yt-1) = E [ϵt ϵt-1 + θϵt-1 θϵt-2 + θ (ϵt-1)^2]

Cov (yt, yt-1) = E (ϵt ϵt-1) + θ^2 E(ϵt-1 ϵt-2) + θ E(ϵt-1)^2]

Cov (yt, yt-1) = θ E(ϵt-1)^2

Cov (yt, yt-1) = θ σ²

Hence, the covariance of yt and yt-1 are not independent.

The variance of MA (1) model

Var(yt) = E[yt-E(yt)] ²

Var(yt) = E[yt-μ] ²

Var(yt) = E[μ + ϵt + θϵt-1-μ] ²

Var(yt) = E[ϵt + θϵt-1] ²

Var(yt) = E[(ϵt)² + (θ)² (ϵt)²]

Var(yt) = E(ϵt) ² + (θ)² E(ϵt)²

Var(yt) = σ² + θ² σ²

Var(yt) =  (1+ θ²) σ²

The yt-1 variance

Var(yt-1) = E[yt-1-E(yt-1)] ²

Var(yt-1) = E[yt-1-μ] ²

Var(yt-1) = E[μ + ϵt-1 + θϵt-2-μ] ²

Var(yt-1) = E[ϵt-1 + θϵt-2] ²

Var(yt-1) = E[(ϵt-1)² + (θ)² (ϵt-2)²]

Var(yt-1) = E(ϵt-1) ² + (θ)² E(ϵt-2)²

Var(yt-1) = σ² + θ² σ²

Var(yt-1) =  (1+ θ²) σ²

Hence, the var(yt) = var(yt-1).

The auto-correlation of order 1 (ACF1)

 

The PACF of order 1 is equal to the ACF. Thus,

Example: Find the ACF and PACF of the MA (1) model given below:

 yt = 10 + ϵt + 0.70 ϵt-1 

Where:

E(ϵt) = 0

Solution: The ACF of MA (1) is given by:

ρ1 = θ^2 / 1+θ^2

ρ1 = (0.70)^2 / 1+(0.70)^2

ρ1 = 0.67

PACF = 0.67

Auto-correlation using MA (2) Model

Consider MA (2) model

yt = μ + ϵt + θ1ϵt-1 + θ2ϵt-2

where:

ϵt is NIID(0, σ²) and E(yt) = μ

Cov(yt, yt-1) = E[{yt - E(yt)} {yt-1 - E(yt-1)}]

Cov(yt, yt-1) = E [{yt - μ} {yt-1 - μ}]

Cov(yt, yt-1) = E [{μ + ϵt + θ1ϵt-1 + θ2ϵt-2 - μ} {μ + ϵt-1 + θ1ϵt-2 + θ2ϵt-3 - μ}]

Cov(yt, yt-1) = E [{ϵt + θ1ϵt-1 + θ2ϵt-2} {ϵt-1 + θ1ϵt-2 + θ2ϵt-3}]

Cov(yt, yt-1) = E [ϵt(ϵt-1 + θ1ϵt-2 + θ2ϵt-3) + θ1ϵt-1(ϵt-1 + θ1ϵt-2 + θ2ϵt-3) + θ2ϵt-2(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] 

Cov(yt, yt-1) = E [ϵt(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] + θ1 E [ϵt-1(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] + θ2 E [ϵt-2(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] ...(1)

Now

E [ϵt(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] = E [ϵt ϵt-1 + θ1ϵt ϵt-2 + θ2ϵt ϵt-3]

E [ϵt(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] = E [ϵt ϵt-1) + θ1 E (ϵt ϵt-2) + θ2 E(ϵt ϵt-3)

E [ϵt(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] = 0 ...(a)

θ1 E [ϵt-1(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] = θ1 E [(ϵt-1)(ϵt-1) + θ1ϵt-1ϵt-2 + θ2ϵt-1ϵt-3)] 

θ1 E [ϵt-1(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] = θ1 E (ϵt-1)² + θ1 E(ϵt-1ϵt-2) + θ2 E(ϵt-1ϵt-3) 

θ1 E [ϵt-1(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] = θ1 σ² ...(b)

θ2 E [ϵt-2(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] = θ2 E [(ϵt-1ϵt-2 + θ1(ϵt-2)² + θ2ϵt-2ϵt-3)]

θ2 E [ϵt-2(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] = θ₂ E (ϵt-1ϵt-2 + θ₁θ₂ E (ϵt-2)² + θ₂θ₂ Eϵt-2ϵt-3)

θ2 E [ϵt-2(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] =  θ₁θ₂ E (ϵt-2) ²

θ₂ E [ϵt-2(ϵt-1 + θ₁ϵt-2 + θ₂ϵt-3)] = θ₁θ₂ σ² ... (c)

Substitute in equation 1 from a, b, and c.

Cov(yt, yt-1) = θ₁ σ² + θ₁θ₂ σ²

Cov(yt, yt-1) = θ₁ (1 + θ₂) σ²

The variance of yt

Var (yt) = E(yt - E(yt)) ²

Var (yt) = E(yt - μ) ²

Var (yt) = E(μ + ϵt + θ1ϵt-1 + θ2ϵt-2 - μ)²

Var (yt) = E(ϵt + θ1ϵt-1 + θ2ϵt-2)²

Var (yt) = E(ϵt) ² +(θ1) ²E((ϵt-1) ²+(θ2) ²E((ϵt-2) ²

Var (yt) = σ² +(θ1)σ² +(θ2)²σ²

Var (yt) = (1 +(θ1) +(θ2)²)σ²

Example: Find the ACF of the MA (2) model is given by

yt = 10 + ϵt + 0.5ϵt-1 + 0.3ϵt-2

Solution: The ACF of MA (2) is given by:

Estimation of parameters in MA (1) model

The MLE method can be employed to estimate the parameters of the MA (1) model.

 Consider MA (1) model

yt = μ + ϵt + θϵt-1 

E (yt) = E (μ + ϵt + θϵt-1)

E (yt) = E (μ) + E (ϵt) + θE(ϵt-1)

where:

ϵt is NIID(0, σ²)

E (yt) = μ

The pdf of follow normal distribution given below: