Moving Average Models (MA Models) Lecture 17

 Moving Average Models 

(MA Models) 

Lecture 17

The autoregressive model in which the current value 'yt' of the dependent variable is based solely on the past error term(s) is called MA models. 

If the dependent variable "yt" is expressed as a linear combination of q-lag noise terms, the model is called MA(q). model

yt = μ + α1ϵt-1 + α2ϵt-2 + ⋯ + αqϵt-q + ϵt

yt = μ + ϵt + ∑αi ϵt-i

Where:

ϵt∼NIID(0, σ²)

If the dependent variable "yt" is expressed as a linear combination of previous noise terms, the model is called MA(1). model

yt = μ + α1ϵt-1 + ϵt

Or we can write as:

yt = μ + ϵt + α1ϵt-1 

Where:

ϵt∼NIID(0, σ²)

ACF & PACF MA (1) Model

Consider MA (1) model

yt = μ + ϵt + θϵt-1 

E (yt) = E (μ + ϵt + θϵt-1)

E (yt) = E (μ) + E (ϵt) + θE(ϵt-1)

We know that

 E (ϵt) = 0

E (yt) = μ

The covariance of MA (1) model

Cov (yt,yt-1) = E[{yt-E(yt)} {yt-1- E(yt-1)}]

Cov (yt, yt-1) = E [{yt- μ} {yt-1-  μ}]

 

Cov (yt, yt-1) = E [{μ + ϵt + θϵt-1 - μ} {μ + ϵt-1 + θϵt-2 - μ}]

Cov (yt, yt-1) = E [{ϵt + θϵt-1} {ϵt-1 + θϵt-2}]

Cov (yt, yt-1) = E [ϵt ϵt-1 + θϵt-1 θϵt-2 + θ (ϵt-1)^2]

Cov (yt, yt-1) = E (ϵt ϵt-1) + θ^2 E(ϵt-1 ϵt-2) + θ E(ϵt-1)^2]

Cov (yt, yt-1) = θ E(ϵt-1)^2

Cov (yt, yt-1) = θ σ²

Hence, the covariance of yt and yt-1 are not independent.

The variance of MA (1) model

Var(yt) = E[yt-E(yt)] ²

Var(yt) = E[yt-μ] ²

Var(yt) = E[μ + ϵt + θϵt-1-μ] ²

Var(yt) = E[ϵt + θϵt-1] ²

Var(yt) = E[(ϵt)² + (θ)² (ϵt)²]

Var(yt) = E(ϵt) ² + (θ)² E(ϵt)²

Var(yt) = σ² + θ² σ²

Var(yt) =  (1+ θ²) σ²

The yt-1 variance

Var(yt-1) = E[yt-1-E(yt-1)] ²

Var(yt-1) = E[yt-1-μ] ²

Var(yt-1) = E[μ + ϵt-1 + θϵt-2-μ] ²

Var(yt-1) = E[ϵt-1 + θϵt-2] ²

Var(yt-1) = E[(ϵt-1)² + (θ)² (ϵt-2)²]

Var(yt-1) = E(ϵt-1) ² + (θ)² E(ϵt-2)²

Var(yt-1) = σ² + θ² σ²

Var(yt-1) =  (1+ θ²) σ²

Hence, the var(yt) = var(yt-1).

The auto-correlation of order 1 (ACF1)

 

The PACF of order 1 is equal to the ACF. Thus,

Example: Find the ACF and PACF of the MA (1) model given below:

 yt = 10 + ϵt + 0.70 ϵt-1 

Where:

E(ϵt) = 0

Solution: The ACF of MA (1) is given by:

ρ1 = θ^2 / 1+θ^2

ρ1 = (0.70)^2 / 1+(0.70)^2

ρ1 = 0.67

PACF = 0.67

Auto-correlation using MA (2) Model

Consider MA (2) model

yt = μ + ϵt + θ1ϵt-1 + θ2ϵt-2

where:

ϵt is NIID(0, σ²) and E(yt) = μ

Cov(yt, yt-1) = E[{yt - E(yt)} {yt-1 - E(yt-1)}]

Cov(yt, yt-1) = E [{yt - μ} {yt-1 - μ}]

Cov(yt, yt-1) = E [{μ + ϵt + θ1ϵt-1 + θ2ϵt-2 - μ} {μ + ϵt-1 + θ1ϵt-2 + θ2ϵt-3 - μ}]

Cov(yt, yt-1) = E [{ϵt + θ1ϵt-1 + θ2ϵt-2} {ϵt-1 + θ1ϵt-2 + θ2ϵt-3}]

Cov(yt, yt-1) = E [ϵt(ϵt-1 + θ1ϵt-2 + θ2ϵt-3) + θ1ϵt-1(ϵt-1 + θ1ϵt-2 + θ2ϵt-3) + θ2ϵt-2(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] 

Cov(yt, yt-1) = E [ϵt(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] + θ1 E [ϵt-1(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] + θ2 E [ϵt-2(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] ...(1)

Now

E [ϵt(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] = E [ϵt ϵt-1 + θ1ϵt ϵt-2 + θ2ϵt ϵt-3]

E [ϵt(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] = E [ϵt ϵt-1) + θ1 E (ϵt ϵt-2) + θ2 E(ϵt ϵt-3)

E [ϵt(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] = 0 ...(a)

θ1 E [ϵt-1(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] = θ1 E [(ϵt-1)(ϵt-1) + θ1ϵt-1ϵt-2 + θ2ϵt-1ϵt-3)] 

θ1 E [ϵt-1(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] = θ1 E (ϵt-1)² + θ1 E(ϵt-1ϵt-2) + θ2 E(ϵt-1ϵt-3) 

θ1 E [ϵt-1(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] = θ1 σ² ...(b)

θ2 E [ϵt-2(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] = θ2 E [(ϵt-1ϵt-2 + θ1(ϵt-2)² + θ2ϵt-2ϵt-3)]

θ2 E [ϵt-2(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] = θ₂ E (ϵt-1ϵt-2 + θ₁θ₂ E (ϵt-2)² + θ₂θ₂ Eϵt-2ϵt-3)

θ2 E [ϵt-2(ϵt-1 + θ1ϵt-2 + θ2ϵt-3)] =  θ₁θ₂ E (ϵt-2) ²

θ₂ E [ϵt-2(ϵt-1 + θ₁ϵt-2 + θ₂ϵt-3)] = θ₁θ₂ σ² ... (c)

Substitute in equation 1 from a, b, and c.

Cov(yt, yt-1) = θ₁ σ² + θ₁θ₂ σ²

Cov(yt, yt-1) = θ₁ (1 + θ₂) σ²

The variance of yt

Var (yt) = E(yt - E(yt)) ²

Var (yt) = E(yt - μ) ²

Var (yt) = E(μ + ϵt + θ1ϵt-1 + θ2ϵt-2 - μ)²

Var (yt) = E(ϵt + θ1ϵt-1 + θ2ϵt-2)²

Var (yt) = E(ϵt) ² +(θ1) ²E((ϵt-1) ²+(θ2) ²E((ϵt-2) ²

Var (yt) = σ² +(θ1)σ² +(θ2)²σ²

Var (yt) = (1 +(θ1) +(θ2)²)σ²

Example: Find the ACF of the MA (2) model is given by

yt = 10 + ϵt + 0.5ϵt-1 + 0.3ϵt-2

Solution: The ACF of MA (2) is given by:

Estimation of parameters in MA (1) model

The MLE method can be employed to estimate the parameters of the MA (1) model.

 Consider MA (1) model

yt = μ + ϵt + θϵt-1 

E (yt) = E (μ + ϵt + θϵt-1)

E (yt) = E (μ) + E (ϵt) + θE(ϵt-1)

where:

ϵt is NIID(0, σ²)

E (yt) = μ

The pdf of follow normal distribution given below:

ACF & PACF of AR (2) Model Lecture 16

 

ACF & PACF of AR (2) Model

Lecture 16

Consider an AR(2) model.

yt = β0 + β1 yt-1 + β2 yt-2 + ϵt

ignoring intercept

yt = β1 yt-1 + β2 yt-2 + ϵt

We use Yule-Walker equations

 Multiply the AR (2) by its immediate lag yt-1, and take expectations and divide by the variance of yt.

yt x yt-1 = β1 yt-1 x yt-1 + β2 yt-2 x yt-1 + ϵt x yt-1

E (yt x yt-1) = β1E (yt-1 x yt-1) + β2 E (yt-2 x yt-1) + E (ϵt x yt-1)

Cov (yt , yt-1) = β1Var (yt-1) + β2 Cov (yt-2, yt-1) 

Now divide by the variance of yt

Cov (yt , yt-1)/ Var(yt) = β1Var (yt-1) / Var(yt) + β2 Cov (yt-2 x yt-1) / Var(yt)

 ${\rho }_{\mathrm{1\; = }}$β1 + ρ1β2

ρ1-ρ1β2 ${\mathrm{= }}_{}$β1 

ρ1(1- β2) ${\mathrm{= }}_{}$β1 

ρ1 ${\mathrm{= }}_{}$β1/(1- β2) 

Now AR(2) model is multiplied by yt-2 and their expectation is taken.

yt x yt-2 = β1 yt-1 x yt-2 + β2 yt-2 x yt-2 + ϵt x yt-2

E (yt x yt-2) = β1E (yt-1 x yt-2) + β2 E (yt-2 x yt-2) + E (ϵt x yt-2)

Cov (yt, yt-2) = β1Cov (yt-1, yt-2) + β2 Var (yt-2) 

Now divide by the variance of yt

Cov (yt, yt-2) / Var(yt) = β1 Cov (yt-1, yt-2) / Var(yt) + β2 Var(yt-2) / Var(yt)

We know that Var(yt) = Var(yt-2)

Cov (yt, yt-2) / Var(yt) = β1 Cov (yt-1, yt-2) / Var(yt) + β2 

ρ₂ = ρ₁β₁ + β₂

As ρ1 ${\mathrm{= }}_{}$β1/(1- β2) 

ρ₂ = β₁β₁/(1-β₂) + β₂

Example: Consider the AR (2) model

yt = 0.75 yt-1 - 0.25 yt-2 + ut

Find the first- and second-order ACFs.

Solution: From the model

β₁ = 0.75,  β₂ =  -0.25

ρ1 ${\mathrm{= }}_{}$β1/(1- β2) 

ρ1 ${\mathrm{=\; 0.75}}_{}$/(1+ 0.25) 

ρ1 ${\mathrm{=\; 0.75}}_{}$/1.25

 ρ1 ${\mathrm{=\; 0.60}}_{}$

${}_{}$

ACF & PACF of AR (p) model

Consider the AR(p) model

yt = β1 yt-1 + β2 yt-2 + β3 yt-3 + ... + βp yt-p + ϵt

Using the Yule–Walker equation

${}_{}$

The AR(p) model is multiplied by yt-j, take the expectation and divided by Var(yt).

yt = β1 yt-1 + β2 yt-2 + β3 yt-3 + ... + βp yt-p + ϵt

yt x yt-j = β1 yt-1 x yt-j + β2 yt-2 x yt-j + β3 yt-3 x yt-j + ... + βp yt-p x yt-j + ϵt x yt-j

E (yt x yt-j) = β1 E (yt-1 x yt-j) + β2 E (yt-2 x yt-j) + β3 E (yt-3 x yt-j) + ... + βp E (yt-p x yt-j) + E (ϵt x yt-j)

Cov(yt, yt-j) = β1 Cov(yt-1, yt-j) + β2 Cov (yt-2, yt-j)   β3 Cov(yt-3, yt-j) + ... + βp Cov(yt-p, yt-j) 

j = 1, 2, 3, ...

Using Cramer's rule to obtain the estimates of β's.

The first, second order of ACF

For third-order PACF

Example

Find the ACF and PACF using SPSS. 

Month – year	Covid-19Cases  (per million)
Mar-20 Apr-20 May-20 Jun-20 Jul-20 Aug-20 Sep-20 Oct-20 Nov-20 Dec-20 Jan-21 Feb-21 Mar-21 Apr-21 May-21 Jun-21 Jul-21 Aug-21 Sep-21 Oct-21 Nov-21 Dec-21	0.001938 0.015525 0.069496 0.209337 0.278305 0.295636 0.312263 0.332993 0.398026 0.479775 0.544813 0.579973 0.667957 0.820823 0.918936 0.957371 1.029811 1.160119 1.245127 1.272345 1.28484 1.292728

Solution:

The ACF and PACF of Covid 19 spread from March, 2020 to December 2021.

Autocorrelations
Series: VAR00001
Lag	Autocorrelation	Std. Errora	Box-Ljung Statistic
			Value	df	Sig.b
1	.871	.203	18.329	1	.000
2	.721	.198	31.541	2	.000
3	.566	.193	40.139	3	.000
4	.432	.188	45.442	4	.000
5	.313	.182	48.402	5	.000
6	.191	.176	49.580	6	.000
7	.064	.170	49.720	7	.000
8	-.055	.164	49.833	8	.000
9	-.147	.158	50.704	9	.000
10	-.222	.151	52.870	10	.000
11	-.292	.144	57.000	11	.000
12	-.354	.137	63.733	12	.000
13	-.392	.129	73.033	13	.000
14	-.400	.120	84.075	14	.000
15	-.395	.111	96.659	15	.000
16	-.387	.102	111.145	16	.000

 

Partial Autocorrelations
Series: VAR00001
Lag	Partial Autocorrelation	Std. Error
1	.871	.218
2	-.158	.218
3	-.100	.218
4	-.007	.218
5	-.042	.218
6	-.117	.218
7	-.122	.218
8	-.070	.218
9	-.012	.218
10	-.065	.218
11	-.107	.218
12	-.072	.218
13	-.010	.218
14	.008	.218
15	-.065	.218
16	-.071	.218

• Read More: MA Models