Applications of ACF and PACF Plots Lecture 15

 

Applications 

of 

ACF and PACF Plots

The ACF and PACF describe the important features of a time series.

• If ACFs of lag p exhibit a geometric decay and PACF has p significant spikes. Then the time series data follow the AR(p) model.

e.g., if the ACF plot portrays geometric decay and the PACF plot has one significant spike, then the data follow the AR (1) model.

If the ACF plot portrays geometric decay and the PACF plot has two significant spikes, then the data follow the AR (2) model, and so on. 

• If the PACF plot portrays geometric decay and the ACF plot has k significant spikes, then the data follow the MA (k) model. 

e.g., if the PACF plot portrays geometric decay and the ACF plot has one significant spike, then the data follow the MA (1) model, and so on.

• If both the ACF of lag p and the PACF of lag q exhibit a geometric decay. Then the data follow the ARMA (p, q) model.

Structures of Auto-correlation Function

The following structures are popular in the autocorrelation functions:

1. Autoregressive (AR) process.

2. Moving average (MA) process.

3. Joint auto-regressive moving average (ARMA) process

Auto-regression (AR) Models

Autoregressive models are used to predict the current or future value of an activity solely based on the lag value(s) of the same activity.

AR(p) model

If the value of a time series "yt" is regressed on immediate p lags, "yt-1, yt-2, yt-3, ..., yt-p", then the model is called the Pth order auto-regressive model or AR (p) model.

yt = β0 + β1yt-1 + β2yt-2 + βpyt-p + ϵt

yt = β0 + ∑βi yt-i + ϵt

Where:

ϵt is the noise term and ϵt ~iid N(0, σ^2).

If yt is regressed on the immediate lag “yt-1”, then the model is called the first-order autoregressive model or the AR(1) model.

yt = β0 + β1yt-1 +  ϵt

Where:

ϵt is the noise term, and ϵt ~iid N(0, σ^2).

If yt is regressed on immediate 2 lags “yt-1, yt-2”, then the model is called a second-order auto-regression model or an AR(2) model.

yt = β0 + β1yt-1 + β2yt-2 + ϵt

Where:

ϵt is the noise term, and ϵt ~iid N(0, σ^2).

ACF & PACF of AR (1) Model

Consider the AR(1) model

yt = β0 + β1yt-1 +  ϵt

ignoring intercept

yt = β1yt-1 +  ϵt

The AR(1) model of yt-1, yt-2, ..., yt-p

yt-1 = β1yt-2 +  ϵt-1

yt-2 = β1yt-3 +  ϵt-2

.

.

yt-p = β1yt-p-1 +  ϵt-p

Thus

yt = β1(β1 yt-2 +  ϵt-1) +  ϵt

yt = β1^2 yt-2 + β1ϵt-1 +  ϵt

Substitute yt-2. 

yt = β1^2 (β1yt-3 +  ϵt-2) + β1ϵt-1 +  ϵt

yt = β1^3 yt-3 + β1^2 ϵt-2 + β1ϵt-1 +  ϵt

The autoregressive equation can be expressed as:

yt = ϵt + β1ϵt-1 + β1^2 ϵt-2 + β1^3 yt-3

In general

yt = ϵt + β1ϵt-1 + β1^2 ϵt-2 + β1^3 ϵt-2 + β1^4 ϵt-4 + ... + β1^p yt-p

 The effect of yt-p is ignored if it is too far from yt. The above-stated model can be expressed as:

yt = ϵt + β1ϵt-1 + β1^2 ϵt-2 + β1^3 ϵt-2 + β1^4 ϵt-4 + ... 

The mean of the model

E (yt) = E [ϵt + β1ϵt-1 + β1^2ϵt-2 + β1^3ϵt-2 + β1^4ϵt-4 + ...] 

E (yt) = E (ϵt) + β1 E (ϵt-1) + β1^2 E (ϵt-2) + β1^3 E (ϵt-2) + β1^4 E (ϵt-4) + ... 

E (yt) = 0

The variance of the model

Var(yt) = E(yt - E(yt)) ²

Var(yt) = E(yt)²

Var(yt) = E(ϵt + β1ϵt-1 + β1^2ϵt-2 + β1^3ϵt-2 + β1^4ϵt-4 + ... ) ²

Var(yt) = E(ϵt) ² + (β1)² E(ϵt−1) ² + (β₁²) ² E(ϵt−2)² + (β₁³)² E(ϵt−3)² + ...

Var(yt) = σ² + (β1) ² σ² + (β1^2) ² σ² + (β1^3) ² σ² + ...

Var(yt) = σ² + (β1)² σ² + (β1)^4 σ² + (β1)^6 σ² + ...

Var(yt) = [1 + (β1)² + (β1)⁴ + (β1)⁶ + ...] σ²

Var(yt) = [1 + (β1)² + (β1)⁴ + (β1)⁶ + ...] σ²

$Var(yt)\; =\; \sigma ²$ (1-β1)^-1

$Var(yt)\; =\; \sigma ²\; /$ (1-β1)

for |β1| ≥ 1

Var(yt) = ∞

The variance of yt depends on the slope of its lag.

The autocovariance function of yt and yt-1

Cov(yt, yt-1) = E[{(yt - E(yt)}{yt-1-E(yt-1)}]

Cov(yt, yt-1) = E[yt yt-1]

Cov(yt, yt-1) = E [{ϵt + β1ϵt-1 + β1^2ϵt-2 + β1^3ϵt-2 + β1^4ϵt-4 + ...} { ϵt-1 + β1ϵt-2 + β1^2ϵt-3 + β1^3ϵt-4 + β1^4ϵt-5 + ...}]

Cov(yt, yt-1) = β₁E(ϵt−1)² + β₁³ E(ϵt−2) ² + (β1)^5 E(ϵt−3)² + ...

Cov(yt, yt-1) = β1σ² + β₁³ σ² + (β1)^5σ² + ...

Cov(yt, yt-1) = [β1 + β₁³ + (β1)^5 + ...] σ²

Cov(yt, yt-1) = β1[1 + (β1)² + (β1)⁴ + ...] σ²

$$Cov(yt, yt-1) = σ² β1(1-β1)^-1

$<span\; style="font-family:\; "Times\; New\; Roman",\; serif;\; font-size:\; 16px;\; text-align:\; center;">Cov(yt,\; yt-1)</span> =$β1 $\sigma ²\; /$ (1-β1)

The autocorrelation function 

We know that Var(yt) = Var(yt-1) because it is assumed that the noise term is homoscedastic. 

The PACF of order 1 is equal to the ACF, so PACF = β1.

Practice Question 

Determine the mean, variance, covariance and ACF and PACF of the following model.

yt = 0.60 + 0.40 yt-1 + ϵt

Var(ϵt) = 0.75

Solution:

β₀ = 0.60, β₁ = 0.40

We know that the ACF of order 1 is

ρ₁ = β₁ = 0.40

The ACF of order 1 is equal to the PACF of order 1, so PACF = 0.40.

Mean

E (ϵt) = 0

Variance:

 (1-β1)

$Var(yt)\; =\; 0.75\; /$ (1-0.40)

Var(yt) = 1.25

Covariance:

$<span\; style="font-family:\; "Times\; New\; Roman",\; serif;\; font-size:\; 16px;\; text-align:\; center;">Cov(yt,\; yt-1)</span> =$β1  (1-β1)

$<span\; style="font-family:\; "Times\; New\; Roman",\; serif;\; font-size:\; 16px;\; text-align:\; center;">Cov(yt,\; yt-1)</span> =$0.40 x 0.75$/$ (1-0.40)

Cov(yt, yt-1) = 0.50

• Read More; ACF & PACF of AR(2) Model