Provide Information Regarding Statistics & Econometrics : Post Hoc Tests Lecture 48

POST HOC Tests

Lecture 48

Introduction

In the case of ANOVA, when null hypotheses are rejected by all of us, we conclude the population means or treatment means are significantly different from each other. The post hoc tests are used to pinpoint which pairs of means are significantly different from one another. The post hoc tests add clarity and deep insight to the initial ANOVA results. There are a number of post hoc tests, each with its own assumption and strengths.

The most common post hoc tests used are given below:

LSD Test

DMRT Test

HSD Test

Scheff’s Test

Bonferroni Test

Dunnett's Test

LSD Test

The LSD (Least Significant Difference) test, often known as Fisher’s LSD test, is the simplest and oldest post hoc test. It pooled the error term from ANOVA to compute the standard error for pairwise comparison of means. The risks associated with the LSD test are not strongly controlled for type error and the difference of all pairs of means compared with a single value.

Procedure to Perform LSD test

1. LSD is applicable in the situation that the null hypothesis or hypothesis is rejected in an ANOVA.

2. Arrange the sample means in ascending order of magnitude.

3. Compute the LSD values as:

Where:

ν: Error degree of freedom in ANOVA.

r: The number of observations in the treatment or sample.

When all treatments have different number of observations, thats

r1≠ r2≠ ..., ≠ rt

Then r can be obtained by the harmonic mean given below:

The sample means difference will be significant, if

Example 12.3 (Repeat): A supervisor is interested in knowing the performance of machines and the operators working on the machines. To perform the experiment, four machines and five operators are selected, and the following observations are collected at the end of the week.

Operator	Machine 1	Machine 2	Machine 3	Machine 4
1	46	56	55	47
2	54	55	51	56
3	48	56	50	58
4	46	60	51	59
5	51	53	53	58

Test the hypotheses at a 5% significance level that the performance of four machines and five operators is identical. Apply the LSD test if applicable.

Solution:

Link: Two Way ANOVA

The following ANOVA table is obtained:

The four machines are significantly different with respect to the number of outcomes. The LSD test is applicable to machine performance only.

The sample means of the output of four machines:

Now computed LSD:

Arrange the sample means in ascending order of magnitude:

Or it can be presented as:

Conclusion: The performance of machine 1 is significantly different from machines 2, 3 and 4.

Duncan’s Multiple Range Test

When the ANOVA-based null hypothesis regarding the equality of several populations' means or treatments is rejected. The conclusion is that there is considerable disparity between the population means. The Duncan’s multiple range test is a follow-up statistical test used to compare sample means when the null hypothesis is rejected in ANOVA. This test is more powerful than LSD when a larger number of pairs is to be compared and to control for type I error.

The DMRT formula is given by;

When the observations per sample are not the same, then r can be obtained as:

If p number sample means are to be compared, the p – 1 Duncan’s (Rp) values are computed.

Where v is the error degree of freedom of ANOVA.

Procedure:

Compute p – 1 Duncan values for p sample means.

Arrange the sample means in ascending order of magnitude.

Compare the absolute difference Yp – Y1 to Rp.

Compare the absolute difference Yp – Y2 to Rp-1.

Continue till all sample means are to the largest sample mean, i.e., Yp.

Drop the Rp for the next loop.

Compare the absolute difference Yp-1 – Y1 to Rp-1.

Compare the absolute difference Yp-1 – Y2 to Rp-1.

Proceed till all the sample means are compared.

Three Duncan values are required to compare four population means.

Operator	Machine 1	Machine 2	Machine 3	Machine 4
1	46	56	55	47
2	54	55	51	56
3	48	56	50	58
4	46	60	51	59
5	51	53	53	58

Test the hypotheses at a 5% significance level that the performance of four machines and five operators is identical. Apply the DMRT.

Solution: The ANOVA table is obtained and given below:

The four machines are significantly different with respect to the number of outcomes. The DMRT test is applicable to machine performance only.

The sample means of the output of four machines in ascending order of magnitude:

From the above pairwise analysis, it is clear that machines 1 and 2 and machines 1 and 4 are significantly different with respect to the out of production.

HSD Test

When the null hypothesis is rejected in an ANOVA, the HSD (Tukey’s Honestly Significant Difference) test is a post hoc test used to determine the significant difference between pairs of sample means.

Where:

qα(p, v) is the studentised range statistic.

P is the number of populations, means or treatments.

V is the error degree of freedom.

To carry the HSD test

Find the HSD value.

Compare the absolute difference to HSD.

The sample means difference will be significant if it exceeds the HSD.

Operator	Machine 1	Machine 2	Machine 3	Machine 4
1	46	56	55	47
2	54	55	51	56
3	48	56	50	58
4	46	60	51	59
5	51	53	53	58

Test the hypotheses at a 5% significance level that the performance of four machines and five operators is identical. Apply the HSD.

Solution: The ANOVA table is obtained and given below:

The sample means:

This pairwise analysis can be expressed as:

According to HSD, the output of machines 1 and 2 is significantly different.

Scheff’s Test

Scheff’s test is a post-hoc test used after a significant ANOVA result to determine which population means or treatments have statistically different means when comparing several population means. This test is used to compare all possible contrasts among the population means or treatments. Contrast is a linear combination of means with the essential that the sum of their contrasts is equal to zero.

If T1, T2, …, Tk is the sum of treatments total based on the sample data, the following linear function is a contrast.

Q = c1T1 + c2T2 + ... + ckTk

c1 + c2 + ... + ck =0

The Scheff’s critical value can be obtained as:

After the rejection of the overall null hypothesis, the pairwise hypotheses are considered as:

H01: μ1 = μ2 vs. H11: μ1 ≠ μ2

This hypothesis is represented by L1 = X¯1 - X¯2 and so on.

The hypothesis will be significant if L1 exceeds Fsi.

Operator	Machine 1	Machine 2	Machine 3	Machine 4
1	46	56	55	47
2	54	55	51	56
3	48	56	50	58
4	46	60	51	59
5	51	53	53	58

Test the hypotheses at a 5% significance level that the performance of four machines and five operators is identical. Apply the Scheffé's method.

Solution: The ANOVA table is obtained and given below:

The performance of four machines is significant; the Scheffé method is applied to four machines.

Bonferroni TestThe Bonferroni test is also known as the Bonferroni correction, developed by Italian mathematician Carlo Emilio Bonferroni. The Bonferroni is a post hoc test that controls the risk of type I error when the number of comparisons increases. This test is pairwise comparing the treatment when the analysis of variance finding is significant. The Bonferroni critical value is given by:

Where:

Reject the follow-up hypothesis: H1i: μi ≠ μj if |X¯i - X¯j| ≥ BC

Operator	Machine 1	Machine 2	Machine 3	Machine 4
1	46	56	55	47
2	54	55	51	56
3	48	56	50	58
4	46	60	51	59
5	51	53	53	58

Test the hypotheses at a 5% significance level that the performance of four machines and five operators is identical. Apply the Bonferroni test.

Solution: The ANOVA table is obtained and given below:

The performance of four machines is significant; the Bonferroni test is applied to four machines.

Dunnett's Test

The Dunnett test is a post hoc test and is applied when the finding of analysis of variance is significant and the experiment consists of a control group and experimental groups. The Dunnett test is used to compare a control group to experimental groups or treatments. If the control group is not available, then Scheff’s test is suitable to utilise. The Dunnett statistic can be computed as:

Where: t Dunnett is the Dunnett table value depending on the number of treatments and error degree of freedom in ANOVA.In Dunnett test control group is comapre with all experimental groups.Reject the follow-up hypothesis: H1i: μi ≠ μj if |X¯c - X¯j| ≥ DExample 12.4: A researcher is interested in checking the performance of three kinds of sales managers working in banks. The first kind of employees are fresh graduates not receiving any type of training and considered a control group. The second and third kinds are received training programmes of weekend training programmes, and the third kind received a full week training programme. The performance of the three kinds of employees is given below:

Fresh Graduate	Receive Training 1	Receive Training 2
7.00	14.00	8.50
16.00	15.50	16.50
10.50	15.00	9.50
13.50	21.00	13.50

Test the hypothesis at 5 % significance level that the performance of three kinds of employees is significantly different. Apply suitable multiple comparison test if applicable.

Solution:

i. State the null and alternative hypothesis:

H0: μ1 = μ2 = μ3 vs. H1: μ1 ≠ μ2 ≠ μ3

ii. The significance level: α = 0.05

iii. The test statistic:

vi. Critical Region:

Reject H0 whn F > F0.05(2, 6) = 5.14

v. Computation:

Presentation in ANOVA Table:

vi. Remarks: The computed F value falls in the rejection area; the sample does not provide sufficient evidence to accept the null hypothesis. Thus, it is concluded that the perforamce of managers recieving three training programs are significantly different.

In this example ANOVA findings is significant and one group (Fresh graduates) is considered as control group. The most suitable test for pairwise analysis is Dunnett test.

The Dunnett critical value can be obtained as: