Loading [MathJax]/jax/output/HTML-CSS/jax.js

Example Problems on Familywise Error

ST703 Homework 6 on Familywise Error

Problems: 1, 2, 3, 4

1

Refer to the data in Exercise 8.17 on page 295 of Rao, where cholesterol levels of women in seven menopausal groups are to be compared. Conduct all (72) pairwise comparisons of group means using the methods specified below. Compare and contrast your conclusions.

Notice that for these problems, the T value was calculated as

t=|¯yi¯yjMSE(1/ni+1/n2)|.

Where the MSE is 1706. From this we get the following table.

ij¯yini¯yjnjt122251121190.7541211511322511195231.98131051142251124830.8549314131522511232130.4136860961622511210230.9906552551722511162214.098095142232119195230.98523686324211924831.343703566252119232131.172496005262119210230.061577304272119162212.977671892341952324832.0903751193519523232132.581645193619523210231.2315460793719523162212.647107958452483232130.604787465462483210231.49875952472483162213.3734446725623213210231.5350322755723213162214.8023122156721023162213.850338847

For the following problems, the table will be sorted by t-value.

(a)

Bonferroni to control familywise error rate to be at most 0.05.

We want to compare our t-values to the following value for the Bonferroni adjustment.

t0.05/(2k).nk=t0.05/(212),96=3.121027

(b)

Scheffe to control familywise error rate to be at most 0.05.

We want to compare our t-values to the following value for the Scheffe adjustment.

(t1)Ft1dferror,α=(71)F7196,0.05=3.628649335

(c)

Tukey(-Kramer) to control familywise error rate to be at most 0.05.

We want to compare our t-values to the following value for the Tukey-Kramer adjustment.

qt,dferror,α1/2=q7,96,0.051/2=3.011567781

(d)

Benjamini-Hochberg to control false discovery rate to be at most 0.05.

For the Benjamini-Hochberg procedure, we will change the comparison value each for each pair.

In the following table, an empty cell will denote a failure to reject the null hypothesis that the pair is different and a will denote a rejection of the null.

ijtBenjamini αBHBonferroniScheffeTukey-Kramer260.0615773040.05150.4136860960.047619048450.6047874650.045238095120.7541211510.042857143140.8549314130.04047619230.9852368630.038095238160.9906552550.035714286251.1724960050.033333333361.2315460790.030952381241.3437035660.028571429461.498759520.026190476561.5350322750.023809524131.981310510.021428571342.0903751190.019047619352.581645190.016666667372.6471079580.014285714272.9776718920.011904762473.3734446720.00952381673.8503388470.007142857174.0980951420.004761905574.8023122150.002380952Totals7434

You’ll notice that the Benjamini-Hochberg procedure has the most rejections with 7, the Bonferroni and Tukey-Kramer both have 4, and the Scheffe has 3 rejections.

2

The conclusions obtained on applying Fisher, Scheffe, Duncan, and Tukey multiple pairwise comparison procedures to the same set of six sample means may be summarized as follows,

a.

b.

c.

d.

Identify, giving reasons, the procedure that was responsible for the conclusion in each case. (Note: We have not discussed Duncan’s adjustment procedure, but it is enough to know that Duncan tends to have fewer rejections than the Fisher procedure, and more rejections than the Tukey procedure.)

In order from least conservative to most conservative:

  1. c - Fisher; this is usually the least conservative as it does not control FWE at all. The line says that means ordered 1-4 are not different. There are 9 rejections.
  2. a - Scheffe; this is the generally most conservative test so the line says that none of the means differ. There are 0 rejections.
  3. b - Duncan; this is more conservative than Fisher, but less than Tukey. The lines say that means ordered 1-4 are not different and means ordered 4-6 are not different. There are 8 rejections.
  4. d - Tukey; this is generally least conservative than Fisher, more than Duncan, and generally less conservative than Scheffe. The line says that means ordered 1-5 are not different. There are 5 rejections.

3

Consider the experiment described as Rao Example 8.2 (p. 280-281). Let the five treatment means be denoted μ1, μ2, μ3, μ4, μ5 Consider these four contrasts:

θ1=μ2+μ3μ4μ5θ2=μ2μ3+μ4μ5θ3=μ2μ3μ4+μ5θ4=μ1+14(μ2+μ3+μ4+μ5)

(a)

Is this set of contrasts mutually orthogonal?

Yes, this set of contrasts are mutually orthogonal. We can show this by looking at them each pairwise. Since they are the same sample size, we just need to multiply the coefficients on each mui together and add those products.

12:00+11+11+11+11=0+111+1=013:00+11+11+11+11=0+11+11=014:01+1/41+1/41+1/41+1/41=0+1/41/4+1/4+1/4=023:00+11+11+11+11=0+1+111=024:01+1/41+1/41+1/41+1/41=01/4+1/41/4+1/4=034:00+1/41+1/41+1/41+1/41=0+1/41/4+1/4+1/4=0

(b)

Compute the sum of squares associated with each contrast.

The general form to compute the sum of squares for a contrast is

SS(ˆθ)=c1^μ1+c2^μ2+c3^μ3+c4^μ4+c5^μ5c21n1+c22n2+c23n3+c24n4+c25n5

Using that, we get

SS(^θ1)=34.3396SS(^θ2)=1.1664SS(^θ3)=1.0816SS(^θ4)=4.49252.

(c)

Compute the SUM of the four sums of squares computed in part (b).

34.3396+1.1664+1.0816+4.49352=41.0811

(d)

Compute the treatment sum of square in the ANOVA.

SSTreat=ti=1nij=1(ˉyi+ˉy++)2=4(30.8433.072)2+4(31.933.072)2+4(34.0233.072)2+4(34.2933.072)2+4(34.3133.072)2=41.0811

(e)

Briefly describe the “effect” of being estimated by each contrast, using language of the experiment.

θ1=(μ2+μ3)(μ4+μ5)

Is the sum of the means for the source A low intensity and source A high intensity groups different than the sum of the means for the source B low intensity and source B high intensity groups?

θ2=(μ2+μ4)(μ3+μ5)

Is the sum of the means for the source A low intensity light and source B low intensity light groups different than the sum of the means for source A high intensity and source B high intensity groups?

θ3=(μ2+μ5)(μ3+μ4)

Is the sum of the means for the source A low intensity light and source B high intensity light different than the sum of the mans for source A high intensity light and source B low intensity light?

θ4=μ1+14(μ2+μ3+μ4+μ5)

Is the mean of the darkness group different than the average means of all the other groups under lights?

(f)

Use SAS to conduct all (52) pairwise comparisons of group means using the methods specified below. Also obtain 95% confidence intervals along with each hypothesis test. Compare and contrast your conclusions.

This is the code for the comparison tests. The proc glm performs Scheffe, Bonferroni, and Tukey-Kramer adjustments and the proc multtest performs the Benjamini-Hochberg procedure. The groups are coded 1 = D, 2 = AL, 3 = AH, 4 = BL, 5 = BH.

proc glm data=plants;
  class group;
  model height=group / clparm e;
  means group;
  contrast 'theta1'  group 0 1 1 -1 -1;
  contrast 'theta2'  group 0 1 -1 1 -1; 
  contrast 'theta3'  group 0 1 -1 -1 1;
  contrast 'theta4'  group 4 -1 -1 -1 -1; 

  estimate 'theta1'  group 0 1 1 -1 -1;
  estimate 'theta2'  group 0 1 -1 1 -1; 
  estimate 'theta3'  group 0 1 -1 -1 1;
  estimate 'theta4'  group 4 -1 -1 -1 -1; 

  means group / t scheffe bon tukey cldiff;
run;

proc multtest data=plants order=data fdr 
          plots=(adjusted(unpack) pbytest(vref=.05));
  class group;
  test mean(height / ddfm=pooled); 
  contrast '1-2' 1 -1;
  contrast '1-3' 1 0 -1; 
  contrast '1-4' 1 0 0 -1; 
  contrast '1-5' 1 0 0 0 -1; 
  contrast '2-3' 0 1 -1; 
  contrast '2-4' 0 1 0 -1; 
  contrast '2-5' 0 1 0 0 -1; 
  contrast '3-4' 0 0 1 -1;
  contrast '3-5' 0 0 1 0 -1;
  contrast '4-5' 0 0 0 1 -1;
run;

i.

Bonferroni to control familywise error rate to be at most 0.05.

Bonferroni has 3 rejections between groups 4 and 3, 5 and 3, and 1 and 3.

Bonferroni

ii.

Scheffe to control familywise error rate to be at most 0.05.

Scheffe has the same 3 rejections as Bonferroni (4-3, 4-3, 1-3), but notice that it has different confidence intervals.

Scheffe

iii.

Tukey(-Kramer) to control familywise error rate to be at most 0.05.

Tukey-Kramer contains the same 3 rejections as Scheffe and Bonferroni and has 2 additional rejections, 4-2, 4-3, 5-2, 5-3, and 1-3.

Tukey-Kramer

iv.

Benjamini-Hochberg to control false discovery rate to be at most 0.05. But don’t attempt to find confidence intervals.

Looking at the false discovery rates that are below our 0.05 threshold gives us the 5 rejections from Tukey-Kramer and 1 additional rejection, 1-2, 1-3, 2-4, 2-5, 3-4, and 3-5.

Benjamini-Hochberg

4

García-Arenzana et al. (2014) tested associations of 25 dietary variables with mammographic density, an important risk factor for breast cancer, in Spanish women. They found the following results:

Dietary valuepvalueTotal calories <0.001Olive oil0.008Whole milk0.039White meat0.041Proteins0.042Nuts0.06Cereals and pasta0.074White fish0.205Butter0.212Vegetables0.216Skimmed milk0.222Red meat0.251Fruit0.269Eggs0.275Blue fish0.34Legumes0.341Carbohydrates0.384Potatoes0.569Bread0.594Fats0.696Sweets0.762Dairy products0.94Semi-skimmed milk0.942Total meat0.975Processed meat0.986

(a)

By hand, apply the Benjamini-Hochberg Step-Up procedure to control the false discovery rate to be at most α=0.25 using the sorted raw p-values: reject each null hypothesis having pjTBH, where

TBH=max{p(j):p(j)αj/k,1jk}p(1)p(k) Dietary Valuep-valueαResultprocessed meat0.9860.25Fail to Rejecttotal meat0.9750.24Fail to Rejectsemi-skimmed milk0.9420.23Fail to Rejectdairy produce0.940.22Fail to Rejectsweets0.7620.21Fail to Rejectfats0.6960.2Fail to Rejectbread0.5940.19Fail to Rejectpotatoes0.5690.18Fail to Rejectcarbohydrates0.3840.17Fail to Rejectlegumes0.3410.16Fail to Rejectblue fish0.340.15Fail to Rejecteggs0.2750.14Fail to Rejectfruit0.2690.13Fail to Rejectred mead0.2510.12Fail to Rejectskimmed milk0.2220.11Fail to Rejectvegetables0.2160.1Fail to Rejectbutter0.2120.09Fail to Rejectwhite fish0.2050.08Fail to Rejectcereals and pasta0.0740.07Fail to Rejectnuts0.060.06Rejectprotiens0.0420.05Rejectwhite meat0.0410.04Rejectwhole milk0.0390.03Rejectolive oil0.0080.02Rejecttotal calories0.00000010.01Reject

(b)

By hand, apply the Benjamini-Hochberg Step-Up procedure to control the false discovery rate to be at most α=0.25 using the sorted raw p-values: reject each null hypothesis having padjjα, where

padj(k)=p(k),padj(j)=min{padj(j+1),kjpadj(j)}j=k1,,1 Dietary Valuep-valueαResultprocessed meat0.9860.986Fail to Rejecttotal meat0.9750.986Fail to Rejectsemi-skimmed milk0.9420.986Fail to Rejectdairy produce0.940.986Fail to Rejectsweets0.7620.907142857Fail to Rejectfats0.6960.87Fail to Rejectbread0.5940.781578947Fail to Rejectpotatoes0.5690.781578947Fail to Rejectcarbohydrates0.3840.564705882Fail to Rejectlegumes0.3410.5328125Fail to Rejectblue fish0.340.5328125Fail to Rejecteggs0.2750.491071429Fail to Rejectfruit0.2690.491071429Fail to Rejectred mead0.2510.491071429Fail to Rejectskimmed milk0.2220.491071429Fail to Rejectvegetables0.2160.491071429Fail to Rejectbutter0.2120.491071429Fail to Rejectwhite fish0.2050.491071429Fail to Rejectcereals and pasta0.0740.264285714Fail to Rejectnuts0.060.25Rejectprotiens0.0420.21Rejectwhite meat0.0410.21Rejectwhole milk0.0390.21Rejectolive oil0.0080.1Rejecttotal calories0.00000010.0000025Reject