Freshman Statistics Seminar
Week 6:Sample Size
- Describe why larger sample sizes not only give better estimates of population means, but also give better estimates of variance and hence mean differences.
- Kolata 2002 NYT “Study Is Halted Over Rise Seen In Cancer Risk”
A menopause hormone replacement study (the Women’s Health Initiative) that was intended to run 8.5 years was halted after 5.5 years because the results showed that women taking the hormone replacement had a slightly but significantly greater risk of breast cancer, heart attack, stroke, and blood clots. Although this article doesn’t state it (though the LA Times article does), the study looked at nearly 17,000 women who were randomly assigned to treatment and placebo groups. The article probably goes into more detail than serves the purpose of this class, but the upshot is that the slight but real increased risk could only be seen with such a huge sample size!
- Gellene 2006 LA Times “Breast Cancer Rates Fall-A 7 pcnt Decrease Is Seen In The US in 2003 After Millions Cease Hormone Therapy Linked To Risks”
Breast cancer rates fell 7% after millions of women discontinued their hormone replacement therapy due the study mentioned above. This is a dramatic decline and shows that even a slight risk has a large effect when millions of people are involved.
Suggested Lesson Structure:
- Start with the active learning module to give students some intuition about data
- End by discussing the very real and life-saving results that were found in the Women’s Health Initiative.
- Most of the salient discussion points are mentioned in the active learning module. In addition, you may wish to discuss the reasons why researchers don’t always use large sample sizes. Most obviously, each sample effectively costs time/money, both of which are in finite supply. Sometimes sample sizes are minimized for ethical reasons, as when animals are used.
Active Learning Modules:
Most, if not all, of students’ pre-college experience conducting experiments makes no use of replication. Measured values on a single replicate or means of a small number of replicates are assumed to accurately reflect the population under study. The goal of this activity is to help students appreciate the necessity of replicates in most biological studies.
- In this activity, students are first given subsamples of a categorical dataset with distributions that do not overlap (categories A & B, see Table 1). The purpose of this first exercise is to expose their assumptions about measurements. Most students assume that the distributions of the populations that they are studying do not overlap. As a result, measurements made on one or a few replicates accurately capture the relationship between the categorical variables (see fabricated data, fig. 1).
|Figure 1: fabricated data with no overlap – any subsample will correctly reflect the fact that population B is greater than population A.|
- Each student or group of students is asked to draw inferences about the underlying populations based on their small subsample (you may wish to have them consider inferences based upon a single measurement of each category or upon the average of two measurements of each category or both). No matter the sample size and no matter which particular subsample they get, they will always draw the correct inference: members of Category B are greater than members of Category A (although they may draw incorrect inferences about the magnitude of that difference).
- Repeat step (1) with a dataset whose population distributions do overlap (see fabricated data, fig. 2, and categories B & C, table 1).
|Figure 2: fabricated data with overlap – subsamples will not always correctly reflect the fact that population C is greater than population B.|
- Again, students are asked to draw inferences about the underlying populations based on their small subsample (again using single measurements, averages, or both). This time, some students will make the correct inference, and some students will make incorrect inferences. The magnitude of the difference will vary widely.
- Ask students to speculate about what’s going on. At some point in the discussion, reveal the full datasets (figs. 1 & 2) or, better, have students plot their subsamples on a class graph on a chalk board or overhead. Ask them which situation they believe is more common in the biological sciences. Ask them to speculate about how a researcher would know that he or she had taken enough samples.
- These data are obviously fabricated, so it may be a useful discussion topic to brainstorm about the sorts of data that might look like figure 1 (no/little overlap) versus figure 2 (lots of overlap). Examples of no overlap might be: height of professional basketball players versus the height of professional jockeys; weight of seals versus the weight of otters; and so on. It’s likely that few of the examples that students come up with will be of real scientific interest (the differences are so apparent that one need not study them!). Examples of overlap might be: the effects of two drugs on blood pressure; the effects of two crop rotations on yield; and so on.