Freshman Statistics Seminar
Week 8:Duct Tape and Warts: Issues in Experimental Design
The purpose of this section is to introduce basic topics in experimental design, especially positive and negative controls. Students should also be able to integrate knowledge from previous weeks, such as randomness and how it applies to choosing the subjects of the study, and perhaps an awareness of the importance of sample size and how it can affect statistics and a study.
Focht, D.R. III, Spicer, C., Fairchok, M.P. 2002. The efficacy of duct tape vs cryotherapy in the treatment of verruca vulgaris (the common wart). Arch Pediatr Adolesc Med. 156:971-974.
This is a prospective, randomized control trial in children with 2 treatments: cryotherapy (applying liquid nitrogen to warts) or duct tape occlusion. The authors state that duct tape therapy was significantly more effective than cryotherapy.
De Haen, M., Spigt, M.G., van Uden, C.J.T., van Neer, P., Feron, F.J.M.m Knottnerus, A. 2006. Efficacy of duct tape vs placebo in the treatment of verruca vulgarus (warts) in primary school children. Arch Pediatr Adolesc Med. 160:1121-1125.
This is a prospective, randomized, placebo-controlled control trial in children. They compared the application of duct tape to a corn pad in the resolution of warts. The authors found so significant difference between the two treatments, although duct tape-assigned warts showed a larger decrease in total size.
Suggested Lesson Structure:
Before class, students will read two papers addressing the effectiveness of duct tape on treating warts in people. Both papers are short and very clearly written. There is an introductory popular press article that could be assigned as well.
Part I. Discuss and Critique the Two Studies
Split the class up into two groups and give each two overheads with some markers. Assign one paper to each group to briefly describe using their overhead, which they will then present to the class. The first part of the assignment is to sketch out the experimental design. The second part is to write up a series of questions or criticisms of the study.
Part II. Identify Hypotheses and Design a Study to Test Them
This part of the discussion could work with the entire class, but splitting into two or three smaller groups might work best. The students should develop a hypothesis to test to settle the duct tape wart question once and for all. A careful reading of the introduction to de Haen et al. should give them a good idea of how to start. How will they design their study? What treatments will they include? What will their controls be? How will they choose the patients? How will they judge success with a treatment? Much of the points should flow directly from the critiques of the studies they have read and then discussed.
A few points to start the discussion:
- What is different about the two studies?
- Who were the patients are how were they chosen?
- Where they assigned to treatment the same way?
- What is the specific question that each study is addressing? Can you directly compare these studies?
- What is a control group? Why do you have one?
- How did the authors decide whether a wart had been successfully treated?
- What are some reasons why you may not find a difference when a difference does really exist?
As a segue from the first to the second half of the discussion:
What if I told you that a group of scientists have published work showing that hypnosis has a significant effect on the healing of warts? (such a study does exist) What would you need to see in such a study to become convinced?
Two additional published articles discuss the shortcomings of the experiments assigned for homework. These might also make good follow-up reading to be assigned to the students.
Ringold, S., Mendoza, J.A.m Tarini, B.A., & Sox, C. 2002. Is duct tape occlusion therapy as effective as cryotherapy for the treatment of the common wart? Arch Pediatr Adolesc Med. 156:975-978.
The main criticisms these authors bring up are:
- If the subjects did not report to the clinic, they were called and asked if their wart has disappeared. Problems with this are:
- The subjects were not blinded to the treatment (they could see the duct tape and knew if they’d undergone cryotherapy). This is a potential confound when they are self-reporting their results.
- More patients in the duct tape group than the cryotherapy group had to be called, thus the duct tape data relies more on self reports.
- There was no placebo group in the study.
- Nothing to compare the two groups to
- A large proportion of warts clear up on their own. Did the duct tape have actual efficacy?
- Many of the patients were lost before the follow-up. The sample size is small enough that the basic results could differ depending on the patterns on wart resolution in these patients.
Van Cleave, J., Kemper, A.R., Davis, M.M. 2006. Interpreting negative results from an underpowered clinical trial: warts and all. Arch Pediatr Adolesc Med. 160:1126-1129.
Their main criticisms:
- Patients electing to take part in the study may have warts especially resistant to resolution.
- e.g. The mean time the warts were present exceeded that of normal wart resolution without treatment.
- The children didn’t seem very randomly allocated: there were differences from the start between the two groups
- There was a longer wart duration for the duct tape group, and the control group had more patients who had tried another treatment before the study.
- This actually makes an interesting point about randomization: with small numbers, a random allocation could still result in these differences in mean. The authors of the critique don’t address this.
- The authors raise questions about how blind the study truly was.
- The observer reports of which patient had been assigned to which group were biased toward recognizing the duct tape group.
- It is a positive that the authors of the original paper both assessed and reported the efficacy of the blinding methods.
- Even patients not completing treatment were analyzed in the groups in which they were assigned.
- While this is a good test of “intent-to-treat,” and a good indication of efficacy in the real word (where patients don’t always follow instructions), an analysis of only the patients completing the assigned treatments may reveal differences between the treatments.
- How do you interpret negative results?
- With no difference between the treatments, statistical power becomes a big issue (especially when the possible outcomes are binary).
- The study lacked the sample size required for adequate statistical power.
- There were differences the change in size of the warts. Had the study been longer, this may have resulted in a difference in the primary outcome of wart resolution.