|Study Designs||To Epidemiology theme page
To EBM theme page
1. Core Knowledge:
The evidence for evidence-based medicine is all collected via research, which uses a variety of study designs. You will be learning about "critical appraisal of the literature," and judging the quality of a study design is a central part of this.
Different study designs provide information of different quality. Of course, we always try to use the best possible design, but sometimes this is not practical or ethically acceptable (you cannot do an experiment to expose some people to a harmful substance to see what effect it has). Therefore, you need to understand the strengths and limitations of each type of study design, as applied to a particular research purpose. The purposes we will consider include (1) describing the prevalence of health problems; (2) identifying causes of health problems (etiological research), and (3) evaluating therapy, including treatment and prevention.
Types of Study Design
First, distinguish between observational and experimental studies.
In observational studies, the researcher observes and systematically collects information, but does not try to change the people (or animals, or reagents) being observed. In an experiment, by contrast, the researcher intervenes to change something (e.g., gives some patients a drug) and then observes what happens. In an observational study there is no intervention.
Examples of observational studies:
a survey of drinking habits among students;
a researcher who joins a biker gang to study their lifestyle (note, as long as the researcher does not try to change their behavior, it's an observational study);
taking blood samples to measure blood alcohol levels during Monday morning lectures (yes, you are intervening to take the blood, but you are not trying to change the blood alcohol level: it's just a measurement).
Examples of experiments:
plying a law student with beer to see whether lawyers argue better when drunk;
encouraging bikers in one group to stop smoking those funny-looking cigarettes to see whether they get less belligerent;
warning one group of students that you are going to take blood alcohol levels next Monday to test for alcohol, and comparing their levels to another group that you did not warn.
When do you do an observational study?
When you merely want to collect descriptive information: "Is the incidence of diabetes rising?"
When you want to report on the causes of a problem without disturbing the natural setting (I want to find out why students do not attend lectures)
When you can't do an experiment: "How fast does the earth move around the sun?"
When it's not acceptable to do an experiment: "How much does not wearing a condom increase the likelihood of HIV infection?"
What types of observational study are there? Lots, but you need to know about three main ones:
Cross-sectional surveys. Example: what is the prevalence of diabetes in this community? Here, you draw a random sample of people and record information about their health in a systematic manner. You can also compare people with, and without, diabetes in terms of characteristics (such as being overweight) that may be associated with the disease. The problem is that you cannot be sure which came first: the diabetes or the weight problem, so this is a very weak design for drawing conclusions about causes.
Cohort, or "longitudinal", or "prospective" studies. These are like surveys, but extend over time. This allows you to study changes and to establish the time-sequence in which things occur. Therefore, you can use this to study causes. For example, you could draw a sample of people (medical students, for example) who do not have the disease you are interested in, and collect information on the factor you have hypothesized to be a cause of the disease. Maybe you want to see whether using a cell phone leads to brain cancer. So, collect information on how many minutes each student uses their phone each week (you might get permission to obtain this from their phone company bills), and collect this information over a long time, and then eventually collect information on who gets brain cancer. You could then see whether the cases of brain cancer arose among the people who used their cell phones most often. In technical terms, you record the incidence of cancer among those who use their phones more than a pre-determined amount and compare this to the incidence in the non-users. You could calculate the relative risk.
The advantages of this study design are that it can establish that the phone usage predates the cancer, and it allows for accurate collection of exposure information ('exposure' = their use of the phones). However, there are some problems with this design. Brain cancer is rare, so you will need a very large cohort of students; you will also need to keep in contact with them for a very long time and you will probably get very bored waiting for the results. We need a quicker solution.
Link to ppt diagram of a cohort study
Self-test question: cohort study Can you can estimate prevalence from a cohort study?
You answered 'Yes', and in general this is not correct.
Where a cohort study is designed to identify causal factors for a disease, you would begin by selecting a sample of people who do not have the disease (so, prevalence = zero). You would then follow them over time. Some incident cases would arise, and these would provide you with an estimate of prevalence. However, because you omitted the existing cases at the beginning of the study, your prevalence estimate would be biased: it would be too low. The only exception would be if you followed people for a long time, and if people die from the disease quickly, so none of the original cases would have survived anyway.
Well done, this is correct. But there is a bit of a trick...
You presumably recognize that a cohort study omits existing (prevalent) cases at the beginning, so the only estimate of prevalence will come from the new cases that accumulate. This will give you a low estimate of prevalence, unless you follow them for so long that all the prevalent cases you excluded will have died anyway. So, if the disease kills quickly and if you follow the cohort for a long time, you could get a good estimate of prevalence.
Self-test question: case-control study
It says that you cannot estimate prevalence from a case-control study.
Click for an answer
You cannot estimate prevalence from a case-control study because the prevalence depends entirely on how you design your study.
For example, you might choose (say) 50 cases and 50 controls. Prevalence? 50%. Or, if you believe there will be considerable variability among the controls (they may have a range of other medical conditions, but not the disease you're interested in), you could choose more controls (say 100) to your 50 cases. So, what's the prevalence? 50 / 150 = 33%. So, the apparent prevalence depends entirely on the way you set the study up: it is arbitrary.
Randomized Controlled Trial ("RCT"), aka "Randomized Clinical Trial". The mainstay of experimental medical studies, normally used in testing new drugs and treatments.
|Self-test question: RCT|
|True or False?
A randomized controlled trial begins with a random (i.e. representative) sample from the population of interest.
Sorry, this is not correct.
The "random" in RCT refers not to the way the sample is drawn, but to the allocation of people to either experimental or control group. It's not a random sample, but a random allocation to either experimental or control group.
The actual sample used in the experiment may be selected in any way the experimenter chooses, and may not be representative at all – although that will limit the value of the study as the results may not be generalizable to other groups.
The "random" in RCT refers not to the way the sample is drawn, but to the allocation of people to either experimental or control group. In fact, the sample may not be randomly selected at all, and hence quite unrepresentative, although that will limit the value of the study as the results may not be generalizable to other groups.
2. Nice to Know:
Finally, there is a category of studies that falls between observational and true experimental studies; they are called "quasi-experimental studies". In these, there is an intervention, but it is often not completely planned by the person doing the research. An example would be a study of the effects of removing ophthalmic services from the OHIP billing schedule: is there a decrease in eye tests after the change? A quasi-experimental study might record the number of eye exams per thousand population over the years up to the policy change, and compare this pattern with the pattern afterwards. This is an observational study, but there was also an intervention, although it was not the experimenter who decided when and how the change would occur and to whom it would be applied, so this is a "quasi-experiment." Typically, random allocation is not involved. A "natural experiment" is similar, but refers to naturally occurring events (e.g., a study of mental health following an earthquake).
|Summary of designs, showing advantages and disadvantages of each:|
Links. A History of Controlled Trials is contained on the Royal College of Physicians of Edinburgh web site. It contains reproductions of old medical texts discussing scurvy, the Pox and other delights best avoided. The British Medical Journal dedicated an issue to the RCT in 1998.
Statistical power in study designs
Patient Safety and Medical Errors
Here is a web site that contains guidelines for critically appraising the quality of studies
Other links: A primer on Multivariable
Analysis for readers of medical articles
Cancer Research UK review of study designs