English Banner
Study Designs   To Epidemiology theme page
To EBM theme page

1. Core Knowledge:

The evidence for evidence-based medicine is all collected via research, which uses a variety of study designs.  You will be learning about "critical appraisal of the literature," and judging the quality of a study design is a central part of this.

Different study designs provide information of different quality.  Of course, we always try to use the best possible design, but sometimes this is not practical or ethically acceptable (you cannot do an experiment to expose some people to a harmful substance to see what effect it has).  Therefore, you need to understand the strengths and limitations of each type of study design, as applied to a particular research purpose.  The purposes we will consider include (1) describing the prevalence of health problems; (2) identifying causes of health problems (etiological research), and (3) evaluating therapy, including treatment and prevention. 

Types of Study Design

First, distinguish between observational and experimental studies. 

Observational versus experimental studies


In observational studies, the researcher observes and systematically collects information, but does not try to change the people (or animals, or reagents) being observed.  In an experiment, by contrast,  the researcher intervenes to change something (e.g., gives some patients a drug) and then observes what happens.  In an observational study there is no intervention. 

Examples of observational studies:

Examples of experiments:

When do you do an observational study?

Observational Designs:

What types of observational study are there?  Lots, but you need to know about three main ones:

3 types of observational design: surveys, case-control and cohort studies

Cross-sectional surveys.  Example: what is the prevalence of diabetes in this community?  Here, you draw a random sample of people and record information about their health in a systematic manner.  You can also compare people with, and without, diabetes in terms of characteristics (such as being overweight) that may be associated with the disease.  The problem is that you cannot be sure which came first: the diabetes or the weight problem, so this is a very weak design for drawing conclusions about causes. 

  • Cohort, or "longitudinal", or "prospective"  studies.  These are like surveys, but extend over time.  This allows you to study changes and to establish the time-sequence in which things occur.  Therefore, you can use this to study causes.  For example, you could draw a sample of people (medical students, for example) who do not have the disease you are interested in, and collect information on the factor you have hypothesized to be a cause of the disease.  Maybe you want to see whether using a cell phone leads to brain cancer.  So, collect information on how many minutes each student uses their phone each week (you might get permission to obtain this from their phone company bills), and collect this information over a long time, and then eventually collect information on who gets brain cancer.  You could then see whether the cases of brain cancer arose among the people who used their cell phones most often.  In technical terms, you record the incidence of cancer among those who use their phones more than a pre-determined amount and compare this to the incidence in the non-users.  You could calculate the relative risk
    The advantages of this study design are that it can establish that the phone usage predates the cancer, and it allows for accurate collection of exposure information ('exposure' = their use of the phones). However, there are some problems with this design.  Brain cancer is rare, so you will need a very large cohort of students; you will also need to keep in contact with them for a very long time and you will probably get very bored waiting for the results.  We need a quicker solution.
    Link to ppt diagram of a cohort study

    Self-test question: cohort study
      Can you can estimate prevalence from a cohort study?


    You answered 'Yes', and in general this is not correct.

    Where a cohort study is designed to identify causal factors for a disease, you would begin by selecting a sample of people who do not have the disease (so, prevalence = zero). You would then follow them over time. Some incident cases would arise, and these would provide you with an estimate of prevalence. However, because you omitted the existing cases at the beginning of the study, your prevalence estimate would be biased: it would be too low. The only exception would be if you followed people for a long time, and if people die from the disease quickly, so none of the original cases would have survived anyway.


    Well done, this is correct. But there is a bit of a trick...

    You presumably recognize that a cohort study omits existing (prevalent) cases at the beginning, so the only estimate of prevalence will come from the new cases that accumulate. This will give you a low estimate of prevalence, unless you follow them for so long that all the prevalent cases you excluded will have died anyway. So, if the disease kills quickly and if you follow the cohort for a long time, you could get a good estimate of prevalence.


    Self-test question: case-control study

    It says that you cannot estimate prevalence from a case-control study.

    Why not?

    Click for an answer

    You cannot estimate prevalence from a case-control study because the prevalence depends entirely on how you design your study.

    For example, you might choose (say) 50 cases and 50 controls. Prevalence? 50%. Or, if you believe there will be considerable variability among the controls (they may have a range of other medical conditions, but not the disease you're interested in), you could choose more controls (say 100) to your 50 cases. So, what's the prevalence? 50 / 150 = 33%. So, the apparent prevalence depends entirely on the way you set the study up: it is arbitrary.

    Experimental Designs:

    experimental studies: RCTs and quasi-experiments

    Randomized Controlled Trial ("RCT"), aka "Randomized Clinical Trial".  The mainstay of experimental medical studies, normally used in testing new drugs and treatments. 

    RCT diagram

    Self-test question: RCT
      True or False?

    A randomized controlled trial begins with a random (i.e. representative) sample from the population of interest.


    Sorry, this is not correct.

    The "random" in RCT refers not to the way the sample is drawn, but to the allocation of people to either experimental or control group. It's not a random sample, but a random allocation to either experimental or control group.

    The actual sample used in the experiment may be selected in any way the experimenter chooses, and may not be representative at all – although that will limit the value of the study as the results may not be generalizable to other groups.


    Well done!

    The "random" in RCT refers not to the way the sample is drawn, but to the allocation of people to either experimental or control group. In fact, the sample may not be randomly selected at all, and hence quite unrepresentative, although that will limit the value of the study as the results may not be generalizable to other groups.

    2. Nice to Know:

    Quasi-Experimental Designs:

    Finally, there is a category of studies that falls between observational and true experimental studies; they are called "quasi-experimental studies".  In these, there is an intervention, but it is often not completely planned by the person doing the research.  An example would be a study of the effects of removing ophthalmic services from the OHIP billing schedule: is there a decrease in eye tests after the change?  A quasi-experimental study might record the number of eye exams per thousand population over the years up to the policy change, and compare this pattern with the pattern afterwards.  This is an observational study, but there was also an intervention, although it was not the experimenter who decided when and how the change would occur and to whom it would be applied, so this is a "quasi-experiment." Typically, random allocation is not involved. A "natural experiment" is similar, but refers to naturally occurring events (e.g., a study of mental health following an earthquake).

    Summary of designs, showing advantages and disadvantages of each:
    Disadvantages and strengths of each type of study design

    Links.  A History of Controlled Trials is contained on the Royal College of Physicians of Edinburgh web site.  It contains reproductions of old medical texts discussing scurvy, the Pox and other delights best avoided.  The British Medical Journal dedicated an issue to the RCT in 1998.
    Statistical power in study designs
    Patient Safety and Medical Errors
    Here is a web site that contains guidelines for critically appraising the quality of studies

    Other links: A primer on Multivariable Analysis for readers of medical articles
    Cancer Research UK review of study designs