Introduction to Experimental Design

Motivation

Whether for just a summer or for an entire Ph.D, working on a scientific problem is a process of trying to explain and predict the way that the natural world operates. You will typically need to use or develop new ways of measuring something about the world or perform experiments to test a hypothesis. Because the amount of time that you have to reach an answer is finite, the whole process is a bit like a game of twenty questions. There are only so many experiments that you can do before the end of summer or before it's time to graduate. Use them wisely. The more effectively that you learn how to pose the questions (and debug protocols) the more progress you will make toward scientific discovery. This page is meant to provide vocabulary, ideas, and questions related to experimental design.

Types of Experiments

1. Observational Experiments
   • Description – "What is found?"
   • Appropriate when you have no prior information about the ream of possible outcomes.
   • Example: microbial evolution experiments, in vitro selection.
2. Manipulative Experiments
   • Explanation – "Why and how is it found?"
   • Appropriate when you are trying to determine the factors which cause a certain outcome.
   • Use a mathematical model or statistical framework to enumerate all of the possible expected outcomes.

Controls

• Is my assay working?
  • Sanity check: Are the values I'm getting realistic? Do they agree with what is known from the literature?
  • Positive control: Do you have a strain or sample that should give an expected result in the assay?
• Am I measuring what I think I'm measuring?

Statistical Design of Experiments

To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of.
–Ronald Fisher, Presidential Address to the First Indian Statistical Congress, 1938.

A sample is the subset of a population that is examined. Generally one uses measurements of the sample to infer the properties of the population.

There will always be variation in measuring a characteristic of the natural world due to:

• Experimental error - unavoidable imprecision in measurements of the same sample.
• Biological variation - differences between individuals (cultures, strains) making up the sample which are nominally the same.
• Variation in space and time - variation in measurements due to unplanned changes in environmental conditions (Ex: performing measurements on different days).
• Sampling error - uncertainty in making inferences about the population because of chance variation in the makeup of the sample that was actually tested.

• Keep the statistical analysis as simple as possible. The more complex it is, the more difficult it will be to explain to others exactly what you did, the more likely you might make mistakes in calculations, and.
• Test one factor at a time. If possible change only one variable at a time.

• Replication
• Randomization
  • Examples: You always put treatment 1 flasks on the left side of the incubator and plate them first. You find that these
  • Solution: When in doubt, use a fully randomized treatment.

Further Information

• http://en.wikipedia.org/wiki/Design_of_experiments

References

1. Heath. Introduction to Experimental Design. Chapter 1: Why biologists need sampling, experimental design, and statistics.
2. Montgomery. Design and Analysis of Experiments

Internal page with more information...