Bryant University Experimental Design Problem Essay – Description
Identify the problems with the following experimental designs and the conclusions derived from these experiments:
-Study #1: Sam wants to know if watching scary movies during pregnancy could increase the blood pressure of the mother. To answer this question, Sam recruits 20 pregnant women, have them watch a scary movie and then hands them a questionnaire asking them how they felt throughout the movie. Sam then analyzes the answers to the questionnaire to see if there was an effect from watching the scary movie.
-Study #2: Jose wants to know if taking an extra credit workshop increases student performance in biology courses. Jose asks for students to volunteer to take the workshop and he then compares the exam scores of students who completed the workshop with the scores of students who didn’t complete the workshop.
-Study #3: Mary wants to know if a new brand of fertilizer would be effective to help her plants grow faster. She has a large plot with 200 plants. To half of the plants she applies the fertilizer. When applying the fertilizer, she has to dig a small hole near each plant and then she covers it with the fertilizer and adds some water to even out the soil. To the other half of the plants don’t receive the fertilizer and she leaves them alone.
-Study #4: Chris wants to know if an herbal remedy protects people from the flu. They recruit 200 college students, half of the volunteers drink the herbal remedy and the other half drink something that tastes the same but doesn’t have the key ingredient. None of the volunteers nor the researchers directly interacting with the volunteers know which drink they got. Chris and their team follow the volunteers for a semester and track who gets the flu and who didn’t. At the end of the semester Chris finds that those who got the herbal remedy were 4 times less likely to get the flu. Chris concludes that this proves that the herbal remedy works and that it should be prescribed to elder people, who tend to be at higher risk of complications with the flu.
-Case #5: Watch this video https://youtu.be/0Rnq1NpHdm (warning: it may contain content that some viewers may find inappropriate but it gives a good illustration of some of the issues with making broad conclusions from scientific experiments). Identify examples in the video for each of the five (5) common problems with experimental design and conclusions.
Five (5) common problems with experimental design and conclusions.
1) Poor methodological approach
The design must include a well-developed and transparent plan for how you intend to collect or generate data and how it will be analyzed. Ensure that the method used to gather information for analysis is aligned with the topic of inquiry and the underlying research questions to be addressed. You must select the appropriate controls.
2) Inappropriate study design for the study aims
The methods and the data collected in the experiment doesn’t directly answer the main question of the study. This can happen by at least two different reasons:
Sample selection: Let’s say you want to test the effectiveness of a new seat belt design in saving lives. But all your dummy test models are modeled after adult male anatomies. Your results cannot be extrapolated to females or to children. You can solve this be either acknowledging that your results will be limited to male individuals, on including all possible model types in your study.
Variable selection: In the example above, let’s say you are measuring the amount of exterior damage to the dummy. That could indicate possible injury for actual human subjects but it may ignore other ways of damage that may be invisible, such as a concussion.
Confounding variables: this happens when multiple things are different between your study subjects so that you cannot attribute any changes to one particular variable. For example, comparing the average weight of people in the US to people in the Mediterranean and attributing the differences to having a Mediterranean diet. There could be other factors causing this difference, people in Mediterranean have lower car ownership than in the US, so maybe they walk more and are overall more active.
3) Controlling for bias
Bias control is what distinguishes good from bad research and measures to control for bias include: randomization of subjects to the areas, interventions and control conditions: measurement and analysis of subjects with the investigators blind to the subject status; and having a credible control condition and verifying at the onset and along the way that the subject is truly blind to the group to which they were assigned.
Example: Let’s say you want to poll people on their opinion about a certain topic, you randomly call people from a list of landline numbers. Although your sample is random, you are biased towards the segment of the population who still use landlines, which tends to be older than the overall population. A similar issue happens when subjects self-select into study treatments, those who already have favorable opinions about the treatment in the study will be more likely to sign up.
4) Over-interpretation of results
All studies have limitations. You need to be able to identify how far you can extrapolate the results of the study. For example, in the study with male dummy car passengers, your study does not give us any information about how this seat belt would work in females or children. Thus, it would be incorrect to state any general conclusions about these populations. Even if the study was properly designed, an individual study is still limited and cannot be used to make broad conclusions.
5) Lack of replication
It is impossible to “prove” something in science. There is always the possibility for an alternative explanation that we are not aware of. However, the confidence in a finding increases as more and more studies conclude similar results. Replication is one of the key ways to build confidence in the scientific merit of results. When the result from one study is found to be consistent by another study, it is more likely to represent a reliable claim to new knowledge. Thus, while one study cannot “prove” something, if there are several studies from different labs showing similar results under different conditions, you can increase your confidence that the results are real.
The post Bryant University Experimental Design Problem Essay first appeared on .
