Hi! and welcome to What Works Wednesdays where historically a success story from clinical files is shared. With all the buzz about the latest “research” on getting the flu while pregnant and the supposed link to autism, it seems logical to help readers better understand research so they can interpret findings themselves. If readers know how to read research, then they are better able to know if an intervention works (or if the conclusions from a study are flawed or misinterpreted).
What is Research?
- “work undertaken systematically to increase the stock of knowledge” (Wikipedia.org)
- “diligent and systematic inquiry or investigation into a subject in order to discover or revise facts, theories, applications, etc.” (dictionary.com)
Most scientists conduct research by utilizing the scientific method. The scientific method requires the development of a hypothesis (which is usually formed from observation or reading other research), conducting the experiment, gathering results, and determining if the results support the original hypothesis.
Different Types of Research
Using the scientific method, scientists design different types of studies. These study types include:
- Experiments. In experimental studies, researchers recruit participants and assign them to treatment groups. Researchers can study one or more treatments and participants may receive some treatments or they may receive a placebo or no treatment at all. Usually, researchers measure one or more important variables before the study and they measure the variable(s) again after the study.
- Single Subject Experimental Studies. In these studies (most often conducted by behavior analysts), researchers recruit participants who are observed and measured carefully for a period of time before receiving treatment. Researchers then implement treatment while continuing to observe and measure carefully.
- Correlational studies. In these studies, researchers use existing data sets (e.g., collected for some other purpose) or they recruit participants. Researchers gather a wide range of information on each participant (e.g., age, SES, education, health history). Participants do not generally receive treatments or interventions of any kind.
- Qualitative studies. In qualitative studies, researchers occasional recruit participants but at times they enroll participants with whom they are already familiar. In qualitative studies, researchers study one or more individuals or one or more groups (e.g., one class). Researchers carefully study the participant and take copious notes. Researchers may interview the participants and they may use focus groups to better understand some of the issues. If a treatment is provided, the researcher continues to carefully study the participants to document the participants’ responses to the treatment.
Conclusions Based on Study Type
Researchers must use caution when drawing conclusions about their studies. Researchers who use well-designed experimental designs can draw cause-effect conclusions. For example, a researcher can enroll a bunch of smokers in a study. Some of the smokers receive a behavioral treatment, some of the smokers receive nicotine patches, and other participants receive both. At the end of the study (if the researchers have conducted the study carefully), the researchers will be able to say that one or more methods is successful at helping smokers quit.
Similarly, in a single subject experimental study, researchers can demonstrate if a treatment changes behavior. Again, the study must be carefully designed and conducted but it is possible to draw cause-effect conclusions. For example, a researcher could study 3 smokers. The researcher would observe the smokers and collect data. One smoker could receive treatment. While she is being studied, the other smokers would still be studied. After the first smoker quits successfully, the next smoker would receive treatment. He would continue to be studied as would the non-treated smoker. Finally, when the last smoker receives treatment, researchers continue to observe him. If the researchers successfully help all 3 participants quit smoking (and the study is carefully designed and carried out), they will be able to say that the treatment caused the behavior change.
Correlational versus Causal
Correlational studies are designed to determine if any relationships exist between variables. Researchers could gather data on 1,000 people from an existing data base. They could sort the data into smokers and non-smokers. They could run a simple data analysis to see if smokers have other tendencies (e.g., like to go to race car events, like to drink socially, and so forth). Researchers may not conclude causal relationships from their studies. They are only able to conclude that a relationship exists. Of more importance is the strength of the relationship. For example, if researchers ran an analysis on the relationship between giving birth to a child and gender, they would find a very strong (almost perfect) relationship between giving birth and being a female. If a weak relationship exists between variables it is more likely due to chance.
Go Forth and Read
In these days of social media, spin rooms, and media crazed talk shows, very poorly designed studies are being presented to the public without appropriate interpretation of the study or its results. If you are interested in reading a few examples of this, check previous posts here and here.
In summary, don’t believe everything you read about the “latest scientific study” unless you read the study itself. When you read the actual study, what you find may actually surprise you.