How to Read a Scientific Article
Other helpful online primers:
Epidemiology for the Uninitiated
(note: these links will take you to the British
Medical Journal Web site. To
return here, use your browser's Back button)
Reviewing research articles and interpreting the literature
by Bob Millikan, DVM, PhD
Department of Epidemiology, School of Public Health
University of North Carolina at Chapel Hill
Getting Started
The first thing I do is to identify an area I am interested in. How good is neoadjuvant therapy? Why do children get asthma? Start with an idea. Identify some question that you really want to know the answer to.
Here is the idea for today:
Do stressful life events cause breast cancer?
Before you start reading, try to get an overview of the research area. Get hold of a review article and do some background reading. Here is where I started. I found this in a review:
"Several recent studies have examined the association of adverse life events and risk of breast cancer. The results have been inconsistent. Chen et al. (1995) reported a strong positive association between severe life events and breast cancer. However, Protheroe et al. (1999) reported no association between stressful life events and personal health difficulties and breast cancer risk. Other studies have also shown no association between stressful life events and breast cancer risk (Roberts et al., 1996; McGee, 1999). All of these studies have serious shortcomings."
Then I read a little bit about stressful life events. I found out that the perception of whether an event is stressful or not is influenced by personality, one's ability to cope, and social support.
So, I thought to myself: I need to go read these articles. Maybe I can design a study to do better, and find out the answer. So, I started with the Protheroe article, and this is what we will talk about today. I will use it as an example of how to approach a research article.
Example: (click on title to see the full article)
Reviewing the article the usual way
Here is what we teach at Project LEAD and UNC. Conduct a systematic reading of the article, addressing the following points.
1. Purpose
a. What were the objectives in performing the study?
b. What were the research questions being asked?
2. Study Design
a. What type of study was performed?
b. How was the study population sampled?
c. Are there potential sources of selection bias?
d. If a clinical trial, was there a control group?
e. If a clinical trial, was randomization adequate?
3. Measurement and Observation
a. Are there clear definitions of the terms used?
b. What outcome criteria were used?
c. What measurements were taken?
d. How?
e. What methods were used to assess validity or reliability of the measurements?
4. Results
a. How are the data presented?
b. Is the data relevant to the study question?
c. Are there data that were not presented?
5. Conclusions
a. What were the main conclusions?
b. Is the study significant?
c. Is the study relevant to other populations?
d. What questions remain unresolved?
It's helpful to work through these questions. But that is NOT what
I do in real life, at least not at first. Here is what I do first.
Reading an article the fun way
I once took an art history course where the instructor told us that when we look at a painting, the first thing to do is look at the four corners of the picture. That way you don't miss anything. Then look at the center and find what catches your eye, then the foreground and background. Then ask yourself, what is going on here? That is how I look at a research article.
Look at page one of the Protheroe article. Look at the lower left corner. That tells you what journal the article was published in. It's a good journal, but not great. Not a top tier journal. Look for the funding source next, it is usually in one of the lower corners. If not, find it. It is important, since there might be competing interests you need to know about before you even read the article. In this article, look on the last page near the end. It was funded by trusts and local money, not a peer reviewed grant. That says a lot. Peer reviewed grants tend to be the gold standard. No competing interests are declared. Hmmm…..
The other things you find in the four corners on page one are the authors' names and where they are from. In this article, the information is along the right hand border. The authors are all non-epidemiologists. Red lights went on when I read that.
The abstract is in the upper left of page one of every paper. It gives you an overview so you know what to expect. Sort of like listening to the taped introduction on the head phones in an art gallery. But I never start with this corner, or I will forget the other corners. I also try not to let the abstract bias me too much. Just like in the art gallery. I want to form my own opinions. Especially when the abstract says, "Our results show conclusively that…" I think, we'll see about that.
There are a few other things you will notice with the four corners rule. The lower right hand corner of page one tells you some extra information on sample size is on the BMJ website. Check this out. There is a reason it is on the website and not in the article.
Now we are ready to start reading the paper. But first I go to the center. Before I even read a word of the introduction, I look at the "Table 1" that must be in every paper. Look at Table 1. It compares cases and controls for traditional breast cancer risk factors. There is something weird going on here. Very few of the traditional risk factors show any association with breast cancer. Take family history: 15% of cases and 16% of controls have a family history of breast cancer. The P-value is 0.997, no statistically significant difference between cases and controls for family history. Wow. Usually, about twice as many cases have a family history as controls, and the controls are only about 5%, not 16%. A lot of controls in this study have a family history of breast cancer. And I also see that three times as many controls had benign breast lumps than cases. So, next I look at Table 2. Here are odds ratios for the traditional risk factors. Again, this looks weird. History of benign breast lumps is inversely associated with breast cancer (the odds ratio [OR] is less than one). That means a lot more controls had benign breast conditions than cases. Alcohol shows an inverse association (again, the odds ratio is less than one). That means more controls drink than cases. So, before I even read the article, I know something is probably messed up in the study population, perhaps the method of selection of participants was inappropriate for the research question.
I skipped the rates graph in the upper left of page 1029 because without reading the article, I was not sure what it meant. But Table 3 was interesting. They found that "2 year personal health difficulty" had an odds ratio of 2.72, a fairly strong association. But wait. They said in the conclusion of the Abstract that nothing was going on. Then I realized that they were using "statistical significance" as their criterion, rather than "public health significance." The OR of 2.72 has a confidence interval of 0.68 to 10.9 (interval includes 1.0) and a P-value of 0.16, not "statistically significant." It is easy to fall into the trap of dismissing any result with a P-value greater than 0.05, regardless of the direction or magnitude of the association. In observational studies, that is wrong. An association could be totally biased and have a P value less than 0.05, so the P value rule does not make the result "right." An association could have P value greater than 0.05, but if the association is fairly strong, it might merit further study, perhaps in a bigger study, which is what I thought when I saw OR of 2.72 and N (number of study participants) = 56. That's tiny. You will never get statistical significance in such a tiny study.
So, before I even read this article I knew:
- It was probably not a top-of-the-line study, and may have been funded by someone who had an agenda.
- There was the potential for flaws in the study design.
- The authors interpreted the results in a different way than I would.
Opinionated? Sure, but everyone has a right to their opinion. Art is art.
Unfair? Of course. I needed to actually read the article.
Reading the article in depth
Next, I read the article from stem to stern. Every word. Reviewed every figure.
To make a long story short, my opinions were not much changed. This study was way too small.
They only asked about life events over the past 5 years. And the controls were totally inappropriate: they had benign breast conditions and a family history of breast cancer. So they could have been "worriers." No wonder they did not find differences between cases and controls for perceived stressful life events. But they did find a positive for health difficulties, which is a little less subjective, and they totally dismissed it.
There are a few other strange things. In Table 2, alcohol is inversely associated with breast cancer (OR less than 1.0) and in Table 3, after multivariable adjustment, alcohol is positively associated with breast cancer. This is not good. To me, it means they did not disentangle the effects of alcohol and stress well enough.
I did find a few things I liked. The psychological assessment was really well designed, validated and administered with care. This part really set a high standard for the next study. Unfortunately, my opinion is that they performed the assessment on the wrong people.
Originally presented to the Florida Breast Cancer Coalition, January 15, 2000.