This post is largely cannibalized from something I posted on a message board, but I think it fits well here, too.
When you read about scientific studies in the newspaper, it can be hard to assess their merit. Here are a few questions to keep in mind for critical reading of science 101:
1. Where was the study published? Was it in a peer-reviewed journal, or was it just presented at a conference? (Findings presented at conferences are not subjected to peer review). In the biological sciences, journals like Science, Nature, and Cell are considered "top-tier." Medical journals, like JAMA, The Lancet, and The New England Journal of Medicine, are considered slightly more suspect, but still respectable. If you don't know the journals, it can be hard to judge, but if it sounds obscure, it probably is. One caveat to this method is that big-name people can sometimes get shitty papers into good journals on the strength of their reputations.
2. Who sponsored the study? Peer-reviewed journals require authors to disclose who financed the research, so this is easier to find out than a lot of people think. Any time a drug company publishes a study showing that its new drug cures cancer, baldness and ennui, I would reserve judgement until the results are verified by an outside party.
3. Are the results of the paper supported by other work in the field pointing in the same direction? That information is usually found in the Introduction section of the paper, where researchers discuss the foundation for the work to be presented. This is particularly important for anything you read about in the mainstream media, as they tend to have a "sensationalism bias" in what gets reported, i.e. if a study's results are unexpected, it's more likely to make the papers.
4. Did the authors draw the appropriate conclusions from the data presented? Does their explanation make sense? Does it account for all of their findings? Is there an alternative explanation, and if so, do they consider it in the Discussion section of the paper? Interpretation of the findings is the most subjective aspect of a scientific paper, but it's also the thing people are most likely to take away from reading it.
Subscribe to:
Post Comments (Atom)
4 comments:
I think this is a great summary of how to critically review an article. I hope it makes people realize the inherent biases and difficulties in interpreting research.
My only worry is that the average joe doesn't have the statistical and research skills to do what you recommend. I have a PhD and I don't feel comfortable evaluating research that is in areas outside my general area of expertise.
Great blog though-I'm so glad Big Fat Deal linked to it. I think you're doing a great job interpreting some of the research for those of us not so experienced in this area! Thanks!!
Thanks, psychsarah! I think you're right that I am kind of glossing over the fact that it can be difficult to critique studies when you don't have much background in the field or even much scientific background at all.
But I also think that science can be unnecessarily daunting for a lot of people, and you don't always need special training to be able to see that a story doesn't add up the way the authors say it does.
I think this summary of how to critically review an article is awesome, as psychsarah said. It's also helpful for those of us just starting out in academic careers who need to know what points to hit in order to be taken seriously.
I was diagnosed as HEPATITIS B carrier in 2013 with fibrosis of the
liver already present. I started on antiviral medications which
reduced the viral load initially. After a couple of years the virus
became resistant. I started on HEPATITIS B Herbal treatment from
ULTIMATE LIFE CLINIC (www.ultimatelifeclinic.com) in March, 2020. Their
treatment totally reversed the virus. I did another blood test after
the 6 months long treatment and tested negative to the virus. Amazing
treatment! This treatment is a breakthrough for all HBV carriers.
Post a Comment