I’m a professor of chemistry, have a Ph.D. and conduct my own scientific research, yet when consuming media, even I frequently need to ask myself: “Is this science or is it fiction?”
There are plenty of reasons a science story might not be sound. Quacks and charlatans take advantage of the complexity of science, some content providers can’t tell bad science from good and some politicians peddle fake science to support their positions.
If the science sounds too good to be true or too wacky to be real, or very conveniently supports a contentious cause, then you might want to check its veracity.
Here are six tips to help you detect fake science.
Tip 1: Seek the peer review seal of approval
Scientists rely on journal papers to share their scientific results. They let the world see what research has been done, and how.
Once researchers are confident of their results, they write up a manuscript and send it to a journal. Editors forward the submitted manuscripts to at least two external referees who have expertise in the topic. These reviewers can suggest the manuscript be rejected, published as is, or sent back to the scientists for more experiments. That process is called “peer review.”
Research published in peer-reviewed journals has undergone rigorous quality control by experts. Each year, about 2,800 peer-reviewed journals publish roughly 1.8 million scientific papers. The body of scientific knowledge is constantly evolving and updating, but you can trust that the science these journals describe is sound. Retraction policies help correct the record if mistakes are discovered post-publication.
Peer review takes months. To get the word out faster, scientists sometimes post research papers on what’s called a preprint server. These often have “RXiv” – pronounced “archive” – in their name: MedRXiv, BioRXiv and so on. These articles have not been peer-reviewed and so are not validated by other scientists. Preprints provide an opportunity for other scientists to evaluate and use the research as building blocks in their own work sooner.
How long has this work been on the preprint server? If it’s been months and it hasn’t yet been published in the peer-reviewed literature, be very skeptical. Are the scientists who submitted the preprint from a reputable institution? During the COVID-19 crisis, with researchers scrambling to understand a dangerous new virus and rushing to develop lifesaving treatments, preprint servers have been littered with immature and unproven science. Fastidious research standards have been sacrificed for speed.
A last warning: Be on the alert for research published in what are called predatory journals. They don’t peer-review manuscripts, and they charge authors a fee to publish. Papers from any of the thousands of known predatory journals should be treated with strong skepticism.
[Read more: When covering conspiracies and disinformation, avoid adding credibility]
Tip 2: Look for your own blind spots
Beware of biases in your own thinking that might predispose you to fall for a particular piece of fake science news.
People give their own memories and experiences more credence than they deserve, making it hard to accept new ideas and theories. Psychologists call this quirk the availability bias. It’s a useful built-in shortcut when you need to make quick decisions and don’t have time to critically analyze lots of data, but it messes with your fact-checking skills.
In the fight for attention, sensational statements beat out unexciting, but more probable, facts. The tendency to overestimate the likelihood of vivid occurrences is called the salience bias. It leads people to mistakenly believe overhyped findings and trust confident politicians in place of cautious scientists.
A confirmation bias can be at work as well. People tend to give credence to news that fits their existing beliefs. This tendency helps climate change denialists and anti-vaccine advocates believe in their causes in spite of the scientific consensus against them.
Purveyors of fake news know the weaknesses of human minds and try to take advantage of these natural biases. Training can help you recognize and overcome your own cognitive biases.
Tip 3: Correlation is not causation
Just because you can see a relationship between two things doesn’t necessarily mean that one causes the other.
Even if surveys find that people who live longer drink more red wine, it doesn’t mean a daily glug will extend your life span. It could just be that red-wine drinkers are wealthier and have better health care, for instance. Look out for this error in nutrition news.
Tip 4: Who were the study’s subjects?
If a study used human subjects, check to see whether it was placebo-controlled. That means some participants are randomly assigned to get the treatment – like a new vaccine – and others get a fake version that they believe is real, the placebo. That way researchers can tell whether any effect they see is from the drug being tested.
The best trials are also double blind: To remove any bias or preconceived ideas, neither the researchers nor the volunteers know who is getting the active medication or the placebo.
The size of the trial is important too. When more patients are enrolled, researchers can identify safety issues and beneficial effects sooner, and any differences between subgroups are more obvious. Clinical trials can have thousands of subjects, but some scientific studies involving people are much smaller; they should address how they’ve achieved the statistical confidence they claim to have.
Check that any health research was actually done on people. Just because a certain drug works in rats or mice does not mean it will work for you.
[Read more: Journalism collective fact-checks religious news in Brazil]
Tip 5: Science doesn’t need ‘sides’
Although a political debate requires two opposing sides, a scientific consensus does not. When the media interpret objectivity to mean equal time, it undermines science.
Tip 6: Clear, honest reporting might not be the goal
To get their audience’s attention, morning shows and talk shows need something exciting and new; accuracy may be less of a priority. Many science journalists are doing their best to accurately cover new research and discoveries, but plenty of science media are better classified as entertaining rather than educational. Dr. Oz, Dr. Phil and Dr. Drew should not be your go-to medical sources.
Beware of medical products and procedures that sound too good to be true. Be skeptical of testimonials. Think about the key players’ motivations and who stands to make a buck.
If you’re still suspicious of something in the media, make sure the news being reported reflects what the research actually found by reading the journal article itself.
Marc Zimmer is a Professor of Chemistry at Connecticut College.
This article is republished from The Conversation under a Creative Commons license. Read the original article. Sign up for The Conversation’s newsletter.
Main image CC-licensed by Unsplash via Hans Reniers.