Most bad science reporting isn't fabricated. It starts with a real study, then loses its qualifications on the way to your feed — a cautious "associated with" becomes "causes," a mouse becomes a person, a maybe becomes a headline. A 2014 analysis in The BMJ traced much of this back to the press release itself: of the releases studied, 40% contained exaggerated advice, 33% overstated causal claims, and 36% inflated animal findings into claims about humans. The exaggeration was usually baked in before a journalist ever touched it.
You don't need a PhD to push back on that. The steps below are the same moves a science desk runs before it trusts a paper. They're ordered roughly from fastest to most involved — most claims fail one of the first three, and you can stop there.
The 7-step read
Find the study itself, not the article about it
Most "new study finds…" headlines are written from a press release, not the paper. Scroll to the bottom of the news article for a link, a DOI, or the journal name, then pull up the actual study on the publisher's site, PubMed, or Google Scholar. If a claim about research has no traceable paper behind it — no journal, no authors, no DOI — you are reading someone's summary of someone's summary, and the interesting caveats have usually been sanded off along the way.
Check whether it was peer-reviewed, and where
Peer review isn't a guarantee of truth, but its absence changes how much weight a single paper can carry. Preprints on servers like medRxiv or arXiv are useful early signals that have not yet been vetted — they say so at the top of the page. For published work, check that the journal is indexed in PubMed or listed in the Directory of Open Access Journals (DOAJ). Journals that spam authors, promise publication in days, and hide their peer-review process are the predatory-publishing pattern researchers have warned about for years.
Read past the abstract into the methods
The abstract is the marketing; the methods section is the product. It tells you what the researchers actually did: who they studied, for how long, what they measured, and what they compared it against. A randomized controlled trial supports stronger claims than an observational survey; a study in mice or cells supports far weaker claims about people than its headline usually implies. If the methods are vague or missing, treat the conclusions as a hypothesis, not a finding.
Separate correlation from causation
Two things moving together is not one thing causing the other. Observational studies can show that coffee drinkers live longer without showing that coffee is the reason — the coffee drinkers might also be wealthier, younger, or less likely to smoke. Those hidden factors are called confounders, and a careful paper names them and tries to adjust for them. When a headline turns an association into a cause ("X causes Y"), check whether the study itself used that word, or whether the press office added it.
Weigh the sample size and the effect size
Ask two numbers of every study: how many, and how much. A dramatic result from twelve participants is a lead worth following, not a settled fact. And a "statistically significant" finding can still be tiny — a 0.2% change in risk is significant with a big enough sample and meaningless in daily life. Look for the effect size and the confidence interval, not just the p-value. Research that reports only relative risk ("doubles your risk!") while hiding the absolute numbers is telling you the smaller number wasn't impressive.
Follow the funding and read the caveats
Reputable papers carry a conflict-of-interest and funding statement, usually near the end. Industry funding doesn't automatically invalidate a result, but it belongs in your assessment — especially when the sponsor sells the thing being studied. While you're there, read the limitations paragraph the authors wrote themselves. Scientists are often far more cautious about their own work than the headlines built on top of it, and that self-declared list of weaknesses is one of the most honest parts of any paper.
Check for retractions and replication
A single study is a data point, not a verdict. Before you treat one paper as proof, search the Retraction Watch Database to see whether it has been withdrawn, and PubPeer for post-publication comments from other scientists. Then ask the bigger question: has anyone reproduced it? A finding that a dozen independent teams have replicated is on much firmer ground than a striking one-off that no one has confirmed. Systematic reviews and meta-analyses, which pool many studies, beat any lone paper.
Quick red flags in the coverage
When you only have thirty seconds, these are the tells that a study is being oversold. Any one of them can have an innocent explanation; two or three together mean slow down.
- ✕The word "proves" — good science almost never proves; it supports, suggests, or is consistent with.
- ✕No link to the underlying paper anywhere in the coverage.
- ✕A cure, breakthrough, or "one weird trick" framing attached to a small or preliminary study.
- ✕Results from animals or cells reported as if they already apply to humans.
- ✕Relative risk with no absolute numbers ("50% higher" without saying higher than what).
- ✕A single study presented as overturning an established scientific consensus overnight.
One study is a starting point, not a conclusion
The hardest habit to build is treating a single striking paper as a lead to investigate rather than a fact to repeat. Science moves by accumulation — replications, reviews, the occasional retraction. The finding that survives a decade of other labs trying to knock it down is worth far more than the one that trended for a week. When a result matters to you, look for the meta-analysis or systematic review before you build a belief on the original.
What this guide is not
None of this tells you a specific study is right or wrong — that's a job for experts in the field, and often for time. What it gives you is calibration: a sense of how much confidence a claim has earned so far, and where the gaps are. Used honestly, that's usually enough to know whether something is ready to share or worth holding.
Check a claim in seconds
FAXTR searches 100+ fact-checking organizations in one query — free, no login required.