6 Simple Questions Everyone Can (And Should) Ask About Data

A data fabrication scandal has rocked the media and academic worlds. In December 2014, many media outlets reported on a seemingly groundbreaking study that purported to show how conversations can change opinions about same-sex marriage. Fast forward six months, and there are allegations that the data used in the study were faked. Stories like this can lead to public mistrust of data and science, especially for the many people who don’t use data or statistics in everyday life. How are people supposed to know what data to trust?

Here’s the good news: You can determine whether to trust the data yourself, without relying on experts with often-differing opinions, even if you’ve never opened a statistics book. What follows are six very simple questions that anyone can ask and answer to assess a data-based article. The answers to these questions wouldn’t have rooted out fraud at the level alleged in the experiment on changing opinions — even experts didn’t find that — but in 99.98 percent of cases that don’t involve fraud, these questions will give you a good indication of the data quality.

1. Where did the data come from?
The source of the data shouldn’t be a mystery. Think of the source of the data the same way you’d think of a source for any piece of information — after all, “data” are simply many pieces of information. If the source of the data is “anonymous” or not provided, there’d better be a good reason.

Not all sources are equal, however. If the analysis is showing that Democrats did something bad, and the data were collected by a Republican-affiliated group, be skeptical. If a toothpaste company collected data that say their brand is preferred over all other brands, be skeptical.

2. Why were —> Read More