Christmas is upon us and Santa will leave his comfy home in the North Pole soon to fly his sleigh around the whole world in one night, delivering presents to all who were good this year. Right? What do you mean you doubt that? I just put it in writing and you read it, so it must be true. Well, you have every right to question that statement, just as you should question the credibility of many research findings you read every day.
Credibility is synonymous with reliability and gets to the core issue of market research – how reliable is the research you read about and more importantly, how reliable is the research you conduct. Before you launch into executing a primary research study, it is very advisable to start the process with exploring secondary research to see if someone has done the work for you. Although this should be your standard approach for nearly all research projects, the key is determining if the research findings you unveil are 1) credible and 2) representative of the group you are targeting.
The majority of the time when you read research findings in traditional media, you will not be provided the most crucial piece of information; who did they survey? When this information is lacking, the reader is then left to determine if the research findings represent the U.S. general population or only a small niche of society. And, if the findings are representative of only a segment of the population, what segment is that? Leaving it to the reader to determine the credibility of a research finding can many times lead people to make conclusions based on either incorrect or distorted information.
For example, the following research finding was quoted in a press release I read about a particular study that was completed by a major research company:
“More than nine in 10 online women (95%) and 86% of men belong to Facebook”
That was the only info provided about this finding—no caveats whatsoever. I am assuming most people would read this, not question it at all, and walk away believing that roughly nine out of 10 Americans who are online belong to Facebook. I hope you don’t; even Mr. Zuckerberg would likely admit that the number is rather inflated. The number more commonly reported and consistent with primary research I have conducted is more in the 50-60 percent range. So, is this number flat out wrong? Well, I won’t go as far as to say the number is wrong, but it certainly is not a number that is representative of the general population, which should be noted. It they had said this is the percent of users among high school seniors, I may have believed the number. However, I am fairly confident I could go anywhere in the country and randomly ask 100 men if they are Facebookers, and I would not average nine out 10 saying, “Yes.”
Further, some research findings simply don’t make sense. Here is a recent research finding from an article about the impact of a ride-and-drive event on the purchase decision of a new car:
“Buyers who participated in a ride and drive were influenced by the event at a 42 percent higher rate than those who did not, the report showed.”
Is it just me, or does that make no sense? How can you be influenced by an event you never attended? I think, or I hope, the author was trying to say something along the lines that those who attended an event were 42 percent more likely to purchase the car than those who didn’t. However, that is clearly not what was written.
So why does this happen so frequently? One reason is that often the person who is writing the article is not a researcher and simply doesn’t have the skill set to clearly communicate research findings. Other causes are the fact that there is clearly a lot of bad research out there. Simple research tools are readily available that allow anyone who can do some basic programming to be a market researcher. However, the downside is that those who use these tools do not always understand research methodology and proper sampling techniques. The result is that you get research findings that are severely flawed. We have all heard the expression, “garbage IN = garbage OUT;” that could not be more representative of research.
So reader beware: Santa does not fly around the entire world in one night, and many of the research findings you read about could be just as farfetched.