Unsettling Market Research: A Researcher's View
by Clement LoWhat to make of it
A while ago, I was talking to the marketing director of a consumer product company that conducted market research periodically. The director was telling me that sometimes he did not know what to make of research. He said that research findings sometimes contradicted what his people in the field (i.e., sales and service staff) were telling him. He added that research, instead of confirming what they knew, would produce very unexpected results. As I was listening, I was thinking to myself that he was actually telling me the remarkable benefit of market research.
Research should do more
When a client tells me, even sincerely, that the research I just completed for him confirms everything he knows, I would take it as a failing. I would conclude that I must have overlooked some important questions or issues. Or, I must have done a poor statistical analysis. Research should do more for you than simply making you feel assured about what you already know. If research cannot contribute to a better understanding of your business, why should you conduct research?
Research should tell you things that you do not know. Research should contain surprising findings. You get your money's worth when research produces unsettling results!
New is unsettling
Market research contributes to a better understanding of your business, market or customers by producing new information, new insights and new data. Anything new is often surprising, unsettling and unexpected.
When you find that the market research you have just conducted does not tell you anything new, it is probably because it was not done as well as it could have been. It could be caused by many reasons. Perhaps the methodology was ill-conceived. The fieldwork was poorly executed. The wrong respondents (i.e., customers or consumers) were interviewed. The questionnaire and questions were badly designed. The sample size (i.e., the number of respondents) was too small. Or, the researcher's data analysis was sloppy!
Market research always has its value, no matter what business you are in. Then, why is that research sometimes does not correspond to the reported experience of sales and service personnel? Let's consider two factors.
Factor I
First, research such as a survey is a snapshot. Its findings capture market dynamics at a point in time. Market conditions can change after the survey was taken. For instance, competitive movement can bring about a change in your market conditions. Customers, for various reasons, can develop different requirements as time passes. Consumer behaviour is always fluid. Results of a survey do become outdated. To ensure that they can anticipate changes in consumer behaviour, many business firms conduct annual research.
Factor II
Second, a research project always has its margin of error. Findings of a survey have a range of data accuracy or reliability. This margin depends on sample size: the larger the sample size, the smaller the margin of error. Without explaining all the technical details, a random survey of 1,000 customers usually has an error margin of ±2.5%. For instance, when 80% of customers say that they would buy Product A, it actually means that it could be 82.5% or 77.5%, a spread of five percentage points.
When field sales or service personnel reported that most customers have mixed feelings about Product A, and that Product A might not turn out to be as well received as research suggests. This field intelligence also has its own margin of error. If the sales force did not gather their information from 1,000 customers, the error margin is much higher than ±2.5%. For instance, if the sales people spoke to 100 customers, responses from these 100 customers have an error margin of ±8%.
Research itself, then, has a much higher degree of reliability than information from the field. Consequently, a business decision based on market research will enjoy a higher degree of certainty.
Quality of decision making
I think this is the invaluable benefit of market research. It might have an unsettling effect on you. It might put in doubt what is generally accepted. And, it might contradict what you think you know. Yet, it enhances the quality of decision making.
Copyright © 2000 Pragmatic Consulting
Top of page | Gap Analysis | Scales of Measurement
Back to Articles
Mission | Service | About Us | Applications | What's New | Contact | Home