Is it a problem when people refuse to take part in my research and how can I avoid it?
So, you sent a survey to 100 people, and only 13 replied. Or, you wanted to compare responses from men and women, but 90% of your respondents are male. How might this affect your research findings? Here, we introduce you to the concept of non-response bias, discuss the threat it poses to your conclusions and outline how you can minimize it.
Non-response bias is the technical term used to describe errors researcher make in estimating a certain population characteristic which occurs because specific types of the respondent are under-represented in their research sample size. Population parameters need to be taken into account.
When the characteristics of those who take part in a research study are markedly different from the characteristics of those who do not take part (for example, in terms of age, gender or lifestyle preferences), the sample size can be said to be non-representative of the population from which it was drawn.
Basing insights or conclusions on samples characterized by a high level of non-response bias can be a disaster. Luckily, there are several steps that researchers can take to minimize non-response bias at the research design phase and to correct for it if it does occur. Before discussing some of these strategies, let’s take a look at how non-response bias affects research results.
Non-response bias can lead to wrong conclusions from the research, so it's important to be aware when it occurs in your study. Let’s consider the following example. A retailer hires a firm of marketing professionals to develop an advertising campaign, involving posters placed in certain subway stations in one large metropolitan area. After the posters have been in place for three weeks, the firm conducts a survey to evaluate how many residents have seen the posters, and what their opinions of its core message are. A researcher calls 1000 city residents randomly selected from the phone book. Calls are made between 9 am and 5 pm over the course of one week between Monday and Friday. After trying each number at least twice, the researcher has managed to speak to 580 respondents. The results of the research suggest that only 4% of city residents were exposed to the posters and that only 40% of those found that the message resonated with them. The posters are pulled.
The researcher might say yes, because a random sample was drawn, which means that all members of the population had an equal chance of being included in the final sample, which in turn increases the chances that the sample is representative of the population.
But, what are the implications of calling people at home between 9 am and 5 pm? This is a time when employed people are less likely to be home, which may systematically omit them from the sample. Had the researcher compared the mean age of those who did answer the phone with the mean age of those who did not, they might have observed systematic differences, so that the average age of the sample is biased upwards compared to the population.
Since older people are more likely to be retired, they may also be less likely to take the subway and therefore have seen the posters. So, the calculated population-level exposure rate might be biased downwards. There might also be differences in aspects such as parental status, income, and other factors that affect resonance with the advertising campaign. If non-respondents had been included in the sample, the results of this study might have been very different. The decision to pull the adverts prematurely might have been a very costly – and avoidable mistake. This also explains why targeting the right sample group at different times of the day comes handy to avoid non-response bias and non-participation bias.
This is where participants start and do not complete a survey during data collection process, or otherwise refuse to supply data. For example, sample members might have privacy concerns about supply data or might find the questions to be embarrassing. Sending a follow up could be helpful.
It is important to note that non-response bias can never be completely eliminated. It is very rare to achieve a response rate of 100%, and there are often shared characteristics among those who do not respond. However, there is much that you can do to minimize response bias and to mitigate its risks.
We hope this article has helped you understand the difference between Quantitative and Qualitative research.
Do not hesitate to contact us to get your market research project up and running. You can find more information on our reach and our panels.
We really hope these pointers have convinced you to consider mobile surveys for helping your company. If yes, please do not hesitate to reach out. The team at TGM will be more than happy to support you with your project.