Many clients often ask about the ideal sample size required for achieving statistical validity in market research. This is an outstanding question and the answer is: it depends.
One of the keys is understanding the value and difference between qualitative and quantitative research. With qualitative research, if you do 50 in-depth phone interviews among targets, the real value is the insights gleaned from the depth of these conversations—the reasons, perceptions, feelings and beliefs behind participants’ responses—rather than statistical data.
With quantitative research, it is all about determining the size of the total audience of interest, or “universe,” and then you can formulate the necessary sample size in order to have statistically valid data. For example, if you are targeting military logistics leaders, there may be only 1,000 people making up the universe of people available to survey. With an audience of this size, if we connected with 150 of these decision-makers and influencers, then the resulting data would have a measurably high level of validity.
For larger quantitative government surveys, the potential audience could be in the hundreds of thousands. As such, we often work with clients to slice and dice the research by agency and job functions. Depending on the number of subgroups or segments of interest, we would recommend a suitable sample size to allow us to cross-tab the results by agency and job function with at least 30 respondents for each.
A good rule of thumb for a survey is to have enough respondents for a margin of error of no greater than ±7% at a 95% level of confidence.
It is all too easy to get wrapped up in the sample size discussion, when the most important outcome is developing the right actionable knowledge that will help guarantee success.
Really, the most important factor in all of this is determining how you will use the data. Sometimes the more in-depth insights from qualitative research provide all of the information needed to guide an effective marketing or influencer program, or strategic business decisions. We often recommend the qualitative and quantitative hybrid approach that provides commentary and insights backed by measurable data.
In terms of the bigger picture, market research is playing an even more critical role for government contractors in today’s austere budget climate. The right insights with regards to customer satisfaction, contract capture, and communications testing, can provide a contractor with the information needed to drive business growth.