In this issue: Important Considerations for Federal Market Research
Determining Correct Sample Size is Key to Survey Success
People working in federal government agencies, especially in the DC metro area, are one of the most sought-after participant groups for market research. Yet, the federal government has a few distinct qualities that make it more difficult to recruit than most other audiences.
Some key considerations when conducting research in government markets include:
Time of year:
One of the most important factors to successfully conducting a quantitative research study is determining the number of required completed interviews – or survey sample size – to achieve statistical validity. Conducting too few interviews will result in the inability to generalize the survey findings, while too many interviews may be unnecessarily costly to your research budget.
Determining the appropriate survey sample size involves the following considerations: Desired margin of error:
Derived from a formula based on the size of the total population as compared to the size of the sample surveyed, the margin of error defines how accurately the research results can be applied to the study’s population as a whole. Typically, researchers strive for a margin of error of 5% or less, which delivers a strong confidence interval of only 10 percentage points. In other words, you will have a level of confidence that the overall target population falls within plus or minus 5% of the sample surveyed. Of course, in order to obtain a certain margin of error, a minimum sample size is necessary.
Sub-segment comparison:
Often, clients need to break down the research population into sub-segments and compare responses between them. For example, perhaps you need to compare spending priorities between program management and IT titles. In order for those comparisons to be statistically valid, researchers need to achieve a certain minimum number of respondents within each sub-segment. In business-to-government studies, a minimum of 30 respondents per sub-group is typically required. Clearly, the more sub-groups the study needs to address, the higher the survey sample must be.
Analysis plan:
The type of analysis needed may influence the minimum sample size required. As noted above, comparative analyses – even simple cross-tabulations – require a minimum number of interviews. Some more complicated statistical analyses, such as factor analysis, market segmentation and predictive analysis, require a minimum sample size in order to be properly conducted and statistically valid. The number of completed interviews required is dependent up on which of the types of statistical analyses the research provider will perform.
Budget:
The cost of research is directly linked to the required number of completed interviews. Your research partner will help you develop a program that maximizes the balance between budget, sample size and statistical validity.
Ability to access and complete surveys with target population: Some populations are much more difficult to access and complete interviews with than others. And, the more difficult it is to reach respondents, the more costly the research will be. As a result, it’s important to be well informed up front about the accessibility of contact information for potential survey respondents and the likelihood they’ll be willing to respond. To achieve an acceptable incidence rate – or ratio of completed qualified surveys to overall sample list – researchers consider the following factors when determining the required size of the calling or email list:
Given the number and complexity of factors to balance when determining the correct sample size, meet with your research experts early on in the planning phase of your quantitative study. They’ll help you align these factors within the framework of your specific project to achieve the most realistic, cost-effective and statistically valid sample size.
| 
|
