As the traditional landline telephone goes the way of the VCR or console TV, the methods and modes of large-scale population surveys are needing to change, too, while still preserving data accuracy.
To meet this challenge in the industry, University of Nebraska—Lincoln sociologists Kristen Olson and Jolene Smyth chaired a task force for the American Association for Public Opinion Research that examined best practices for transitioning away from telephone surveys to mail, web and mobile devices.
This task force reviewed existing empirical literature and survey practices across many survey organizations, large and small, such as the Bureau of Sociological Research at Nebraska, which Smyth directs.
A report on this work was released Oct. 23, and is designed to help social science, health, economic and other researchers understand how to conduct surveys in today’s changing environment.
“Overall, the survey industry is incredibly adaptable,” said Olson, professor and vice-chair of sociology, said. “We change as the world around us changes. We do this with a scientific and empirical basis, but when there are these infrastructure-level shifts occurring in the survey industry, having a moment to step back and evaluate all of the parts that are changing simultaneously is important.
“Survey organizations have a lot on their plate, as with any data-driven industry, but really want to have a good scientific base for making decisions. Many survey organizations haven’t had mail and web surveys as part of their toolkit for collecting high-quality data. They needed guidance on how to do this. This task force report came out of this need.”
The 300-page report from the AAPOR Task Force on Transitions from Telephone to Self-Administered and Mixed-Mode Surveys report provides empirical guidance and best practices for survey researchers.
The task force found that there is not one single design that is being used as surveys transition away from telephone to self-administered and mixed mode surveys. Large-scale government surveys and smaller regional and local surveys are all making decisions to move away from telephone, as response rates decline and other methods of identifying households have become available.
“This change in standard practice isn’t bad – we found that response rates actually can be higher now using self-administered modes than with telephone surveys; that a wide range of incentives can be provided to sampled people for effectively increasing response rates and potentially reducing whether the people who participate resemble the general population; that new opportunities arise for innovating in how to ask survey questions; and costs for high-quality probability surveys can be contained without dramatically losing quality, for instance,” Olson said.
Most surveys that have transitioned away from telephone interviews use a mix of mail surveys and web surveys, and most continue to use a probability sample for survey estimates. These updated research modes have been largely successful, but with some caveats, Olson noted.
“Surveys that are shifting from telephone to self-administered and mixed modes find that it can be complicated to implement existing survey processes in these new modes, with many challenges,” Olson said. “For instance, we find that surveys are struggling with how to maintain a probability sample of people from households when you don’t have an interviewer helping to randomly select a person. Long-standing surveys that previously interviewed people for an hour or more have to rethink the length of a questionnaire when an interviewer isn’t present.”
The report examines these other factors related to questionnaire design, data processing and ethical considerations. A summary of the report is available here
