Discrepancies in state and national polling predictions raise questions about their reliability.
In the final days of the presidential race, Mitt Romney and President Barack Obama remained roughly tied in national polls, while state-by-state polls showed Obama leading in the Electoral College. The schism has prompted many to question the nature and methodology of various pollsters. In telephone surveys, factors such as polling voters via cell phones versus landlines, the wording of questions, the timing of the surveys and the relative sample size of respondents could impact the results’ accuracy, said Jiali Luo, assistant director of institutional research.
According to a state poll released by Gravis Marketing Tuesday, Romney led Obama 50 to 46 percent at the start of election day. The national Gallup poll, however, showed Romney leading Obama by only 49 to 48 percent among registered voters Nov. 5. Both showed Romney winning, and yet ultimately, Obama was re-elected.
“There are plenty of reasons why we should worry about polling methodologies,” said Sunshine Hillygus, associate professor of political science. “Given that the response rates [to polls] have been 10 percent, you have to wonder if there is a systematic bias in these popular polls.”
An ‘imperfect’ method
Telephone surveying is a main polling methodology. One of the current challenges, however, is the fact that a majority of people are abandoning landlines in favor of cell phones, Hillygus said. As a result, popular national pollsters, such as Pew Research Center, Fox News and ABC News, now collect two samples—the traditional landline sample and the cell phone-only sample.
Given that it is illegal to auto-dial—or use a computerized autodialer to send pre-recorded messages—a cell phone, reaching out to individuals using only cell phones has become an expensive endeavor, Hillygus said. It is an obstacle for many of the smaller pollsters, such as those on the state level, in accumulating more accurate and reliable data.
Individuals who have both cell phones and landline phones are considered landline users for polling purposes. Such classification does not account for the fact that these voters primarily rely on their cell phones and may therefore neglect to answer their landlines, which could skew polling results.
“It’s still an imperfect method,” she said.
Because young adults and minorities are more likely to use cell phones, such sampling under-counts the younger, more Democratic-leaning vote, Hillygus said.
“If we do not consider such factors carefully, the poll results can be quite misleading,” Luo wrote in an email Monday.
Reliability
Predictions of who will win based on national polls are more influenced by small amounts of error than predictions based on state polls, said Christopher Johnston, professor of political science.
“[If] I had to place even money on either the state-based forecast or the national forecast, I would put my money on the state-based forecast,” he wrote in an email Monday.
Johnston also attributed the reliability of the state-by-state polls to the homogeneous target populations of individual states and the frequency of swing state polls.
“Swing state polls are conducted more often, and these are exactly the places where more information is helpful for reliably predicting the outcome,” he said. “ Our estimates might be more efficient on average for states where the election will be decided.”
Hillygus noted, however, that national-level polls tend to be relatively more accurate and more reliable than state-level polls.
“State-level polls tend to be less frequent and sample a smaller size pool, which leads to a lower-quality of data when compared to the national polls,” she said.
The media environment that surrounded this year’s election was drastically different from that of four years ago, Hillygus said. One difference, she noted, is that organizations other than news media groups have conducted and sponsored their own polls, adding that some agencies—such as some entrepreneurial groups—are driven by personal motivations.
Pundits and bloggers, for example, have also contributed to the skepticism about polling results, she said.
“It’s unfortunate that polls have been criticized or praised on the basis of their results rather than on the basis of their methods,” she added.
Get The Chronicle straight to your inbox
Signup for our weekly newsletter. Cancel at any time.