I’ve been retrieving polling data (not the detailed stuff, but the overall results) from HuffPost Pollster. While checking the latest, I found an interesting article about a possible “mode effect” in poll results.
The claimed effect is this: some voters will give a different answer if they’re polled by a live phone call — and are therefore talking to an actual person — than if they’re polled by automated phone call or via the internet.
The effect, if correct, seems to apply to republicans choosing Donald Trump as their candidate. Apparently, if they’re talking to a live person they are less likely to choose Trump, but if not they are more likely to choose Trump. The idea is that they’re less likely to admit they pick Trump if there’s an actual human being doing the listening.
The analysis is based on the “mode” of polling, which in the data I’ve retrieved falls into four categories: automated phone, internet, IVR/online, and live phone. From what I can guess, in the “live phone” category the subject is talking to an actual person, not so in the other categories. When analysts compared Trump’s results from live-phone polls to the others, they noticed a difference; Trumps numbers were higher without a human being listening to the subject. You can see it in one of their graphs:
I ran a similar analysis and got a similar result:
Of course, there are many reasons the result could be real. Maybe Trump supporters are less willing to admit it when there’s a person listening rather than a machine. Or maybe, when talking to an actual person, they’re more likely to give an untrue response so they can somehow “game the system.” Or maybe … Fact is, I can only speculate and about this question, I don’t think my opinion is especially better than yours.
There is another possibility of course: the effect might not be real. In fact, I think part of the result is because it’s skewed by polls that don’t reflect the overall voter population and aren’t properly compensated for that fact. Consider that Trump’s lowest recent numbers come from the latest poll, taken just after the republican debate. That particular poll gets more weight than most because it surveyed more republican voters. But I don’t think it reflects the entire republican voting population.
One factor is that it seems the poll was only taken of people who actually watched the debate. Most voters didn’t, and such a sample might be skewed in two ways. First, it could have a heavier-than-average number of people who haven’t made up their minds yet, a fact which works against Trump because those who have made up their minds have tended to be strongly for Trump. Second, the poll provides information about some demographics, although it specifically states that the poll numbers were compensated for gender but doesn’t mention using the other factors. One of those factors is education.
The numbers suggest only 11% of those surveyed had only a high school diploma or less:
However, a poll I studied some time ago, which I think is more representative, suggests that the number of republican voters with a high-school-diploma or less is more like 30%:
It seems likely that those surveyed in the post-debate poll, i.e. those who watched the debate, tend to be more educated than the general republican voting population. And that means, that the results of the poll might be good for answering “Who won the debate?” but not so good for answering “Who will republican voters choose in the primaries?”
The poll was sponsored by the republican party, which doesn’t mean it’s biased, but it does mean that those few which share that attribute might have similar skew when it comes to representing the overall population. They also tend to be the polls which are most “out of line” with other results. Therefore I removed them from the sample (there were only four such polls in the last three months) and re-ran the analysis. With that adjustment, the “mode effect” is still there:
However, the mode effect isn’t quite as strong during August, but it is more consistent throughout the late-June to early-September period I studied.
One thing mentioned in the previously linked article is that no other candidate seems to show this effect. But it seems to me that I’ve detected it for another republican candidate recently, the man in 2nd place, Ben Carson:
This time, however, the effect is reversed; people seem to be more likely to name Carson when there’s a human being listening, than when just giving answers to a machine.
Whether the whole effect is real, I’m not sure. There are a lot of uncertainties, and I don’t have the detailed data to track down how the various demographics were used to adjust poll numbers. But it’s an interesting thought.
Finally, for those of you who are interested, my candidate of choice is a democrat: Bernie Sanders.