I’ve been retrieving polling data (not the detailed stuff, but the overall results) from HuffPost Pollster. While checking the latest, I found an interesting article about a possible “mode effect” in poll results.
The claimed effect is this: some voters will give a different answer if they’re polled by a live phone call — and are therefore talking to an actual person — than if they’re polled by automated phone call or via the internet.
The effect, if correct, seems to apply to republicans choosing Donald Trump as their candidate. Apparently, if they’re talking to a live person they are less likely to choose Trump, but if not they are more likely to choose Trump. The idea is that they’re less likely to admit they pick Trump if there’s an actual human being doing the listening.
The analysis is based on the “mode” of polling, which in the data I’ve retrieved falls into four categories: automated phone, internet, IVR/online, and live phone. From what I can guess, in the “live phone” category the subject is talking to an actual person, not so in the other categories. When analysts compared Trump’s results from live-phone polls to the others, they noticed a difference; Trumps numbers were higher without a human being listening to the subject. You can see it in one of their graphs:
I ran a similar analysis and got a similar result:
Of course, there are many reasons the result could be real. Maybe Trump supporters are less willing to admit it when there’s a person listening rather than a machine. Or maybe, when talking to an actual person, they’re more likely to give an untrue response so they can somehow “game the system.” Or maybe … Fact is, I can only speculate and about this question, I don’t think my opinion is especially better than yours.
There is another possibility of course: the effect might not be real. In fact, I think part of the result is because it’s skewed by polls that don’t reflect the overall voter population and aren’t properly compensated for that fact. Consider that Trump’s lowest recent numbers come from the latest poll, taken just after the republican debate. That particular poll gets more weight than most because it surveyed more republican voters. But I don’t think it reflects the entire republican voting population.
One factor is that it seems the poll was only taken of people who actually watched the debate. Most voters didn’t, and such a sample might be skewed in two ways. First, it could have a heavier-than-average number of people who haven’t made up their minds yet, a fact which works against Trump because those who have made up their minds have tended to be strongly for Trump. Second, the poll provides information about some demographics, although it specifically states that the poll numbers were compensated for gender but doesn’t mention using the other factors. One of those factors is education.
The numbers suggest only 11% of those surveyed had only a high school diploma or less:
However, a poll I studied some time ago, which I think is more representative, suggests that the number of republican voters with a high-school-diploma or less is more like 30%:
It seems likely that those surveyed in the post-debate poll, i.e. those who watched the debate, tend to be more educated than the general republican voting population. And that means, that the results of the poll might be good for answering “Who won the debate?” but not so good for answering “Who will republican voters choose in the primaries?”
The poll was sponsored by the republican party, which doesn’t mean it’s biased, but it does mean that those few which share that attribute might have similar skew when it comes to representing the overall population. They also tend to be the polls which are most “out of line” with other results. Therefore I removed them from the sample (there were only four such polls in the last three months) and re-ran the analysis. With that adjustment, the “mode effect” is still there:
However, the mode effect isn’t quite as strong during August, but it is more consistent throughout the late-June to early-September period I studied.
One thing mentioned in the previously linked article is that no other candidate seems to show this effect. But it seems to me that I’ve detected it for another republican candidate recently, the man in 2nd place, Ben Carson:
This time, however, the effect is reversed; people seem to be more likely to name Carson when there’s a human being listening, than when just giving answers to a machine.
Whether the whole effect is real, I’m not sure. There are a lot of uncertainties, and I don’t have the detailed data to track down how the various demographics were used to adjust poll numbers. But it’s an interesting thought.
Finally, for those of you who are interested, my candidate of choice is a democrat: Bernie Sanders.
Open Mind 
Maybe they should test if the mode effect for Trump is amplified when the live phone caller has a pronounced Spanish accent.
Sorry to post something off topic, but I was wondering if anyone had seen this study: http://www.pnas.org/content/112/11/3241.abstract
It would seem to make a clear connection between GW-influenced drought and the current Syrian conflict. But a mathematician friend of mine noted that they are doing funny things with p values, and concluded that it was a fatally flawed paper. I would love to hear anyone’s further insights. I’m not far along in my maths to make a judgment call one way or another. Thanks ahead of time, and sorry again to be off topic, and further apologies if this has already been dealt with here somewhere.
I am sure Tamino could address any mathematical deficiencies much better than I. However The Syrian conflict has multiple causes, one of which is the extreme drought conditions of 2007-2010.
How much of the severity of that drought can be attributed to AGW is always going to be a difficult judgement, with various mathematical methods and assumptions competing to show or dismiss a null hypothesis that the event was an entirely natural variation.
However the assertion that AGW has had, and will have no effect whatsoever on the Drought in the ‘fertile’ crescent seems physically implausible. However dubious the maths and ‘p’ values, continuing climate change will inevitably have a regional effect on rainfall. The best ‘guess’ is that droughts will increase and intensify. That will increase conflict.
The winter rains in Syria and in most of the Middle East failed again this year, the drought is ongoing.
Thanks, izen. I tend to agree.
Here’s a link to the full article: http://www.researchgate.net/publication/273060165_Climate_change_in_the_Fertile_Crescent_and_implications_of_the_recent_Syrian_drought
Supplemental materials here:
Click to access pnas.201421533SI.pdf
I asked about the same paper recently, too, but I think few saw the comment. Tamino commented, very briefly. It’s all here.
(Well, it is if I did the HTML correctly. This is one of those times when I wish there were an edit function here. But it still looks OK on second glance, so…)
I don’t think Donald Trump is the worst of the republicans. Mike Huckabee and Ted Cruz look worse, and I really don’t want any Bush in there. My candidate of choice is a Social Democrat: Bernie Sanders. The only worry I have is that he may be unelectable, but that’s not going to change my choice.
I think Nate Silver at 538.com may have examined this kind of polling bias in some detail, at least for previous elections. Another good source for examining polling bias is Sam Wong at the Princeton Election Consortium (find it with Google). Caution: following Nate and Sam can become addictive if only because those who comment display the full range of logical foibles associated with climate change denial. Or, is it the other way around?
For what it’s worth: the election results of the right-populist, anti-immigration Dansk Folkeparti (Danish People’s Party) are always better than what the polls indicate. It would be interesting to find out if this also holds for other parties (or candidates) that voters might feel embarrassed to support.
Reblogged this on mt's Science Blog.
This made me think about something that I have noticed in the last few years. Since I got a Facebook account, I started realizing that some of my friends that I have know for a long time, have some rather disconcerting prejudices, and credulous tendencies on many issues, that I had never detected in my long association with them.
It is though, in face to face interactions, they are checking themselves, but curiously, feel free to expose some very worrying social attitudes, and fail to fact check dubious stories that they eagerly pass on and ‘like’ in a social media setting. The computer adds a bit of insulation apparently.
Maybe people who like Trump’s more embarrassing positions and unfiltered prejudice and misogyny on a gut level, are more reluctant to openly state that to a live voice, but find the anonymity of online polling easier to say what they feel.
And here I thought you were going to talk about a measure of central tendency!
Here in the UK we have the ‘Shy Tory Effect’, i.e. people who vote Conservative but don’t tell the pollster that this is their intention. It was coined after the 1992 general election: the Conservatives had been in power since 1979, John Major was prime minister and Labour were just ahead in the polls in the run up to the vote. In the end, the Conservatives won with a slim majority. It seems it struck again at this year’s election: Conservatives in power, neck and neck with Labour in the polls, a hung parliament predicted… then on the day the Conservatives won with 6.5% more votes than Labour, and the next day the pollsters were looking sheepish.
Quoting Morrissey’s “Glamorous Glue”- “We won’t vote Conservative because we never have/everyone lies”
The “Shy Tory effect” where the opinion polls tend to underestimate the Conservative vote in elections is often explained by the Conservative voters being “shy” or unwilling to talk to pollsters. I think the real reason is more prosaic, that demographically the people who vote Conservatives are more likely to actually vote. This is one of the reasons that the main parties try to please those who are retired while the young are ignored.This explanation explains why the exit polls are usually accurate.
Though it cannot explain the “Trump effect” other than it may be the differing demographics of the methods used.
And here I thought you were going to talk about a measure of central tendency!