Tenato logo


Why Were the Polls Wrong?

Insights on the Alberta 2012 Election

The poll slippage in last night’s election was certainly extraordinary, and all the pollsters on the evening newscasts were saying how surprised they were.

There are several common reasons why polls are “off”. The pollsters are going to have to (and they will) do their own research and then duke it out as to what happened. Pollsters live by reputation and their reputation rests partly on congruence with election results. So they will look in hindsight to see what happened. Because I personally didn’t poll this election, I can’t comment on the specifics. However, I can speak to where the reasons usually lie, and give some hints as to where to look.

First reason for “off” results is that there was a large undecided vote until very close to the election, and the polls closed down too soon to pick it up or account for it. That might have been largest factor this time. Several of the serious concerns about Wildrose surfaced only in the last week before April 23.

Second is – that the large undecided vote was conflicted emotionally, and in truth the undecided didn’t know how he / she/ they were going to vote so they couldn’t have told you anyway. That is very possible in this election. Many were very very offended by the PCs (an emotional reaction) but were also very very wary of the closest rival, the Wildrose (also an emotional reaction). And the PC campaign played up words like “scary”, “frightening”.

Third is – methodology matters, and relates to the other two. Some pollsters used telephone interview. Some used online. Some used “robo calls”. Some monitored digital and / or social media. All have their pros and cons. Telephone usually has a larger time gap from field calls to report. Robo calls are more immediate, but lack explanatory comment. Social media monitors have demographic biases. That’s what the pollsters will be duking out. How much to rely on single research media and how much to triangulate between multiple media? The best projections from poll results take all into

Fourth is – projections. Pollsters, unless they are hugely sophisticated, well-financed and poll very large samples, usually poll to “popular vote” statistics. The real issue, of course, is how that translates into seats. In Alberta, we of course have a “first past the post system”, and many of the poll results we saw tonight were very close. In other words, a small difference in popular vote made a big difference in seats.

Fifth is – pollsters (and others) were predicting a greater-than-average turnout. Early results (and I write this very early in the evening after the election) are that the turnout was not abnormally large. So some of the “intuitive” or “experiential” predictions (yes, pollster project their data intuitively and experientially as well as mathematically) were off again because the “don’t-usually-vote” people didn’t show up. This could affect the demographics of those who in fact voted, which likely relates to party

Sixth is – again putting previous points together, it is quite possible that the undecided vote turned to PC due to fear of the unknown (i.e, Wildrose being seen as “this dangerously upstart part with no governing experience”) and fear of the possible damage to the economy and good governance from “reactionary” Wildrose. Habit and the familiar won out in the polling booth.

So put all this together. In this 2012 Alberta election, we had an unusual situation with conflicted platforms, personalities and parties. It is very possible that the pollsters used traditional methodologies which weren’t strong enough to pick up the underlying dynamics.

I definitely expect one or more of the affected pollsters to do the research needed to find out what happened. How to do it? Maybe one or more of the pollsters did exit interviews (i.e, asking people after they voted what their choice was, and why) at the polling stations, and the exit interviews will eventually be made public. Or maybe the more experienced pollsters will relate the 2012 Alberta election to earlier ones when the results were “off”. In truth, however, there aren’t many such. Polls usually predict very well!

Alberta 2012 was an anomaly.

Share this:

About the Author - Joanne O'Connell
Joanne, O’Connell, BSc, BA, MA, CMRP As a CMRP, (“Certified Marketing Research Professional”, an accreditation from the Marketing Research and Intelligence Association of Canada) Joanne brings over 30 years’ experience in quantitative and qualitative research methods encompassing a vast array of private and public sector clients.

4 thoughts on “Why Were the Polls Wrong?

  1. There’s a lot of moaning and groaning going on today about the polls being wrong, surprise about that, and for some a — tendency to dismiss polling as unscientific prognosticating. I appreciate your thoughtful analysis.

    As for your point about methodology. I would like to add that I think some people just get so aggravated by all the phone calls that if they answer at all, they lie out of pique. Especially when it’s a robo-call that has interrupted their train of thought or their dinner!!

    The other thought on that subject is that so many people don’t even have and/or answer land-lines anymore. I don’t think the pollsters can get at mobile numbers yet.

    1. Yes, I agree. I almost wonder if there’s going to be a robo-call scandal, people were getting so many of them. Everyone I know said the calls were all being made by Wildrose, which was angering people.

  2. Read your post ‘Why Were the Polls Wrong?’ seeking to understand why they were also wrong in the recent 2015 UK General Election. The six points you make apply exactly to the UK vote. The only point you make that doesn’t correlate is the final one – ‘Alberta 2012 was an anomaly’. Rather I suggest it was an example the way of things will be.

    1. Glad to hear that Ken -yes, I think new techniques will be needed to make polling faster, or altogether irrelevant. Social media tracking is for sure a real contender.

Comments are closed.