About EKOS Politics

We launched this website in order to showcase our election research, and our suite of polling technologies including Probit and IVR. We will be updating this site frequently with new polls, analysis and insight into Canadian politics. EKOS's experience, knowledge and sophisticated research designs have contributed positively to many previous elections.

Other EKOS Products

In addition to current political analysis, EKOS also makes available to the public general research of interest, including research in evaluation, general public domain research, as well as a full history of EKOS press releases.

Media Inquires

For media inquires, please contact: Frank Graves President EKOS Research Associates t: 613.235-7215 [email protected]

A BRIEF POST MORTEM ON POLLING ELECTION 41 – May 3, 2011

EKOS VS. THE ACTUAL RESULTS

[OTTAWA – May 3, 2011] Canada’s 41st election was extremely interesting and surprising. While we believe EKOS did a very good job in charting the direction of the election and some of the historical shifts that occurred, we were caught flat footed in capturing the majority victory for the Conservative Party. We correctly predicted a Conservative victory but we failed to predict the outright majority. Our final estimate of Conservative support fell well outside of the margin of error and, while it was not that far off the industry average, we were unfortunately on the low side of this crucial prediction. Having acknowledged this serious error, we believe that we did a good job in being the first to clearly identify three crucial new developments which have reshaped the Canadian political landscape:

  1. EKOS correctly noted early in the campaign an important shift to the NDP which we correctly estimated would see the NDP as Official Opposition with over one hundred seats. We were roundly pilloried for this prediction from those who claimed nothing was happening in the campaign and that the NDP vote would collapse.
  2. We correctly noted the death spiral of the Bloc Quebecois in Quebec early in the campaign which was also greeted with near universal scepticism. Our final prediction of a virtual sweep of the Bloc and a 60 plus seat performance by the NDP was borne out.
  3. We also correctly documented and commented upon the collapse of the Liberal Party although we failed to predict the final denouement which saw a crucial switch of some of the residual Liberal vote go to the Conservatives. This miss was most of the source of missing the Conservative majority.
  4. Less crucially, but of interest, we were the only pollster to correctly predict the victory of Elizabeth May in British Columbia.

As noted, we seriously underestimated the final Conservative majority victory but in fairness, the aggregate of all polling firms final numbers did not point to such an outcome. Some can say they clearly saw it but most of the hard numbers leaned to a Conservative minority (as estimated on Three Hundred and Eight). Our numbers were on the lower boundaries but within the margin of error of the industry aggregate estimate for the Conservative Party. We were off by an unacceptably large 5.7 points, although we were much closer on all other party estimates. This error did, however, put us in the bottom half of the industry performance this time. This is also the first time ever that we have failed to accurately predict the final outcome of a federal election.

We believe that beyond random errors due to sampling and measurement error that our underestimate was rooted in two factors:

  1. There was a late movement of Liberal supporters to the Conservative Party which shifted about three points to the Conservatives (half of our shortfall).
  2. The actual voters had a heavier concentration of Conservative voters who were more committed and enthusiastic and hence came out and voted in greater numbers than other parties. We included an adjusted estimate of final vote which we ran on Sunday but regrettably decided not to run with. This model combined firmness and enthusiasm measures and if we had used it we would have quite accurately called the race. Frankly, we failed to believe our own results and the fact that the gap on those who said they were “certain to vote” had largely disappeared. We therefore didn’t apply the correction that we had developed which showed the CPC advantage which was clearly revealed on Monday. We aren’t claiming vindication as it’s not okay to root in the bin after and find the right answer. Rather, we want to share the finding that the commitment factor is crucial and that it includes enthusiasm, where the Conservatives had a clear, measurable advantage. Next time we will use the index.

In closing, we would like to thank the tens of thousands of Canadians who generously responded to our surveys. We would also like to acknowledge sincere appreciation of our partners at iPolitcs who did a great job in covering this election. We would also like to thank those who have provided advice, comments, and feedback throughout the election, particularly in the world of social media. We congratulate some of our competitors that came closer to the final result and commiserate with those who missed. We do, however, feel that our research was second to none in charting the evolution of the campaign even if it fell short in the match to final outcomes. Finally, we congratulate all those who participated in the political process and recognize the passion and effort that they bring to Canadian democracy. Prime Minister Stephen Harper and his Conservative Party are to be congratulated on having won a clear and decisive mandate to lead the country.

Click here for the full report: ekos_vs_actual_results

14 comments to A BRIEF POST MORTEM ON POLLING ELECTION 41 – May 3, 2011

  • Mike Green

    That is a fair summary. A query: is it possible that the extraordinary event of late Sunday played a role in the last-minute decision-making of voters torn between Liberal and Conservative choices by pushing security/terrorism concerns to front of mind? This might explain the industry consensus was so far below the actual Conservative vote.

  • J. Jansen

    This election proved that polls are best utilized by dogs.

  • I blame Ignatieff.

    Liberal leader Michael Ignatieff had a choice in the final week as his party faded. He could have said “Let us show a united front and block the Tories by any means possible.”

    Instead, he went the route of “OMG! Scary socialists! Be afraid! Be VERY VERY AFRAID!”

    Needless to say, this did not stop the Liberal bleeding. The last days of the campaign are the time for you to be solidifying your support with the positive message of why your supporters are voting for you (and no, this is not a winning message either:”Vote for us because we used to be awesome, and we might be again some day! Uh…. Vote for us because your dad did!”)

    Iggy was half right. The scare tactics did, in fact, work. So when the faltering Liberal supporters broke in the final days, they didn’t go to the party’s natural ally, Jack Layton. Instead, they went to pad the Tory victory margin.

  • Dave

    I find it regrettable that potential methodological problems, including those raised here, were not aired publicly. In fairness, this criticism is addressed not to Ekos in particular but to all major polling firms active in this election.

    It is possible that other pollsters who were closer to the election night results used a similar commitment index, or adjusted the data in some other way. The various issues surrounding poll methodology should have been able to be scrutinized by the public, and if necessary have become a matter of public debate. Instead, the lack of transparency surrounding the pollsters’ methods left the public having to trust them, almost blindly.

    An important potential tool for the public is section 326 of the Canada Elections Act. It mandates that sponsors of election surveys must release, among other things:

    – “the number of individuals who were asked to participate in the survey and the numbers and respective percentages of them who participated in the survey, refused to participate in the survey, and were ineligible to participate in the survey,”

    – “the dates and time of day of the interviews,”

    – “the method used to recalculate data to take into account in the survey the results of participants who expressed no opinion, were undecided or failed to respond to any or all of the survey questions” and

    – “any weighting factors or normalization procedures used in deriving the results of the survey.”

    See http://lois-laws.justice.gc.ca/eng/acts/E-2.01/page-122.html

    Despite the requirements of section 326, response rates for surveys were not available in the reports published on the internet by any of the firms, and neither were the full methods for taking non-responses into account. The use of any “commitment index” in determining results would certainly have been a kind of “normalization procedure,” so if in use by Ekos or other firms, should also have been disclosed, in full numerical detail.

    Pollsters seem to treat these methods as some kind of trade secret in their competition to make the most accurate predictions. When they do, the public interest takes a back seat to these concerns.

    It is a good thing that at least some firms were closer to what actually happened, so the public wasn’t caught completely unawares. But what if all of them had agreed, but based on the same faulty assumptions? Without complete transparency, the public (with the help of expert observers) can’t form an informed opinion about what assumptions made by pollsters might be unreliable.

    Unfortunately, poll sponsors (usually news organizations) have not taken seriously enough their legal and moral responsibility to ensure complete transparency in the reporting of election surveys. They need to hold polling firms to the highest standards in this regard. If Elections Canada receives complaints, I hope it will decide to take action to ensure compliance in future elections.

  • shirley

    Frank … A fair and unbiased review. I gotta say you guys did a great job. First to note the NDP surge (with all the attendant anxiety that caused) and the 100 seat prediction was spot on.

    I have to say I preferred your outcome prediction to what happened. 😛

  • Alan Pater

    Jeremy

    I don’t agree that the enemy of my enemy is my friend. The Liberals are -at base- a right wing party that has at times played with centrist or even leftist policies in order to placidate the electorate. In fact, their natural allies are the Tories, as the some voter correctly surmised.

  • That’s a very good analysis of the outcome. Interestingly, I made a prediction over on my own blog/website, http://www.stevenbritton.com, where looked at the differential between Ekos’ polling data for the 2008 election compared to the actual result. I found that Ekos polled 3% low for the Conservative Party in that campaign, so I adjusted the Conservative number up by 3%, and each of the other three parties (except the Bloc) down by 1%, across each regional breakout that Ekos provided.

    The result I came up with when I projected the result was interesting:

    Conservative: 159
    NDP: 114
    Liberal: 29
    Bloc: 6

    With an accuracy of 81% on an individual riding basis. (Which is pretty good!)

    My intuitive prediction, which was based partially on the algorithm and also based on how I figured the voting patterns would go was worse than the purely mathematical projection, so that tells me not to go with my gut instinct when trying to figure out an election result.

    Plugging the electoral results in to my algorithm, again, spread out across the same regions Ekos provides in their data, my program produced this:

    Conservative: 160
    NDP: 110
    Liberal 36
    Bloc: 2

    With an accuracy of 88% on an individual riding basis.

    Seat projections are extremely erratic and difficult, simply because regional data is being applied to very localized information, and makes assumptions which, while necessary to get the projection, are still assumptions, and therefore uncontrollable variables.

  • Tired of Conservative Fear Mongerers

    Satisfied with your assessment of the election results. As a individual, I could not see a Conservative majority either. (Altough I hoped for one.) My hope for a majority was in a possible vote split; with the Liberals & NDP, allowing the Conservatives to come up the middle.
    While this happened, your assessment of the erosion of the Liberal support is important. I might add to this: More conservative minded Liberals went to Harper, for various reasons including stop the NDP. While more social minded Liberals went the other way, thinking the NDP had a better campaign and chance of winning.
    The Conservative support was from the beginning, and remained very firm. (My mind was made up to vote Conservative the day the election was called. I was very upset that we had to have an election, and saw it as unnecessary. I made sure to cast a ballot, and to the Conservatives for this reason.) It is interesting to note that the 2 parties that were steadfast in their resolve to take us to the polls are decimated. Both the Bloc, and Liberals were hit hard, and their leaders are gone. Jack Layton was the only one who made a real effort to engage and negotiate with the Conservatives before dropping his support.
    Finally, I am glad there will be no federal election for 4 years! If Harper now proves to be moderate, as he has been; he could be around for quite some time.

  • I went to bed Sunday night after reading your late Sunday report feeling quite good about what was going to happen the next day. I was excited and happy thinking that maybe, just maybe, I was going to see the kind of result I’ve waited for my entire adult life. I was sadly, sadly dissappointed and , although a grown man, I sat there and cried as I watched the returns come in. Harper and his gang really frighten me. I think Canada can aspire to more than a northern imitation of Republican America.
    My real fear is that this will be the last free and fair election we’ll have for quite some time. Sounds a bit extremist I know, and of course I hope I am wrong and that this government pleasantly surprises me, but I doubt it.

  • Oh, and just a quick word to Tired. This election wasn’t “unecessary” and I got sick and tired of hearing that during the campaign. No exercise in democracy is ever “unesessary” unless you live in a dictatorship where they are just empty window dressing. We’re not there yet.

  • Tridus

    Interesting analysis, thanks for posting it. I think a lot of the issue is around turnout and how the Conservative base is very motivated and the party is extremely good at “getting out the vote”. That advantage doesn’t show up in polls but does show up on election day.

  • Peter

    I was surprised that “blue” Liberals would actually vote Conservative. This is not the “Progressive” Conservative Party. They are well right of centre. Let’s hope that Harper keeps his hands off social issues (gay marriage, abortion) and governs moderately. I do wonder if these blue Liberals will support Harper’s spending billions on warplanes and prisons. Many people forget that this Conservative Party has big deficits.

  • Roger Palfree

    Although I get annoyed with the way polls are handled in much of the media, as if they are the latest score in a sports event, I am quite impressed with the Ekos data, and enjoy the more thoughtful, analytical discussions they can stimulate. Of course, the discussion has to wade into the murky waters of psychology, marketing, tribalism and charisma. With the latter, Ignatieff could have survived the Conservatives’ character assasination and broken through the straight-jacket of approved messages. But while Ignatieff could engage an audience of willing listeners, he could not capture hearts and minds with a content-loaded, witty remark to reporters. Meanwhile, boring Harper, also charismatically challenged, could rely on voters’ reluctance for change during perceived difficult times, and had surprising success in influencing opinion by using of the Bellman principle (In Lewis Carroll’s Hunting of the snark): “I have said it thrice: What I tell you three times is true.”

  • Greg

    I find the analysis in the post-mortem balanced and fair – it was clear Ekos had the lead on the NDP surge, among other trends observed. I also enjoyed reading the thoughtful comments, pro and con, with the exception of one comment. J. Jenson’s comment shows was a fool he is when everyone else took the time to substantively make thoughful remarks. Did anyone laugh?