About EKOS Politics

We launched this website in order to showcase our election research, and our suite of polling technologies including Probit and IVR. We will be updating this site frequently with new polls, analysis and insight into Canadian politics. EKOS's experience, knowledge and sophisticated research designs have contributed positively to many previous elections.

Other EKOS Products

In addition to current political analysis, EKOS also makes available to the public general research of interest, including research in evaluation, general public domain research, as well as a full history of EKOS press releases.

Media Inquires

For media inquires, please contact: Frank Graves President EKOS Research Associates t: 613.235-7215 [email protected]



[Ottawa – October 12, 2012] The Quebec election was remarkable for a number of reasons. The narrow victory of a Parti Québecois (PQ) government, with an avowed goal of separation from Canada, is not causing that many ripples in English Canada. While Levesque prescribed a Valium to deal with the visceral angst in 1980, the current reaction to the prospect of another referendum in Canada appears to range somewhere between indifference and hostility. There is a consensus that the issue of separation is pretty well dead and that this result somehow seems to feed that comfortable delusion.

The rather inauspicious performance of the polls seems to have reinforced that apathetic response. While many of the polls were hinting, if not virtually guaranteeing, a PQ majority , the surprising resilience of the Quebec Liberal Party (PLQ) was unexpected. For many, these results were seen as vindication of the general lassitude with respect to all things national unity and also as vivid evidence of the broad based ineptitude of polling in Canada.

I will argue that the two shocking results of this election, the near parity results for the supposedly exhausted regime of the PLQ, and the relatively huge turnout result, are inextricably linked in ways that have not been properly considered. The notion that a PQ victory of this slender margin isn’t terribly troubling for those committed to a united Canada can be added as a third surprise in the making.

So let’s consider why the polls might have been so off and how this might be related to the issues of turnout and possible Quebec secession. First of all, we need to acknowledge that our somewhat flippant and very late entry into the polling of this campaign was ill-advised and flawed. Despite the fact that we had no formal releases and merely put out some eleventh hour Twitter notes on our polling this does not expiate us from the sin of having been very far off the mark. Yes we predicted a PQ victory, but that is about the end of the legacy of achievement here. We wanted to test some new technology, use some new approaches to measuring likely turnout, and see how our results compared to the outcome. Unfortunately we really didn’t have the time or resources to properly implement the test and our final results were distressingly different than the election outcome. In this sense, we agree with Warren Kinsella’s rather harsh indictment that you get what you pay for (in our case, nothing).

Some have castigated us for such a cavalier entry in the final stages and there may be some validity to those criticisms. In a very tight and uncertain contest, there are always concerns that late polling, particularly errant late polling, can have an unwanted and deleterious impact on the democratic process. While our view, shared by the hard literature, is that there is little evidence that this is true, I can still sympathise with the view of those who ask why pollsters would choose to submit such specious data when it might have a mischievous and corrosive impact on the actual vote.

While it is possible that whatever marginal Twitter following I might have in Quebec may have been influence by this eleventh hour poll, I doubt very much that my late night Twitter feed had any discernible impact on the outcome of the election. Others may want to argue that other front page stories virtually assuring a majority PQ government (we never made any such assurances) may have had some marginal impacts on strategic voting.

So why did everyone (us more than others) miss the surprising resilience of the PLQ? With a very slight shift in voter preference (possibly less than one per cent), Jean Charest would have moved from retirement to political hero and become the new heavy weight champion of Canadian politics. No one saw that as a likely outcome the day before.

As we can see, the razor thin margin of victory was very different than the near run away that was seen in the late poll. The PQ was slightly outside the margin of error (lower than estimated), as was the Coalition Avenir Québec (CAQ). Québec Solidaire (QS) was well below the last poll, but our analysis expected a drop out in actual voter turnout for QS and “other” party supporters. The really stunning error is the eight-point miss on the PLQ, which was mirrored, albeit less dramatically, in other polls.

While the pillorying of polling may exceed its actual shortfalls, the pollsters have provided ample ammunition to critics and sceptics over the past year and a half. First of all, no one clearly predicted the Conservative majority in the 2011 federal election despite a blizzard of polling. In our retrospective analysis, we tested and eliminated several popular hypotheses, including that of a late shift and a “polling sucks” hypothesis. The key to understanding the gap lays in understanding turnout, which is more and more a product of recent political ‘technology’, both relatively benign as in the case of get out the vote, and criminal as in the case of the yet–to-be-determined vote suppression or ‘roboscandal’. In the case of Ontario, we had no reason for apology and basically nailed the result early and correctly. In the Ford victory in Toronto, that was also the case, although the rest of the polling community was less successful. In Alberta, where we prudently abstained, the polling results looked awful. Greg Lyle has produced a very plausible explanation based on late shifting and loosely anchored impressions of a new party which seems to suggest accurate polling was linked with lousy narratives . In Quebec, the final results were very different from the polls again.

We did a number of tests to assess whether the issues of differential turnout may have explained the gaps. We also considered the issues of sampling errors and the degree to which those might account for the problems. There is also the possibility that there were late shifts in Quebec. In particular, did fearful federalists, confronted with the increasingly plausible prospect of a PQ majority, as well an inevitable referendum redux, shift to the PLQ to forestall this possibility? Finally, we considered (but can’t really evaluate clearly) the hypothesis offered by Claire Durand that the issues aren’t so much sampling errors as measurement errors. In her view, the tendency, which is not unique to Quebec, is for respondents to basically understate their support for conservative parties in recent electoral polling. In other words, abashed respondents conceal the fact that they are voting for a right wing government or, in this case, one which had been harshly criticised as a tired and possibly ethically challenged regime. We believe that there is some merit to this hypothesis as the other explanations simply cannot account for the overall magnitude of the PLQ surge from polls to Election Day. In fact, we cannot provide a clear answer to these hypotheses but it seems that it was some amalgam of these factors. We suspect that the measurement error and emotional impetus of concerned federalists may be the key drivers of this gap.

Before considering the referendum as the driver hypothesis, let’s consider some more prosaic explanations.

The turnout explanation argues that the differences between the actual voters who show up and the entire population of all eligible voters can explain the gap between a final poll (of all eligible voters) and the election result. This was, in fact, the conclusion that was not rejected by the empirical tests in our retrospective of the last federal election . We look at this in a number of ways here. First, we consider the differences between the vote intentions of some of the groups which historically don’t tend to vote as often (e.g., younger voters, those who have not voted in past elections, cellphone-only respondents, etc.). These are presented in the figure below. We also include a test which weights the data to restore its representativeness to the 2008 election. We note that the large discrepancy that we found on that variable can reflect sampling errors or measurement errors (the previously noted tendency to under remember a now unpopular past voting choice). Note that the weighting to 2008 vote also removes the non-voters in 2008 and there is a tendency for non-voting patterns to continue into the future.

So why the egregious underestimate of the PLQ? And why did Quebec voter turnout exceed 70 per cent compared to such humble recent benchmarks as the roughly 40 per cent in Alberta and somewhat shy of 50 per cent in the last Ontario election? Even the big show federal election of May 2 last year barely nosed above 60 per cent. What was it about this election which caused such a strong turnout? The issue of regime fatigue and corruption issues seems like an unlikely explanation (and flies in the face of the much stronger than expected PLQ performance). The student revolt, the pots and pans demonstrations, and even the one per cent movement were all powerfully expressed in Quebec over the past year. It is hard , however, to link the surprising PLQ strength with these drivers. The only issue with the potent combination of core values and large vested interests capable of explaining this sort of turnout is the issue of sovereignty and the question of another referendum. As seen in the figure below, this issue was galvanized along party lines and fear of another referendum is the most plausible explanation for why an older, clearly unenthusiastic PLQ constituency seemed to hold their noses and vote in surprising numbers for a regime that was depicted as tired and incapable of dealing with the construction industry’s corruption that seemed to plague Quebec during their tenure. There are clearly other factors as well but we offer this speculation based on our limited data.


Having stirred the still smouldering ashes of the Québec election, we offer the following tentative conclusions. First of all, the polling was indeed as daunting a challenge as experts such as Durand predicted it would be. As in other recent campaigns, there is some ‘splaining’ to do on the part of the polling community, but it is not of the magnitude that it renders the polling as suspect as some commentators have remarked. We do, however, believe that the comment that you “get what you pay for” is increasingly accurate. It becomes increasingly wearisome to hear ongoing complaints about the quality of polling which coincide with a precipitous decline in the resources available to properly poll. The only things which have dropped more precipitously than media polling budgets are the methodological fluency levels of the media and the response rates achieved by pollsters, so is it reaching a point where we would be better served with no polling. The real answer is not yet but we are pretty damn close.

Proper budgets would have allowed for higher quality methodologies; better, larger, and more frequent polls; and the proper diagnostic tools to do the immediate post election work to find out what worked, what didn’t, and what the election really meant. Budgets alas are no larger than media attention spans in the cluttered new AADH world of media polling and the harsh answer is that the system is badly broken. There is little in the way of an open source, blue ribbon source of polling data. Instead, media and public subsist on an uneasy mixture of different methodologies driven more by budget constraints than a search for real quality and insight. Eager pollsters line up to play in this auction to the bottom with increasing evidence that the risks far outweigh the potential benefits in this cheapened arena. The alternative of no polling and leaving these tools to the insiders in the party machines is equally unsavoury but something is badly awry.

Are there any clear answers to why the PLQ did much better than expected and why the participation rates rose so dramatically? There is no single answer and the preliminary analyses provide some tentative guidance as to the mix of factors which explain this:

  1. The Quebec political market place continues to be quite, well, ‘er, distinct. Quebeckers appear to be more engaged and dynamic that their fellow citizens in English Canada. The undecided rates were somewhat higher than usual here but, unlike in English Canada, indecision can mean the propensity for real change rather than “I haven’t really thought about it and probably won’t vote”.There seems to be a continued tendency to see an advantage for the incumbent which is hidden to the pollster.
  2. Whether this is a tendency to understate support for a socially undesirable choice or a more elegant form of pollmanship, it seems that respondents misrepresented their choices to pollsters about their true choices.
  3. Pursuant to the first point, it appears likely that there were some very significant late shifts and that these may have been driven by the sovereignty issue and fears of another referendum.
  4. The higher turnout suggests an emotionally engaging factor and the patterns point to a referendum and fear thereof as that factor. Clearly, some of the dominant issues cannot explain the PLQ surge (e.g., corruption and regime fatigue, student unrest of concerns with social justice, etc.).
  5. Turnout, while high, was differentially higher in a way that explains part of the prediction error in the surveys. In particular, it continues to be the case that seniors and boomers are punching above their already formidable demographic weight in the new political normal.
  6. The irony is that better polling might produce worse predictions continues to be the case. Ignoring young and first time voters would have produced “better” polling, as would ignoring the growing and systematically different subpopulations of cell phone only voters.
  7. It seems that the dynamics of each election include a large portion of idiosyncratic factors which render the task of prediction and figuring out who the likely voter is before he or she actually votes is exceedingly difficult
  8. As in the case of strenuous claims that voters will honestly and truly vote, the notion of “enthusiasm” as a predictor seems useless. Enthusiasm was not only a useless predictor of turnout, it actually dramatically worsened error (by about 50 per cent).
  9. More workmanlike indicators such as knowledge of one’s polling station location, past voting behaviour and weighting to past political choice do a reasonable job of reducing prediction errors but clearly late shifts and lying to pollsters were also occurring.
  10. It may also be the case that the vagaries of long weekend polling and other issues around response rates and selection biases at the stage of deciding whether or not to participate in the survey are contributing more noise than in the past.

On a final note, it is worth taking one last look at the impacts of views on sovereignty and their impacts on the Quebec election outcome. In short, support for the status quo is as low as it has ever been. Results are particularly ominous among youth, where the majority of respondents now support some form of sovereignty, while federalists are becoming increasingly confined to the aging boomers and seniors. From our results, it appears that it was the issue of sovereignty – or, more specifically, fears of another referendum – that propelled voter turnout to over 70 per cent, as none of the other major election issues (corruption, income inequality, etc.) can explain the better-than-expected performance of the PLQ.

In another interesting finding, the decline in support for Quebec federalism is not limited to Quebec. Throughout English Canada, there is growing support for Quebec sovereignty. The results are particularly troublesome in Alberta, where support for complete independence now rivals that in Quebec itself.

This is a potentially incendiary combination of public opinion and the current political landscape. In English Canada, there is a spurious conviction that the sovereignty movement is basically moribund and that the closeness of the Quebec election shows that the threat of another referendum is quite remote. Unlike the passionate anxiety that the referendum debate produced in English Canada in 1995, the current attitudes seem to range somewhere between indifference and hostility. In Quebec, the only party which attracted a dominantly federalist vote was the diminished PLQ at 31 points. All other parties contain supporters which lean towards sovereignty.

There is nothing in the tracking indicators suggesting an optimistic outcome in any future referendum for federalists. The fact that the PQ was elected with a clear commitment to pursuing a referendum seems to have caused no palpable anxieties in English Canada. The ruling Conservative Party of Canada is not connecting well at all with Quebeckers right now and the official NDP opposition is rooted in supporters from Quebec who are overall at least as comfortable with sovereignty as federalism. This unusual and potent mix of factors seems to at least merit some serious attention. While we are not saying that a referendum or a negative result for federalism are inevitable or even the most likely outcomes, we are saying that the current apathetic silence on this issue in English Canada is based on a comfortable but dangerous delusion about the real risks.

Click here for the full report: Quebec Election Retrospective (October 12, 2012)

Comments are closed.