About EKOS Politics

We launched this website in order to showcase our election research, and our suite of polling technologies including Probit and IVR. We will be updating this site frequently with new polls, analysis and insight into Canadian politics. EKOS's experience, knowledge and sophisticated research designs have contributed positively to many previous elections.

Other EKOS Products

In addition to current political analysis, EKOS also makes available to the public general research of interest, including research in evaluation, general public domain research, as well as a full history of EKOS press releases.

Media Inquires

For media inquires, please contact: Frank Graves President EKOS Research Associates t: 613.235-7215 [email protected]

New Brunswick polling retrospective

By Earl Washburn, Senior Analyst

[Ottawa – September 16, 2020] In our New Brunswick provincial poll release last Saturday evening, prior to Monday’s provincial election, we correctly stated that the Progressive Conservatives were on track to winning the most seats, and that they were “on the cusp of a majority”. This was based on our poll, which suggested the Tories and the Liberals were tied at 32 points each. In reality, the Tories did much better than this; they won 39% of the popular vote, five points ahead of the Liberals. In terms of seats, they won 27, which I would later correctly predict would be their “high-water mark”. Based on how polarized the New Brunswick electorate has become, it didn’t matter if the two parties were tied or not, the PCs were likely going to win most of the Anglophone seats, and the Liberals were going to win most of the Francophone seats. And, that’s exactly what happened. I regret not doing an official seat prediction, because I would have only been wrong in Miramichi (where Liberal leader Kevin Vickers lost), and would have provided some solace for what turned out to be an anxious election night. I knew that while we had the two parties tied in our poll, it still represented a large enough drop in Liberal support to suggest the Tories would win nearly all of their battleground ridings.

On the face, it appears as though our poll missed the mark, by suggesting a tied race, when the Tories in fact won the popular vote by five points. And when it comes to the Tories, we did in fact miss the mark, under-polling them by seven points, which was outside the margin of error of 4%. However, we were within the margin of error for all of the other parties. We under-polled the Liberals by two points and the Greens by three points, while we over-polled the People’s Alliance by one point and the NDP by two, for a total error of 15 points. This error is no worse than other polls released the weekend before the election.

So, how did we underestimate the Tory vote so much? There may be a few factors at play. The biggest factor may have been how the data was weighted. As our poll was crowdfunded, we made every effort to be as transparent as possible about the process, as for this survey, the public was our client. We even released a data file, so that the public could inspect it, and analyze it. It is very clear from the data file that we had far more respondents saying they would vote Progressive Conservatives than Liberal. It was through the weighting that brought the Liberals up, and the Tories down to a tie. This is because younger voters and Francophone voters were more likely to indicate they were going to vote Liberal, while older voters and Anglophone voters were more likely to indicate they were going to vote Tory. Young voters and voters living in the Francophone north responded to our poll at a lower than desired rate, and were weighted up as a result, putting the Liberals and PCs into a tie. Curious about the affects of our age weight, we decided to re-run our tables without it, and lo-and-behold, the results were almost identical to the popular vote: PC 38%, Liberal 32%, Greens 15%, People’s Alliance 9% and NDP 3%. Does this mean we should have dropped the age weight altogether? Certainly not, but looking more into age weights is something to consider when conducting these polling exercises. Most polling firms weight age based on census proportions, while I prefer to weight based on past turnouts. For this poll, the data were weighted based on the estimated 2015 federal election turnout by age in New Brunswick. According to the 2016 Census, 23% of the adult population of New Brunswick is between 18 to 34, while 21% of the 2015 New Brunswick electorate was under 35. Admittedly, not a huge difference, and considering the 2015 election saw a huge turnout among young people, it is unlikely that 21% of voters in Monday’s provincial election were under 35. (Addendum: Recently, Elections Canada published their estimates from the 2019 federal election, which put the under 35 proportion of the electorate at just 18%) Plus the province is getting older – in 2006, 25% of the adult population was under 35, two points more than 2016. All this to say, it would not be surprising if the under 35 share of the electorate was far below the 20% mark in Monday’s election. Additionally, this group is becoming harder and harder to poll accurately. Most young people live in cell phone only households, and few will answer their phones for unsolicited calls. If an unreliable group is getting weighted up two, three or more times in a sample, their results can skew the data.

Another factor that could have affected the results was the fact that we did not ask respondents if they had already voted, and then ask how they voted. Due to the pandemic, over a third of voters cast their ballots in advance, and these voters may have skewed older, and therefore more conservative. Our vote intention question expressly asked who respondents were going to vote for on September 14th, election day. If you had already voted, how do you answer that question? In our election polling, we almost always ask if people had already voted, but because this poll was a quick add-on to our recruitment survey, we did not have the resources to add additional questions.

Our poll also pegged ‘other parties’ at 4%, when in fact other parties and independents combined for just 0.2% of the vote. Only 13 candidates for other parties or independents actually ran in the election, so most people who said they were voting for one of these parties did not end up doing so. It may be prudent in the future to ask second vote intention, and apply second vote intention to respondents who live in ridings who lack candidates of their desired party of choice.

And finally, maybe our poll was bang on all along, and undecideds just overwhelmingly broke for the “devil they know” in the voting booth. Our poll stated that 18% of voters were undecided, which is a lot, and could have shifted a lot over the last two days before the campaign. Undecideds tended to be older, Anglophone, male, and high school educated, all traditional PC voting demographics. These voters could have been upset with the government for having to vote during the pandemic, but ultimately decided they didn’t want to change the course.

Considering this poll was funded by the public, we feel terrible that we missed the mark on the vote share for the PCs. When it comes to polling and predicting elections in general, we aim for perfection and always want to ‘get it right’. And, one can only do that by identifying how they can improve. By providing this retrospective, we hope that we have learned the right lessons from this exercise, and have outlined how we can improve our polling in the future.

Leave a Reply