Did the polls get Brexit wrong?

Swordfish does a lot of poll analysis at The Standard. Here he claims that Brexit was “Another Epic Fail for UK Pollsters”:

Another Epic Fail for UK Pollsters, Gov’nr. Especially the Phone-based polls.

Here are the final polls released immediately before (and, in two cases, during) the vote:

Populus (On-Line) Remain by 10 points
Com Res (Phone) Remain by 8 points
ORB (Phone) Remain by 8 points
BMG (Phone) Remain by 7.6 points
Ipsos Mori (Phone) Remain by 4 points
YouGov (On-Line) Remain by 2 points
Survation (Phone) Remain by 1 point

Opinium (On-Line) Leave by 1.3 points
TNS (On-Line) Leave by 2.4 points

Populus and Ipsos-Mori were the last two polls to be released.

I think that this highlights an epic fail on polls – looking at them and trying to use them as a prediction of an election or referendum result.

Polls have known statistical margins of error, typically plus or minus 3-4% with 95% confidence. This means that there is a one in twenty chance the poll will be more then 3-4% wrong AT THAT POINT IN TIME.

That last point is important. Polls are not a prediction of something in the future. They are an attempt to gauge opinion of voters prior to when the poll is published.

Swordfish states that two polls were released during the voting period, but the polling would have to have been done prior to the voting period, typically over several days.

Polls typically asked “if an election was held today how would you vote?”. No polls – or at least none of the above polls – attempt to measure how people have actually voted. Exit polls try to measure actual voting but none of the above are exit polls.

Apart from and related to the margin of error polls also have a proportion of ‘undecided’ and no response. These may be people who end up voting but didn’t know which way they would vote at the time of the poll or didn’t want to say how they would vote.

Also, it is well known that there can be significant voter shifts in the last few days leading up to an election or referendum. A significant number of people don’t decide how they will actually vote until voting day. Some don’t decide until they actually vote.

Polls don’t measure and can’t measure swing voting before the event.

It’s a common mistake to look at polls as a prediction of future voting. They are not guessing something that has not yet happened.

Polls are a measure of voting intent measured in the past with known margins of error, with known statistical confidence levels. Variations between poll results show they can’t all be  accurate measures of voter intent of the whole population.

There is also a critical unknown – last minute shifts in voter decision making.

It’s a common mistake for media and pundits to try to compare polls with actual elections and referendums.

A population of a thousand days or weeks before an election  is often going to give a different result to a population of 40 million people on referendum day.

I would be alarmed if pollsters tried to manipulate their polling methods and results in order to try and be the closest to the actual election result.

Polls don’t predict the future. Neither should they try.

Leave a comment

3 Comments

  1. Missy

     /  26th June 2016

    Just a quick note on the polls. On Referendum day some polls were conducted in the absence of exit polls, and most had the figures correct at 52-48, but for Remain, however leave.eu commissioned a private poll that had a sample size that was higher than the other polls, and their poll came back at 52-48 for Leave – the only poll to be on the money. So, not all polls were wrong, and the polling for the referendum was better than for the election, just the wrong result, and that has been put down in large to the pollsters calling the undecideds wrong, the assumption was the undecideds would swing to the status quo and vote Remain as in the Scottish Independence referendum.

    I also have another theory on why the polls were wrong, and for the swing to Remain in the last week, and that is the tragic death of Jo Cox. After her murder the vitriol directed at anyone voting leave was tinged with accusations of being murderers, as well as racists etc, so I think those voting leave just stopped telling pollsters, and kept quiet, and then when it was time went into the polling station and voted leave.

    Reply
  2. You make some valid points here , Polling isn’t a vote. British polling has been fairly off beam for at least 40 odd years , however in most elections they get the right result just wrong margin. When this happens no one really cares ( i think the uk polls in 1997 overestimated Labours vote more than they did in 2010 but no one cared as the win was never in doubt ) it time though for the media to be less poll obsessed and driven, but i suspect it will never happen !

    Reply
  3. Halliver

     /  26th June 2016

    When has a poll ever not got it wrong. The polls are rigged especially the colmar brunton b.s

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s