Rogue polls versus statistics

It’s common for politicians to claim that unfavourable polls are inaccurate (and nearly as common for them to accept favourable polls as ok).

Gerry Brownlee went as far as claiming a Newshub/Reid Research poll published on Monday was ‘rogue’.

RNZ: Gerry Brownlee questions methodology used in latest Newshub Reid Research poll

The latest Newshub Reid Research poll, released last night, has put the Labour Party on 60.9 percent and National on 25.1 percent, as the election draws closer.

The National Party released a statement just one minute before the news of the poll, dismissing it as rogue.

“I don’t believe it at all, I think it’s entirely out of kilter, it’s absolutely opposite to what we’re hearing in the electorates. The poll itself doesn’t go anywhere near where our polling is, the polling itself is clearly wrong,” party leader Judith Collins said.

National’s election campaign chair and deputy leader Gerry Brownlee told Morning Report that he meant no disrespect to the people who participated or those at Reid Research, but questioned the methodology being used.

“[The methodology used] potentially could not be random. When they applied that methodology, you’re going through selecting people who meet certain criteria that you want to have inside your polls – age groups and diversity, but that doesn’t mean you are always getting a truly random sample of what people are thinking politically.”

He reiterated the same message he had from last night, that statistically one in 20 polls would be wrong and that this was that one.

The poll has a margin of error of 3.1 percent, and was done between 16-24 July with 1000 people surveyed – the majority by phone and the remainder via an internet panel.

One of the problems with Brownlee’s claims is that while statistically a 1 in 20 poll may be outside the margin of error it is very likely to be 10% outside the margin of error. It would be much more likely to be just 0.1% outside the margin of error, or 1% outside.

According to statistical methods with the 95% confidence used is there is a 95% (19 in 20) the 25.1% result for National will be between 22.0% and 28.2%, and a 1 in 20 chance it will be outside this range. But the chances of it being 35% (or 15%) are very slim.

National leaked an internal poll result of 36% (but gave no details about polling period or sample size) – this means there is a 95% chance of it of actually being between 33% and 39%.

The 1 News/Colmar Brunton poll published on Thursday had a different polling period and a different result.

It was published as 32% with a margin of error of 3.1% (at 50%, it reduces the further you get from 50%). But that’s a rounded result, it could have been anywhere between 31.51% and 32.49%.

Accounting for the margin of error that’s a 95% confidence range somewhere between about 28.5% and 35.5%, with a 1 in 20 chance it is outside this.

Labour were published as 53%, but that’s a 95% confidence range somewhere between about 49.5% and 56.5%, still a big lead over National.

So any poll is quite approximate, despite how Newshub and 1 News try to portray their results.

Political news will affect who people think they may vote for. Sensationalised news of poll results is also likely to affect voter decisions.

And these poll results are already out of date. The Colmar Brunton poll published on Thursday:

  • Interviewing took place from Saturday 25 to Wednesday 29 July 2020.
  • Sunday (50% of sample size target was reached on this day).

So political news (including the Monday Reid Research poll) and social contact through the week would barely be reflected in the Colmar poll.

Brownlee making a fuss about a poor poll result drew more attention (some negative) to the result, but will probably only play a very small in the next poll.

Rogue MPs are a much bigger deal than rogue polls.

Polls are a useful but very approximate indicator of voter preferences in the past.

A rouge seal

This story has remained misspelled all day at NZ Herald.

Seal bites boy

A rouge seal attacked a 10-year-old in Mahia yesterday

A rouge seal attacked a 10-year-old in Mahia yesterday

I guess it’s mouth is pink, that’s close-ish to rouge.

The Herald must have a rogue editor.

Here’s a rouge seal:

Also a rouge Seal apparently:

A red seal:

Turei on ‘rogue poll’

Green co-leader Metiria Turei has commented on the latest poll result to Radio NZ – Greens play down latest poll.

The Green Party is playing down the significance of the latest political poll, saying it’s too soon to know whether it’s real change or just an anomaly.

The One News Colmar Brunton Poll has the Greens on eight percent support, down from 13 percent in the same poll in October.

The Greens haven’t scored that low since 2011.

Co-leader Metiria Turei says she’s not concerned.

“The polls are going to jump around like mad in the beginning of this year. We’ve been polling on average about 12 percent since last election.

“So this is quite an unusual rogue poll, whether it is pattern we are yet to see.”

Unlike other claims of ‘rogue poll’ this is a fair call from Turei. I’m sure Turei and the Greens have some concerns but this could be a rogue poll for Greens.

Other recent polls had  Greens at 12 (Roy Morgan) and 10 (Fairfax). The margin of error at this level is about +/- 2 so 8% could be outside the margins of  recent polls, but there’s always a one in 20 chance of this happening in polls.

So we have to wait for more polls to see if this is an unusual variation, a temporary drop  or a downward movement to be concerned about.

Greens should hope it’s a rogue and results will return to recent ranges. The next Roy Morgan poll due out next week will give us a better idea.