Rogue polls versus statistics

It’s common for politicians to claim that unfavourable polls are inaccurate (and nearly as common for them to accept favourable polls as ok).

Gerry Brownlee went as far as claiming a Newshub/Reid Research poll published on Monday was ‘rogue’.

RNZ: Gerry Brownlee questions methodology used in latest Newshub Reid Research poll

The latest Newshub Reid Research poll, released last night, has put the Labour Party on 60.9 percent and National on 25.1 percent, as the election draws closer.

The National Party released a statement just one minute before the news of the poll, dismissing it as rogue.

“I don’t believe it at all, I think it’s entirely out of kilter, it’s absolutely opposite to what we’re hearing in the electorates. The poll itself doesn’t go anywhere near where our polling is, the polling itself is clearly wrong,” party leader Judith Collins said.

National’s election campaign chair and deputy leader Gerry Brownlee told Morning Report that he meant no disrespect to the people who participated or those at Reid Research, but questioned the methodology being used.

“[The methodology used] potentially could not be random. When they applied that methodology, you’re going through selecting people who meet certain criteria that you want to have inside your polls – age groups and diversity, but that doesn’t mean you are always getting a truly random sample of what people are thinking politically.”

He reiterated the same message he had from last night, that statistically one in 20 polls would be wrong and that this was that one.

The poll has a margin of error of 3.1 percent, and was done between 16-24 July with 1000 people surveyed – the majority by phone and the remainder via an internet panel.

One of the problems with Brownlee’s claims is that while statistically a 1 in 20 poll may be outside the margin of error it is very likely to be 10% outside the margin of error. It would be much more likely to be just 0.1% outside the margin of error, or 1% outside.

According to statistical methods with the 95% confidence used is there is a 95% (19 in 20) the 25.1% result for National will be between 22.0% and 28.2%, and a 1 in 20 chance it will be outside this range. But the chances of it being 35% (or 15%) are very slim.

National leaked an internal poll result of 36% (but gave no details about polling period or sample size) – this means there is a 95% chance of it of actually being between 33% and 39%.

The 1 News/Colmar Brunton poll published on Thursday had a different polling period and a different result.

It was published as 32% with a margin of error of 3.1% (at 50%, it reduces the further you get from 50%). But that’s a rounded result, it could have been anywhere between 31.51% and 32.49%.

Accounting for the margin of error that’s a 95% confidence range somewhere between about 28.5% and 35.5%, with a 1 in 20 chance it is outside this.

Labour were published as 53%, but that’s a 95% confidence range somewhere between about 49.5% and 56.5%, still a big lead over National.

So any poll is quite approximate, despite how Newshub and 1 News try to portray their results.

Political news will affect who people think they may vote for. Sensationalised news of poll results is also likely to affect voter decisions.

And these poll results are already out of date. The Colmar Brunton poll published on Thursday:

  • Interviewing took place from Saturday 25 to Wednesday 29 July 2020.
  • Sunday (50% of sample size target was reached on this day).

So political news (including the Monday Reid Research poll) and social contact through the week would barely be reflected in the Colmar poll.

Brownlee making a fuss about a poor poll result drew more attention (some negative) to the result, but will probably only play a very small in the next poll.

Rogue MPs are a much bigger deal than rogue polls.

Polls are a useful but very approximate indicator of voter preferences in the past.

Colmar Brunton poll – July 2020

The latest 1 News/Colmar Brunton poll:

  • Labour 53% (up 2)
  • National 32% (down 6)
  • Greens 5% (down 1)
  • ACT 4.8% (up 1.7)
  • NZ First 2 (up 0.2)
  • New Conservatives 1.2%
  • Maori Party 1%
  • Don’t know/refused 14%

Still obviously very good for Labour.

Bad for National but nowhere near as bad as the Reid Research poll (25%). They could improve more from there but look a long way off challenging Labour.

Probably at National’s expense ACT are in a very good place for them.

Greens are still hovering around the threshold which for them having no electorates is high risk.

Further evidence that voters are giving up on Winston Peters and his party.

Preferred Prime Minister:

  • Jacinda Ardern 54% (no change)
  • Judith Collins 20% (up 18)
  • Winston Peters 1% (down 1)
  • David Seymour 1% (up 0.2)
  • Christopher Luxon 1%
  • Billy Te Kahika 1%

That’s a fairly good result for Collins considering how bad National has been over the last few weeks, and how uneven her performance has been.

On trust:

  • Jacinda Ardern: yes 82%, no 16% = +68
  • David Seymour: yes 48%, no 36% = +12%
  • Judith Collins: yes 47%, no 45% = +2
  • James Shaw: yes 47%, no 31% = +16
  • Marama Davidson: yes 44%, no 34% = +10
  • Winston Peters: yes 34%, no 59% = -25

Judith Collins: Approve 50%, Disapprove 23 = +27
– Todd Muller got +10 in June 2020, Simon bridges -40 in May 2020

Polling was done from Saturday 25th to Wednesday 29th July.

More details will come out in while.

Another awful poll for National (and great for Labour)

Today’s 1 News/Colmar Brunton poll is bad news for National, and very similar to the Newshub/Reid Research poll out earlier this week and also a recent leaked UMR poll:

  • Labour 59%
  • National 29%
  • Greens 4.7%
  • NZ First 3 %
  • ACT 2%
  • Maori Party 1%
  • The Opportunities Party 1%

Refused to answer 5%, undecided 11%. Fieldwork conducted 16-20 May.

The ‘preferred Prime Minister was as bad (and exceptional for Ardern):

  • Jacinda Ardern 63% (+21 from last October))
  • Simon Bridges 5% (-6)
  • Judith Collins 3%
  • Winston Peters 1% (-2)
  • Nikki Kaye 0.4%
  • Todd Muller 0.2%

Notable also is that both Greens and NZ First are under the threshold, and Peters is also dropping to negligible ‘preferred’.

So Labour very strong, Act ok (if Seymour can hold Epsom), Greens in the danger zone and NZ First/Peters really struggling.

Of course tomorrow’s National leadership showdown adds importance to this result.

David Farrar very quick off the mark with the poll results but no commentary: Latest poll
(but his Curia poll average hasn’t been updated for three months).

And Greg at The Standard was ready to rumble: The Colmar Brunton poll

High public approval of NZ Government handling of Covid-19 pandemic

Colmar Brunton polling shows strong public support for the Government handling of the Covid-19 pandemic, with 87% of people saying the approve or strongly approve, with only 9% disapproving.

A poll was done from 3-5 April after the Level 4 lockdown had started, and another has been done from 20-21 April, after the move to Level 3 lockdown was announced on Monday.

The latest poll: How much do you approve or disapprove of how your Government is responding to the Covid-19 pandemic?

  • Strongly approve 68% (up from 55%)
  • Somewhat approve 19% (down from 29%)
  • Total approve 87% (up from 84%)
  • Somewhat disapprove 5% (down from 6%
  • Strongly disapprove 3% (no change)
  • Total disapprove 8% (down from 9%)
  • Neither approve nor disapprove 4% (down from 6%)
  • Don’t know 1% (up from 0)

Colmar Brunton, Margin of error +/- 4%
The poll was conducted via 601 online interviews with New Zealanders over the age of 18 between April 20 and 21.

Stuff: The Government’s Covid-19 lockdown measures have overwhelming public support, according to a poll

Colmar Brunton pollster Edward Langley said New Zealand seemed to be seeing “something special” in the number of new Covid-19 cases each day.

“People feel there’s light there at the end of the tunnel which other nations haven’t seen”.

“I think we are seeing something special. We are setting aside our party political affiliations and we’re getting behind the Government”.

New Zealand support is much higher than G7 countries:

  • Average for G7 countries 50% support their Government (down from 54% two weeks ago)
  • Canada 74% support
  • France 43% (down from 61%)
  • USA 46% support (down from 52%)

France continues with a high death rate currently running at over 500 deaths per day.

USA has the highest total cases (866,148) and deaths (48,868). Deaths increased by 2,341 yesterday. Support there is dropping along with President Trumps approval (and he is publicly arguing with one of his top health officials again today).

Cannabis legalisation polls and trends

Two recent polls suggested majorities opposing cannabis legislation, but one poll has a more supporting change, especially “When New Zealanders Have More Information”.

And data from Canada where cannabis is already legal suggests fewer young people are now using cannabis.

1 News (14 February): New Zealanders likely to vote against cannabis legalisation – 1 NEWS Colmar Brunton Poll

Those polled were asked, ‘At this stage, do you think you will vote for cannabis to be legalised, or for cannabis to remain illegal?’

Remain illegal – 51%
Legalise cannabis – 39%
Will not vote – 1%
Don’t know / refused – 9%

The groups of people who were more likely than average to intend to vote against legalising cannabis were Asian New Zealanders, National Party supports and people aged 55 and over.

Those who were more likely to intend to vote for legalisation were Green Party supporters, women aged 18 to 34, Māori, people with annual household incomes between $30,001 to $70,000 and Labour Party supporters.

Between February 8 to 12, 1004 eligible voters were polled by landline (402) and mobile phone (602). The maximum sampling error is approximately ±3.1%-points at the 95% confidence level.

November/December 1 NEWS Colmar Brunton poll saw 49 per cent against legalisation and 43 per cent for, with the June 2019 poll seeing 52 per cent of people against and 39 per cent for legalisation.

In the October 2018 1 NEWS Colmar Brunton Poll, the results were slightly more in favour of legalisation than against, with nearly half wanting the drug to be legal. Forty-six per cent of Kiwis were in favour of legalisation and 41 per cent were against.

In the July 2017 1 NEWS Colmar Brunton poll, 47 per cent were in favour of cannabis legalisation and 41 per cent were opposed.

Newshub (18 February): New poll shows support for both recreational cannabis and euthanasia dropping

The latest Newshub Reid-Research poll asked the referendum question the public will be asked in the referendum this election: do you support the proposed Cannabis Legalisation and Control Bill?

  • 39.4 percent said ‘yes’
  • 47.7 percent said ‘no ‘
  • 11.6 percent said ‘don’t know’

The Bill would make recreational cannabis legal for over 20s, with restrictions.

Since the last time Newshub polled on this in June, despite additional details released in December, more people have moved from the ‘yes’ camp to the ‘don’t knows’.

Very few voters will know what Cannabis Legalisation and Control Bill actually proposes.

(Note that the euthanasia part of the headline is a bit misleading, the result was 61.9% in favour, 23.7% against).

NZ Drug Foundation (21 February): Poll Shows Support For Cannabis Legalisation When New Zealanders Have More Information

Survey results released today by the Helen Clark Foundation and the New Zealand Drug Foundation show that support for cannabis legalisation grows when people know more about the proposed legislation.

When respondents were asked how they would vote in September’s referendum based on what they already know:

  • 46% said they would vote for the legalisation of cannabis
  • 44% said they would vote against it
  • 10% undecided

When people were then told more about the limits and restrictions on cannabis use and sale in the proposed legislation:

  • support for legalisation increased to 50%
  • opposition decreased to 42%
  • 8% undecided

Fieldwork for the survey was conducted between 22 January and 3 February 2020. The maximum sampling error for a sample size of 1000 at the 95% confidence level is ± 3.1%.

That looks promising for those wanting change, but there is likely to be a battle of information and misinformation.

“These results suggest New Zealanders are likely to support a sensible approach to cannabis harm reduction when they have accurate information about what is being proposed,” said Holly Walker, Deputy Director of the Helen Clark Foundation.

“The details matter. Armed with the facts, voters see that putting in place rules and enforcing these is better than the status quo.”

New Zealand Drug Foundation saw similar results in research commissioned in November last year. “When initially asked how they would vote, participants were evenly split, with around 14 percent undecided. Once the participants were given more information on the legislation, we saw stronger support for a yes vote,” said Ross Bell, Executive Director, NZ Drug Foundation.

Over the last two months the proportion of undecided voters has dropped, following the release of the draft Cannabis Legalisation and Control Bill in December.

The draft legislation includes an age limit of 20, redistribution of tax into harm reduction, health and education programmes, a ban on all marketing and advertising of cannabis products, strict controls on the potency of cannabis, and other restrictions.

“When people learn about these proposed restrictions, they are more likely to support a law change,” said Ms Walker.

NZ Herald: Legalising cannabis: Supporters, opponents take swipes at each other as polls show knife-edge decision

The foundation said it showed more support for legalisation when voters were more informed, but Family First national director Bob McCoskrie said it was loaded to ask the same question either side of highlighting the proposed legal framework.

McCoskrie attributed the decline of the ‘yes’ vote to the strength of the ‘no’ campaign so far, including a 24-page pamphlet that had been delivered nationwide.

But Drug Foundation executive director Ross Bell said the downward trend in ‘yes’ support was because of “well-funded and relentless opposition scaremongering”.

He has asked supporters to donate funds to the ‘yes’ campaign, which was putting together a strategy that included billboards, TV advertising and social media.

McCoskrie responded by saying he was giving the public the “facts”, adding that he had little faith in the Prime Minister’s expert advisory panel, headed by her Chief Science Advisor Professor Juliet Gerrard.

The panel is putting together publicly-available information about the impacts of cannabis use, what changes have occurred overseas, and how applicable that might be in New Zealand.

So McCoskrie doesn’t like people being informed when being polled, but is keen to ‘inform’ people against the legislation.

Meanwhile (NZH):

New data from the Youth Insights Survey, published yesterday in the New Zealand Medical Journal, found that between 2012 and 2018, the proportion of Year 10 students who had tried the drug fell by more than a quarter.

“This was predicted, since cannabis trends in this age group are strongly associated with tobacco trends, and it was already known that smoking in Year 10 students had continued to decline since 2012,” said the study’s Otago University authors.

However, the authors note that other research shows cannabis use is increasing among New Zealand adults generally.

Past year use increased from 9 per cent in 2012/13 to 15 per cent in 2018/19 overall – and from 19 per cent to 29 per cent among 15 to 24 year olds, the age group with the highest cannabis usage.

The authors said there were likely two key reasons for the conflicting trends.

“Firstly, the average age at which young people are initiating risk behaviours, including cannabis use, has increased in recent years,” they wrote.

“Secondly, normalisation of cannabis use has been counteracted by decreasing prevalence and frequency of smoking and drinking in this age group.

“The evidence suggests that adolescents’ willingness to try cannabis has increased, but their opportunities for doing so have decreased due to less face to face time with friends and fewer drinking and smoking occasions.”

But statistics from Canada shows the opposite has happened there, with youth use of cannabis dropping significantly since legalisation and regulation.

And:

 

One News/Colmar Brunton poll – February 2020

The first One News/Colmar Brunton poll of the year hasn’t moved much, but on these numbers National+ACT could probably form a government.

Greens have moved towards the danger zone and NZ First slips further under the threshold.

  • National 46% (no change)
  • Labour 41% (up 1)
  • Greens 5% (down 1)
  • NZ First 3% (down 1)
  • ACT 2% (no change)
  • Maori Party 1% (no change)
  • New Conservatives 1% (no change)

Those are rounded to whole numbers so the small party results in particular are very approximate.

Polling ran through last week which included Simon Bridges ruling out doing anything with NZ First this election, and the Waitangi Day/week political manouvering.

Preferred Prime Minister:

  • Jacinda Ardern 42% (up 6)
  • Simon Bridges 11% (up 1)
  • Judith Collins 3% (down 1)
  • Winston Peters 3% (no change)
  • Don’t know 30%

Economic outlook

  • Optimism 40% (Up 4%)
  • Pessimism 34% (Down 1%)

Between February 8 to 12, 1004 eligible voters were polled by landline (402) and mobile phone (602). The maximum sampling error is approximately ±3.1%-points at the 95% confidence level.

For party support, percentages have been rounded up or down to whole numbers.

The data has been weighted to align with Stats NZ population counts for age, gender, region, ethnic identification and mobile or landline access. 

https://www.tvnz.co.nz/one-news/new-zealand/1-news-colmar-brunton-poll-national-and-act-hold-numbers-form-government

1 News Colmar Brunton poll – December 2019

What is likely to be the last political poll of the year, from 1 News/Colmar Brunton, has national in a strong position, and ACT rewarded for David Seymour’s work on the End of Life Choice Bill.

  • National 46% (down 1)
  • Labour 39% (down 1)
  • Greens 7% (no change)
  • NZ First 4% (no change)
  • ACT 2% (up 1)
  • The Opportunities Party 1%
  • New Conservatives 1%
  • Maori Party 1%

Don’t know or refused to answer – 17%

(Results rounded to the nearest whole number so small party results and movements can be exaggerated) .

Preferred Prime Minister:

  • Jacinda Ardern 36% (down 2)
  • Simon Bridges 10% (up 1)
  • Judith Collins 4% (up 1)
  • Winston Peters 3% (down 1)

Between November 23-27, 1006 eligible voters were polled by landline (504) and mobile phone (502). The maximum sampling error is approximately ±3.1%-points at the 95% confidence level. The data has been weighted to align with Stats NZ population counts.

Recent polls charted here: http://www.polled.co.nz/

Two political polls with similar results

Newshub released a Reid Research a poll on Sunday with ridiculous headlines and claims. 1 News released a Colmar Brunton poll last night with less dramatic but still over the top claims. Polls are just polls, especially this far from an election, but they try to get value from the expense of polling by making stories out of them that aren’t justified.

Last time the two polled the biggest talking point was how different their results were. The Reid Research poll was regarded as an outlier, being quite different to any other polls this term.

The most notable thing about the polls this time is that the results are very similar, taking into account margins of error of about 3% for the larger results, and the fact that Colmar results are rounded to the nearest whole number.

  • National: RR 43.9% (+6.5%), CB 47% (+2)
  • Labour: RR 41.6% (-9.2), CB 40% (-3)
  • Greens: RR 6.3% (+0.1), CB 7% (+1)
  • NZ First: RR 4.0% (+1.2), CB 4% (+1)
  • ACT: RR 1.4% (+0.6), CB 1% (-)
  • TOP: RR 1.1% (+1.0), CB 1% (-)
  • Maori Party: RR 0.7% (+0.2), CB 1% (-)

I don’;t think it’s surprising at this stage to see National a bit ahead of Labour, Labour has had a mixed month or two and is struggling to make major progress due to the restraint of coalition partner NZ First.

Green support looks at a safe level, but is well below what they were getting last term (about half).

NZ First are still polling below the threshold and will be in a battle to stay in Parliament.

Is is fairly normal these days there are a number of borderline governing scenarios with these numbers, with National+ACT and Labour+Greens thereabouts but not certainties.

A lot may depend on whether NZ First make the threshold or not next election. Both other times they have been in a coalition government they have lost support at the next election.

Trends from Opinion polling for the next New Zealand general election (Wikipedia):

That shows the last Reid Research anomaly well.

Preferred Prime Minister:

  • Jacinda Ardern: RR 38.4% (-10.6), CB 38% (-3)
  • Simon Bridges: RR 6.7% (+2.5), CB 9% (+3)
  • Judith Collins: 5.2% (-1.9), CB 5%
  • Winston Peters: CB 4%

Ardern a bit down, Bridges a bit up but still a big difference.

Newshub also did a poll on performance:

  • Ardern: performing well 62.4%, performing poorly 23.1%
  • Bridges: performing well 23.9%, performing poorly 52.7%

UPDATE: 1 News/Colmar Brunton have also started asking a similar question:

  •  Ardern handling her job as Prime Minister:  +33
    approve 62%
    disapprove 29%
    don’t know or refused 8%
  • Bridges’ handling his job as National Party leader: -22
    approve 29%
    disapprove 51%
    don’t know or refused 20%

Ardern performance is well above her party support, while Bridges is well below National support (about half).

  • Newshub-Reid Research Poll was conducted between 2-9 October 2019.
    1000 people were surveyed, 700 by telephone and 300 by internet panel
  • 1 News-Colmar Brunton poll conducted between 5-9 October
    1008 eligible voters were polled by landline (502) and mobile phone (506)

So both now rely on some polling by something other than landline, Reid Research 30% by internet panel and Colmar Brunton 50% by mobile phone.

1 News link here.

Newshub/Reid Search links here and here.

The Newshun headline says “Jacinda Ardern, Labour take massive tumble in new Newshub-Reid Research poll” but a more accurate description would have been “Newshub poll looks more likely following last rogue poll”. It wasn’t a massive tumble for Ardern, more like a large correction by Reid Research.

Two polls suggest a movement against cannabis law reform

While there are more options than legalisation of cannabis, nd we don’t know what we will be voting on in next year’s referendum, that’s the question asked by two polls.

Newshub/Reid Research: Should we legalise Cannabis?

  • No – 48%
  • Yes – 41.7%
  • Don’t know – 10.4%

1 News/Colmar Brunton: At this stage, do you think you will vote for cannabis to be legalised, or for cannabis to remain illegal?

  • Remain illegal – 52%
  • Legalise – 39%
  • Unsure/refused – 9%

These results are based on largely uninformed opinions. We don’t know what we will be voting on. One thing is certain – there won’t be total legalisation. Current proposals being considered by Parliament are for limiting legal use to 20 years of age and over, and very limited means of obtaining cannabis for use.

There is a lot of deciding still to happen in Parliament, and a lot of lobbying and campaigning. Some of the campaigning so far has been inaccurate and comes close to scaremongering misinformation.

When we know what we will be voting on we can make our choices.

Until the pollsters know what the vote will be on all they can do is give us a rough idea of possible outcomes.

 

 

Pollsters comment on vastly different poll results

I think there has been much ado about a couple of poll results, with much more made of the results than they deserve, The polls are of interest but not of great importance in the whole scheme of things.

But it’s worth listening to what the polling companies think.

The Spinoff – Two polls. Two wildly differing results. What happened?

Last night’s polling results had little in common, with the only consistency across both being that Simon Bridges is a very unpopular party leader. In short: the Colmar Brunton poll showed National up to 44%, and Labour dropping to 42%. It showed Jacinda Ardern as preferred prime minister for 45%. The Newshub Reid poll showed National at 37.4%, and Labour at 50.8%. It showed Ardern as preferred PM for 49%.

The Colmar Brunton explanation:

…the divergence doesn’t necessarily mean one is wrong. “Without digging into what Reid’s methodology is and what the details are it’s hard to comment on why there’s a difference,” said Jason Shoebridge, CEO of Kantar, Colmar Brunton’s parent company.

I asked him why he thought there was a difference in the results, and it turns out there’s a simple explanation: “Reid use an online methodology as well as landlines, and we just use landlines and mobile phones. Then there’s a difference of when we were collecting the data – we were collecting later than they were.” Colmar Brunton was conducting their research from the 4th to the 8th June, and Newshub-Reid Research did theirs from the 29th May to the 7th June.

The last Colmar Brunton poll was taken in the days following the Christchurch terror attacks on March 15th, an event for which Ardern was near-universally praised for her handling, so it’s not surprising to see Labour’s popularity declining closer to what it was before the attack. With this in mind, alongside the recent furore around the wellbeing budget, Ardern’s result in both polls could be seen as a win.

One way to tell the accuracy of a poll is how it fits into the over-arching trends, and in general, polls prior to this showed Labour going up and National going down. Shoebridge believes the only true measure of a poll’s accuracy is an election.

that should be impressed on the media who promote their polls as major news.

“Where the stress really comes in is on election night – that’s the real test,” he explained. Political opinion polling is the most high profile work the company does, even though it’s only a small proportion of their business. “We always want it to be as accurate as possible, and we’re confident in our numbers.”

Reid Research was confident but offered no explanation.

Reid Research was not at liberty to release more information to us due to its commercial relationship with Newshub, but said it was confident its poll was correct.

A poll with a small sample cannot be ‘correct’. The only correct polls are elections. Everything else is an approximation with well known margins of error and possibilities of greater errors.

It is almost certain that if another poll was taken this week it would have a different result.