Pollsters comment on vastly different poll results

I think there has been much ado about a couple of poll results, with much more made of the results than they deserve, The polls are of interest but not of great importance in the whole scheme of things.

But it’s worth listening to what the polling companies think.

The Spinoff – Two polls. Two wildly differing results. What happened?

Last night’s polling results had little in common, with the only consistency across both being that Simon Bridges is a very unpopular party leader. In short: the Colmar Brunton poll showed National up to 44%, and Labour dropping to 42%. It showed Jacinda Ardern as preferred prime minister for 45%. The Newshub Reid poll showed National at 37.4%, and Labour at 50.8%. It showed Ardern as preferred PM for 49%.

The Colmar Brunton explanation:

…the divergence doesn’t necessarily mean one is wrong. “Without digging into what Reid’s methodology is and what the details are it’s hard to comment on why there’s a difference,” said Jason Shoebridge, CEO of Kantar, Colmar Brunton’s parent company.

I asked him why he thought there was a difference in the results, and it turns out there’s a simple explanation: “Reid use an online methodology as well as landlines, and we just use landlines and mobile phones. Then there’s a difference of when we were collecting the data – we were collecting later than they were.” Colmar Brunton was conducting their research from the 4th to the 8th June, and Newshub-Reid Research did theirs from the 29th May to the 7th June.

The last Colmar Brunton poll was taken in the days following the Christchurch terror attacks on March 15th, an event for which Ardern was near-universally praised for her handling, so it’s not surprising to see Labour’s popularity declining closer to what it was before the attack. With this in mind, alongside the recent furore around the wellbeing budget, Ardern’s result in both polls could be seen as a win.

One way to tell the accuracy of a poll is how it fits into the over-arching trends, and in general, polls prior to this showed Labour going up and National going down. Shoebridge believes the only true measure of a poll’s accuracy is an election.

that should be impressed on the media who promote their polls as major news.

“Where the stress really comes in is on election night – that’s the real test,” he explained. Political opinion polling is the most high profile work the company does, even though it’s only a small proportion of their business. “We always want it to be as accurate as possible, and we’re confident in our numbers.”

Reid Research was confident but offered no explanation.

Reid Research was not at liberty to release more information to us due to its commercial relationship with Newshub, but said it was confident its poll was correct.

A poll with a small sample cannot be ‘correct’. The only correct polls are elections. Everything else is an approximation with well known margins of error and possibilities of greater errors.

It is almost certain that if another poll was taken this week it would have a different result.

Leave a comment


  1. Duker

     /  11th June 2019

    “Reid use an online methodology as well as landlines, ”

    Yes , I thought there was a fundamental different way they went about it.

    “Shoebridge believes the only true measure of a poll’s accuracy is an election.”

    Which is true , and Reid was the best overall and smallest in their actual margins of error.

    This is bizarre from PG
    “A poll with a small sample cannot be ‘correct”

    The sample size is chosen to give a correct result with only relatively small margins of error.

    of course some would say , not everyone votes and we should have compulsory voting like Australia.
    Well a sample of 75% of electorate is going to be pretty well tiny margins of error.

  2. Pete is right, polls do need to be a reasonable size. I forget what the number is that a poll should be, but a small poll can be useless.

    The non-voters in Australia go into the booth and spoil the voting paper, wasting everyone’s time and money.

    • Duker

       /  11th June 2019

      For NZ around the current size is OK , (1000) doubling it doesnt reduce the margin of error much.
      Thats the whole point of polls having the smallest sample size that is mathematically valid, and doing them often so that trends are evident.
      NZ is a good spot for political polls as we have the primary party vote on which party to vote for. ( but we dont have compulsory voting)
      Other countries use electorates with first past the post where local factors come into play, or US where the electorate college numbers dont reflect country as a whole.
      Australia is a higher level of complication with preferential voting where the polls they to predict a winner by ‘two party preferred vote magic counting’

      • Kitty Catkin

         /  11th June 2019

        I can’t believe how undemocratic the US with millions being disenfranchised and their votes effectively stolen.

        It is unimpressive that the non-voters in Australia just waste their votes by spoiling the paper (I hope that they don’t whinge about the government)

        Our system is as good as any.

        I forget what the ideal size for a poll is.

        • Duker

           /  11th June 2019

          Not so much ‘spoiling ‘ the paper . Its just misunderstanding the ballot paper and not number all squares but not when you number the ‘partys only’ or number some of the candidates ( Senate). Then States have slightly different methods to the nationwide one.
          Winston Peters won his first election based on say 50 people not crossing out ALL the candidates who they didnt want to vote for .
          No its just ‘tick one’. Simple is best

          • Kitty Catkin

             /  11th June 2019

            No, this is deliberately spoiling the paper because the person doesn’t want to vote and does this as a protest.

          • Kitty Catkin

             /  11th June 2019

            It’s obvious when someone’s done it on purpose rather than making an error. Young people are more likely to deliberately spoil the paper, it seems.

            • Duker

               /  11th June 2019

              No again.
              ” The rate of informal voting increased from 5.0% to 5.5%, but those seeking to tie this to an outbreak of apathy are probably thinking too hard. Antony Green notes the shift was peculiar to New South Wales, and puts this down to the proximity of a state election there, maximising confusion arising from its system of optional preferential voting.”- Poll Bludger

            • Kitty Catkin

               /  11th June 2019

              Don’t believe it then.

              I can’t be bothered to try to convince you that people deliberately do this, although there is evidence that they do.

              Crossing out every name or none or writing/drawing an obscenity is hardly likely to arise from confusion, but suit yourself.

  3. Duker

     /  11th June 2019

    More crazy thinking
    ‘Heather du Plessis-Allan: Both polls were wrong – the system is worthless’

    they can never be the same, thats how polls work.
    That the difference between them is ‘divergent’ just means the 1 in 20 poll error is occurring . Thats how polls work
    Look in the mirror HPDA.
    examples of her nonsense thinking that means she is a complete ignoramus about how polling works.
    Because she is ‘renovating’ and is going to remove ‘land lines’ ( its another story that land lines are replaced by fibre optic cables which phones can use too)

    “But this is the future, and the present. You can’t reach people on landlines and there’s no White Pages for cell phones, so how do you find the numbers to reach people to get a representative sample for a poll?

    Stupid stupid stupid . The computer randomly call numbers, they dont have people looking up phone books

    She says people dont tell pollsters what they think – Trump? Where does she get nonsense like this.
    Clinton won the national vote like the polls said she would. The US polling is even more meticulous they call people randomly who have previously voted . Its public information apparently.
    If lower population parts of two states . Florida and Michigan that jut into other states ( Wisconsin and Alabama and vote like them) were excluded from the majority population areas of those states Clinton would have won both states and would be President .
    It was that close. A small overall margin in two states, where the Trump vote was higher in small but distinctive areas ( Florida Pan handle and Michigan Upper peninsula, they even have names !)

  4. Duker

     /  11th June 2019

    What are your numbers? I’m sure it happens but is it 1% or 4%?
    There is something like 10% who aren’t on the roll, these are most likely to be uninterested in politics and voting. They don’t even go to the polls
    From what I can find back in 2001 they looked into informals in more detail.
    22% of informals back then were blank and 6% had a slogan or a mark. This 6% would be the spoilt ones you refer too.
    So that was 6% of the informals. Most of course we’re confusion over a complicated voting system between federal state and local that’s the same but different.
    Eg13% used ticks and crosses.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s