Rogue polls versus statistics

It’s common for politicians to claim that unfavourable polls are inaccurate (and nearly as common for them to accept favourable polls as ok).

Gerry Brownlee went as far as claiming a Newshub/Reid Research poll published on Monday was ‘rogue’.

RNZ: Gerry Brownlee questions methodology used in latest Newshub Reid Research poll

The latest Newshub Reid Research poll, released last night, has put the Labour Party on 60.9 percent and National on 25.1 percent, as the election draws closer.

The National Party released a statement just one minute before the news of the poll, dismissing it as rogue.

“I don’t believe it at all, I think it’s entirely out of kilter, it’s absolutely opposite to what we’re hearing in the electorates. The poll itself doesn’t go anywhere near where our polling is, the polling itself is clearly wrong,” party leader Judith Collins said.

National’s election campaign chair and deputy leader Gerry Brownlee told Morning Report that he meant no disrespect to the people who participated or those at Reid Research, but questioned the methodology being used.

“[The methodology used] potentially could not be random. When they applied that methodology, you’re going through selecting people who meet certain criteria that you want to have inside your polls – age groups and diversity, but that doesn’t mean you are always getting a truly random sample of what people are thinking politically.”

He reiterated the same message he had from last night, that statistically one in 20 polls would be wrong and that this was that one.

The poll has a margin of error of 3.1 percent, and was done between 16-24 July with 1000 people surveyed – the majority by phone and the remainder via an internet panel.

One of the problems with Brownlee’s claims is that while statistically a 1 in 20 poll may be outside the margin of error it is very likely to be 10% outside the margin of error. It would be much more likely to be just 0.1% outside the margin of error, or 1% outside.

According to statistical methods with the 95% confidence used is there is a 95% (19 in 20) the 25.1% result for National will be between 22.0% and 28.2%, and a 1 in 20 chance it will be outside this range. But the chances of it being 35% (or 15%) are very slim.

National leaked an internal poll result of 36% (but gave no details about polling period or sample size) – this means there is a 95% chance of it of actually being between 33% and 39%.

The 1 News/Colmar Brunton poll published on Thursday had a different polling period and a different result.

It was published as 32% with a margin of error of 3.1% (at 50%, it reduces the further you get from 50%). But that’s a rounded result, it could have been anywhere between 31.51% and 32.49%.

Accounting for the margin of error that’s a 95% confidence range somewhere between about 28.5% and 35.5%, with a 1 in 20 chance it is outside this.

Labour were published as 53%, but that’s a 95% confidence range somewhere between about 49.5% and 56.5%, still a big lead over National.

So any poll is quite approximate, despite how Newshub and 1 News try to portray their results.

Political news will affect who people think they may vote for. Sensationalised news of poll results is also likely to affect voter decisions.

And these poll results are already out of date. The Colmar Brunton poll published on Thursday:

  • Interviewing took place from Saturday 25 to Wednesday 29 July 2020.
  • Sunday (50% of sample size target was reached on this day).

So political news (including the Monday Reid Research poll) and social contact through the week would barely be reflected in the Colmar poll.

Brownlee making a fuss about a poor poll result drew more attention (some negative) to the result, but will probably only play a very small in the next poll.

Rogue MPs are a much bigger deal than rogue polls.

Polls are a useful but very approximate indicator of voter preferences in the past.

Leave a comment

31 Comments

  1. John J Harrison

     /  2nd August 2020

    Gerry was right.
    But the best news is that Peters is gone burger – period !
    Fantastic.

    Reply
  2. Alan Wilkinson

     /  2nd August 2020

    The confidence estimates assume random sampling. If the sampling is non random all bets are off.

    Reply
    • Maggy Wassilieff

       /  2nd August 2020

      And that’s the rub…
      most folks who place weight on these polls have no idea what random sampling entails.

      If we looks at the methodology used by Newshub/Reid… we can see that having 70% of your sampling based on phone interviews and 30% on folks on internet consumer marketing panels… is far from a random representation of NZ’s voting public.
      http://www.reidresearch.co.nz/TV3+POLL+RESULTS.html

      The poll is really just sampling those folks that have the time and inclination to be sampled.

      Reply
      • Duker

         /  2nd August 2020

        So how do you explain the last polls get the actual election result- within the margin of error- then ?
        The random part comes from the random calling of phone numbers, why are you making out something else ?
        Brownlee was happy to claim a Stuff self selected ‘internet survey’ was more like the results they were getting….really a survey of people who read Stuff webiste

        National and the pollster Farrar played the same trickery last election when they didnt like the Colmar Brunton pre- election polls.

        Reply
        • Alan Wilkinson

           /  2nd August 2020

          People who answer random phone calls are not random.

          Reply
          • Duker

             /  2nd August 2020

            The numbers are randomly chosen by a computer to dial…so the households are random.
            Yes the sample who answer doesnt match the population exactly , but then neither does the people who vote.
            Thats why experience counts to to put the ‘weights’ in the right place, which would happen wheter National or Labour is leading.

            Even Brownlee, the campaign chairman wouldnt tell the caucus the actual results, they got a ‘smoothed number’… they arent normally even told that , just trends

            For a non random sample , count the numbers of rats leaving a sinking ship, and which ship doesnt have any !! hahahaha

            Reply
        • Maggy Wassilieff

           /  2nd August 2020

          So how do you explain the last polls get the actual election result- within the margin of error- then ?

          1. It could be chance
          2. Push-polling….. The media polls are accompanied by many opinion pieces painting up one team in a positive light and others negatively

          and as far as I recall…Around this time in the election cycle… the newshub/reid poll had the Nats on 55%, but at election they got 44% of the vote.

          Reply
    • Alan Wilkinson

       /  2nd August 2020

      We know the sampling is non random because demographics don’t match the population and require weighting adjustments which then make the sample number an inaccurate basis for estimating the errors.

      Reply
      • Much depends on the wording of the questions; the ones I heard were leading and emotive.

        Reply
      • Duker

         /  2nd August 2020

        You keep making the same error of understanding. Its called pigheadedness

        Randomness is only achieved by the phone numbers they called being randomly chosen.
        The survey companies make millions from selling their surveys , while you are small town innkeeper

        Reply
        • Alan Wilkinson

           /  2nd August 2020

          Idiot. Having a phone and being willing to be interrupted to answer survey questions is non-random selection.

          Reply
  3. Duker

     /  2nd August 2020

    This is a non random collection from 2014

    Reply
    • Alan Wilkinson

       /  2nd August 2020

      Hate crime? Nah, just the usual loony Left obsession with their obsessions.

      Reply
      • Duker

         /  2nd August 2020

        Which book is the hate crime…oh thats right these things are in your head not paying rent
        Poor old Max Cryer would be mortified his little anecdotes are thought of like that

        Reply
        • Alan Wilkinson

           /  2nd August 2020

          No, these things are in your head which is why you put that image up.

          Reply
      • The ‘bitch sandwich’ photo was probably taken in Whitcoulls. Their display shelves at the front looked exactly like the shelves in the photo, there were two copies of Judith Collins’ book in the middle of the top shelf with a non-fiction book on either side. LIAR was underneath with other novels and the David Walliams was on a table of children’s books nearby.

        It wouldn’t be too hard to swap the end books on the display for the other two.

        Reply
  4. Corky

     /  2nd August 2020

    ”Polls are a useful but very approximate indicator of voter preferences in the past.”

    More so in the Covid environment. There’s a volatile voting public out there looking for something new. When a bro with a guitar and a portfolio of conspiracy theories can pack out a large hall that previously played host to a small NZ1 meeting, you know the times are a changin’.

    Reply
    • duperez

       /  2nd August 2020

      When some guy with such a bizarre collection of notions can pack out a large hall and they’re not there to simply see what a freak show looks like, we know that Covid is the least of our worries.

      Reply
      • Corky

         /  2nd August 2020

        The problem is the majority weren’t a bunch of Europeans along for a laugh at the dumb native who knows nothing. The audience was mainly Polynesians, immigrants and Asians.

        That’s important, and will become more so as the years pass. I was in the fruit and veg section of my supermarket yesterday. While in that department, I did not hear one word of English spoken. Samoan, Indian,Tagalog,Thai and Spanish were among some of the languages I pick up on.

        Ask yourself a simple question, Duperez…who would many of these folk feel more comfortable with? While I personally don’t believe Billy will get anywhere near 5%, he will however become a template for those who come after him…people with political cunning and a purpose.

        Reply
        • How on earth did you know that the people in the audience were immigrants ? One of my bridesmaid’s ancestors came from China several generations ago. I am the child of immigrants and my husband was one; could you tell by looking at my (Pakeha) parents that they were not born here ?

          What a bizarre collection of languages in the supermarket. What were the people saying ?

          ‘Indian’ is NOT a language, that’s like saying that people were speaking ‘European’ or ‘African.’.

          I don’t know why any of these people would feel comfortable with a conspiracy theory nutcase simply because they happen to have been born overseas; assuming that this odd collection of nationalities really were there all at the same time.

          Everyone is a native of wherever they happened to be born. It’s not a race.Natus; born (Latin)

          Reply
        • duperez

           /  2nd August 2020

          What I believe is that people with their heads programmed to be fearful racists are going to be scared racists.

          What I think is that immigrants who are upset that there are other immigrants who make things undesirable should emigrate to where they’ll be happy.

          What I realise is that conquest seems to be an age old part of the human condition. What I’ve learned is that conquest comes in many forms. I’ve learned that bullies don’t like it when what they dole out is done back to them.

          What I appreciate is that Jacinda Ardern and Judith Collins ]will become templates for those who come after them …people with political cunning and a purpose.

          Reply
          • Corky

             /  2nd August 2020

            ”What I believe is that people with their heads programmed to be fearful racists are going to be scared racists.”

            What mush does that mean apart from everyone is programmed in some way to some degree.

            ”What I think is that immigrants who are upset that there are other immigrants who make things undesirable should emigrate to where they’ll be happy.”

            Another meaningless opinion?

            ”What I realise is that conquest seems to be an age old part of the human condition. What I’ve learned is that conquest comes in many forms. I’ve learned that bullies don’t like it when what they dole out is done back to them.”

            What are you on about?

            ”What I appreciate is that Jacinda Ardern and Judith Collins ]will become templates for those who come after them …people with political cunning and a purpose”

            What?

            Well, that was a fugging waste of column space, eh, Duperez. My suggestion is go back over One News tonight. You may learn something about immigration and a looming Maori crisis. It’s only a matter of time before the floodgates have to be opened. That won’t necessarily be all bad. But it won’t be good.

            Reply
            • duperez

               /  2nd August 2020

              It’s just about what appears to be a central driving force in the way you see the world – you have problems with people with different coloured skin to yours.

              You accept people from the United Kingdom conquering the world through the superiority of their western culture but seem to be affronted or scared shitless at any semblance of the same being reciprocated or reversed by others doing the same thing, albeit by different methods.

              You’re worried about Billy becoming a template for those who come after him…people with political cunning and a purpose. The lunacy might be different but is he really any different to Collins and Ardern with their political cunning and purpose?

  5. Sunny

     /  2nd August 2020

    At least they are upfront about their concerns. Better than the usual “Leaked poll shows labour has big jump.” ( RNZ May 2020) that went across all major media in May . Hmmmm, I wonder who leaked it?

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s