A robust critique of Ministry of Health modeling?

It is important and healthy to have people critiquing official Covid modelling, even if they have available much more information than earlier models were based on.  But critiques are open to criticism too.

David Farrar ar Kiwiblog has posted A robust critique and refers to the “key takeaway” from Ian Harrison at Tailrisk Economics critiques the modelling done by the Ministry of Health:

When we ran the Covidsim model we found credible paths that could reduce the pace of infections to sustainable levels. Deaths in the range of 50 – 500 over a year are more realistic numbers. 500 deaths is around average for the seasonal flu. We found that the higher OCRG numbers were mostly generated by their assumption that tracing and testing would be abandoned.

This OCRG assumption is almost incomprehensible, unless there was a deliberate attempt to blow up the numbers. Whether the Ministry was ‘in on it’, or simply didn’t understand what was being reported to them, we do not know. We have attempted to discuss the issue with the OCRG but have had no response.

Farrar comments “So the figure of potentially 14,000 dead was not at all robust.”

Perhaps that was too high, knowing what we know now. But how robust is “Deaths in the range of 50 – 500 over a year are more realistic numbers”?

New Zealand took drastic action and if we play safe in relaxing restrictions we may come out somewhere near the lower end of that range. But what if we had taken a more relaxed approach?

Sweden has about twice our population and has had 1,511 deaths in about a month. That equates to about 750 deaths in a month here, so we could easily have been outside Harrison’s 50-500 range in a year.

Switzerland has a little more population and in a month has had 1,368 deaths. At a similar rate New Zealand would have had about 800 deaths in about a month. If that rate continued for a year we would have close to 10,000 deaths.

New York has had 17,761 deaths so far, and at that rate we would have had several thousand deaths. New York is quite different to here, but it shows how quickly and deadly Covid-19 can be.

Whether we would ever have got to anywhere near 14,000 deaths in a year remains debatable, but we could easily have gone past 500 in a month let alone a year – and we are yet to find out how the virus and the death rate progresses.

The robustness of the MoH models should certainly be examined, but so should the robustness of any critiques.

There isn’t much coverage of Harrison’s criticque, but it gets some support here at Croaking Cassandra: Coronavirus economics and policy: from the mailbox and it comes up in comments at interest.co.nz here.

 

Leave a comment

55 Comments

  1. Pink David

     /  19th April 2020

    Compare to Taiwan, Japan, Australia, South Korea perhaps?

    There is zero evidence that the lockdown has achieved anything, and has cost a vast amount that will, without doubt, cost significant lives.

    Reply
    • Blazer

       /  19th April 2020

      Do nothing was never an option Pink David.

      Reply
      • Pink David

         /  19th April 2020

        Where have I ever supported ‘do nothing’?

        All the above nations have taken actions, none of them have locked their entire population into house arrest.

        Reply
        • Duker

           /  19th April 2020

          Those other countries, because of closeness to China, had robust public heath measures that they used previously and tracing capabilities.
          We in NZ had 9 years of neglect so had no capability built up over many years
          indeed a previous Public Health Agency was shut down when national ministers were lobbied by the liquor and fast food / soft drink/snacks business to get these people of their backs

          Reply
  2. Alan Wilkinson

     /  19th April 2020

    I linked that a few days ago. I think the key point is that the model falsely assumed human behaviour would not change in response to the threat and that no counter measures at all would be taken.

    That suffices to invalidate it before even beginning to examine its other assumptions.

    Reply
    • Duker

       /  19th April 2020

      really ?.

      The epidemiological model often cited by the White House, which was produced by the University of Washington’s Institute for Health Metrics and Evaluation, originally predicted 100,000 to 240,000 deaths by midsummer. Now that figure is 60,000.

      NYtImes
      So if the White House favoured model was ‘high’ what hope did we have ? NZ pro rata of the US 60,000 is around 1000. Just as well we went to level 4 as that is far too high for us as a direct result
      All those Trump loving pickup driving rural/small town Americans may sneer at the big cities and demand ‘Liberation’ but their turn for Covid to rip through their towns ( with larger elderly pop) will come and there will be nothing that can be done for them apart from Sheriff block all roads in and out

      Reply
      • Duker

         /  19th April 2020

        Official US website and their predictions running IHME model
        https://covid19.healthdata.org/united-states-of-america

        They allow you to plug in other countries ( but not NZ or Australia)
        This is Sweden
        https://covid19.healthdata.org/sweden
        They are a bit more than twice our population and deaths per day are 170

        This readers is how you add value to posts rather than Youtube links of old music

        Reply
        • Blazer

           /  19th April 2020

          People can and do decide that value for themselves.

          Reply
          • Gezza

             /  19th April 2020

            Yes, the emotionally intelligent decide for themselves what & who they are interested in & what should just be skipped over because its just niggling or batshit boring, & it’s smarter to just wait & see what comes out in the final wash & who’s right and who’s wrong. When you’re interested in people more than things, it’s always interesting watching out for the obsessives & point scorers & other personalities & enjoying the jousts.

            The good things about this blog include its range, its characters, & its moderator, who’s a pretty good judge of character & gives everyone a fair go

            Reply
      • Alan Wilkinson

         /  19th April 2020

        Look at Switzerland, similar size, similar stats, no stay at home, no travel ban.

        Reply
        • Duker

           /  19th April 2020

          It would seem it’s when you close up business that reduces a peak but 1500 dead?
          It would be unacceptable here …because of what we know what we have achieved now.
          Australia did a different way with the GFC with it’s labour government and it worked, Key and English were too cautious , and did the wrong thing with increasing GST.
          We lost 5 years because of that failure but neoliberals can do no wrong in your eyes.

          Reply
          • Duker

             /  19th April 2020

            Guess what comes after Swedenand Switzerland…. UK
            37,000 deaths by August….or 3000 in NZ terms

            Reply
  3. Geoffrey

     /  19th April 2020

    Has it yet been made clear what happens to the numbers when “probable” cases prove to be uninfected? Is the growing total a net total or simply the cumulative total of daily totals (including “probables”)?

    Reply
    • Alan Wilkinson

       /  19th April 2020

      I believe the probable cases are moved into the confirmed list if tested positive or dropped out if proven otherwise. The excel spreadsheets appear to be updated to current knowledge of all cases AFAICS.

      Reply
    • Duker

       /  19th April 2020

      probables already prove they are uninfected if the test is negative, but they are considered recovered probables if they had symptoms.
      We dont yet test for anti bodies …The antibody tests may give false positives for existing ordinary human coronaviruses that cause colds.

      Reply
  4. Alan Wilkinson

     /  19th April 2020

    The consequences of this event will be so dire that never again will public health officials and academics be permitted to set government policy.

    Just as the Vietnam war marked the end of our Government’s power to draft young men into wars.

    Reply
    • Blazer

       /  19th April 2020

      the Govt enacts legislation.
      That includes banning conscription.

      The consequences will indeed be dire.The Govt did not introduce C19 to the population
      In unchartered territory it is better to be safe than sorry and take the advice of our most qualified.
      That excludes all commentators on this..thread.

      Reply
      • Alan Wilkinson

         /  19th April 2020

        Tell that to those who died in the Grenfell tower building.

        Reply
        • Blazer

           /  19th April 2020

          is that in Keri Keri?
          You should know Al,that at your age…only Gemma listens to a word you…say.

          Reply
          • Alan Wilkinson

             /  19th April 2020

            Oh dear. For an hour or two I thought you were a recovering human, B. Sorry to see such a prompt relapse.

            Reply
            • Blazer

               /  19th April 2020

              this post was way earlier Al…I venture that Gemma does listen to…you.

        • Duker

           /  19th April 2020

          The failure of the Rogernomics ‘long term’predictions should have ended the neoliberals influence on government policy….wilco remains a lutheran for the neoliberal bible

          Reply
          • Alan Wilkinson

             /  19th April 2020

            Not sure how you think Douglas’s predictions were tested when Lange stopped it dead in 1989 and Bolger blocked Richardson a couple of years later? Actually I know. You just rabbit.

            Reply
  5. David

     /  19th April 2020

    What is missing from all these models is out of the 1511 deaths say in Sweden are 1397 people who are in the 80s plus in poor condition and likely to not live more than another year. Another 100 who have bad heart conditions etc who unless they change lifestyles will probably have a heart attack soon and the balance is just very sad.
    It makes a huge difference if it was fit healthy 30 year olds rather than who is actually dying from this thing in terms of tanking the economy and for what ?
    The models and discussion need more uncomfortable nuance which will happen when the 2nd and 3rd waves hit and the media are focused on other things and we stop caring about death tolls as happens now with road tolls.

    Reply
    • Griff.

       /  19th April 2020

      The Actual the numbers of deaths by age for Sweden are available on line if you look.
      Just making shite up as you have makes you opinion worthless .
      It also makes your rationality somewhat questionable.

      Reply
      • David

         /  19th April 2020

        It would be nice if you would tone down the personal abuse its tiresome and reflects poorly on you, it generally just lowers the tone of the site without adding anything interesting.
        My point that I made that you ignored in your haste to be abusive is the models dont measure the pre existing conditions, co morbidity,s, life expectancy they just give you age which is not overly helpful when you take into account the economic destruction.

        Reply
    • Blazer

       /  19th April 2020

      meanwhile back in Pomgolia…a PR masterstroke for Bojo…a wag writes…
      ‘This government’s special advisers were wheeled in at an early stage once BoJo the Clown and Goebbels Cummings realised they had made a catastrophic mistake in simply doing nothing and allowing the misapplication of the concept of creating a herd immunity to take root. Using the fundamental advertising principle of repetition of slogans, they quickly wove the NHS into the theme that to do the government’s bidding was to protect the NHS. This was quite clever in that it personified the Service as a national champion against an evil scourge threatening us all and linked the Tory government on a daily basis with the NHS to the extent the two are indivisible – no mean feat for a party that has systematically denuded it of funds over a decade.

      Once the ball was rolling it was simply a matter of glad-handing, the placement of feel-good stories and the portrayal of a plucky nation’s champions waging war against the foe and the campaign got a boost when the nation was led into that absurd pantomime of clapping for the NHS the length and breadth of the country.

      But the real coup was the focus upon a little old man trudging up and down his garden in a bid to do his part for the NHS – it was a godsend that he was a former soldier in the Second WW in that they could now re-create the whole sentimental panoply of emotions recalling how Blighty vanquished Johnny Foreigner. He was automatically a ” hero ” and if that were not enough the Nudge Unit got him dressed up in his best bib and tucker suitably regaled with all his medals and surrounded him with serving soldiers, in his words, the ” best sort of people “. Cue the biography of his war service and his fight against cancer, parade the ranks of plucky nurses wracked with emotion as they celebrate his devotion to them replete with tears and smiles and orchestrate the public clamour in the press that Captain Tom be knighted, sanctified and elevated to the pantheon of Gods – ” Tom is a Legend ” – and now we have a living, breathing opera of the great and good and above all, it’s British!

      It is only a matter of time that BoJo the Clown and Capt. Tom are photographed together in a comradely embrace.

      Tears, rampant emotionalism, the creation of heroes and the manipulation of a credulous nation so easily swayed by the legerdemain of a government without any principles of its own other than populism.’

      Reply
      • Alan Wilkinson

         /  19th April 2020

        Enough lemons on that tree to keep your milk curdled for weeks, B. Enjoy.

        Reply
  6. david in aus

     /  19th April 2020

    When I first commented on the modelling, I was critical of the all or nothing approach. The modelling lacked sophistication. So my first thought was that it was a selective release of the worst-case scenario to justify the government’s action. Jacinda seemed to base her actions on talking to mates in Europe, instinctive based decision.

    This is not to say the government was wrong to do a level 4 lockdown, but the spin on the reasons was deceptive and meant to show them in the best light. Talk about transparency was just that. To be fair, that is what most politicians do.

    I suspect that their ministry of healths’s capability to test and track was so limited at that time, that a level 4 lockdown was not unreasonable. Talk about the elimination of the virus as the reason for level 4 restriction makes no sense if they foreshadowing going to level 3 when the per capita rate of new infections is higher than Australia.

    It is easier to be critical in hindsight but they should be honest about their reasons for changes. Their original modelling assumptions were wrong but when there was uncertainty on the local status and capabilities, the lower risk was to overreact. Now we have more information on the effects of level 3, in Australia, and the results are similar to NZ with lower economic fallout. That is the reason to move to level 3, the facts have changed.

    I am going to assume that politically it is impossible to admit this. Jacinda keeps on saying not to compare to Australia but before the lockdown, they were comparing themselves to Europe and not the best parts of Asia. Mightly selective comparisons.

    Reply
    • Pink David

       /  19th April 2020

      “Their original modelling assumptions were wrong but when there was uncertainty on the local status and capabilities, the lower risk was to overreact.”

      The overreaction comes at a massive cost. Just the wage subsidy alone is now at $10bn and will run many times this. That will exceed an entire years spending on health care for NZ.

      That will cost lives.

      Reply
      • Duker

         /  19th April 2020

        No they werent wrong ,this was 6 weeks ago, when no one knew
        Hindcasts are always better at choosing parameters as they can exclude what ever they like to make the model fit the current numbers.

        https://nz.linkedin.com/in/ian-harrison-57a80655

        Ian Harrison isnt even an epidemiologist , his knowledge and back ground is economics and insurance modelling.

        IN OTHER WORDS complete nonsense

        Reply
        • Pink David

           /  19th April 2020

          “No they werent wrong ,this was 6 weeks ago, when no one knew”

          Yes, they were wrong 6 weeks ago. As I have said, many times, the computer models have no predictive ability. This has been proven time and again. The only purpose of a computer model in that context is as propaganda.

          “Ian Harrison isnt even an epidemiologist , his knowledge and back ground is economics and insurance modelling.”

          He clearly understands this better than you, whoever he is.

          Reply
          • Duker

             /  19th April 2020

            Like I said previously it was what I worked on for my very first job. They had $millions , a lot then and years to create base data and model the future 10 and 20 years out. Every year monthly monitoring data was compiled and every 5 years renew the assumptions – especially census data and re calibrate.
            No one had any of this except what they had from previous epidemics.
            Im sure you know all this ( unless you have no real maths understanding unlikely but might explain) but keep with these half baked opinions for ideological reasons

            Reply
            • Pink David

               /  19th April 2020

              “Like I said previously it was what I worked on for my very first job. They had $millions , a lot then and years to create base data and model the future 10 and 20 years out.”

              Cybersyn? How well did that work out? When has anyone been able to ‘model the future’? 20 years out? Are you serious?

              “No one had any of this except what they had from previous epidemics.”

              Total bullshit. Modelling has been used for many decades. As I keep repeating the science on it is clear, it has no ability to predict.

            • Alan Wilkinson

               /  19th April 2020

              What was your role?

          • david in aus

             /  19th April 2020

            The modelling data was deceptive in what was left out from analysis. It was the do-nothing let it rip model which was never an alternative or lockdown level 4.

            Was Level 3 was modelled? Was Hong Kong, South Korean, or Taiwanese experience modelled as viable alternatives?

            If transparency is valued as it is claimed, an official inquiry may be the answer.

            Reply
    • David

       /  19th April 2020

      Even at level 3 we are tougher than Australia where at least you can get your haircut and pop into Bunnings.

      Reply
      • Duker

         /  19th April 2020

        We probably had less ventilators per head than Australia …9 years of neglect
        The key parameter to consider was to get the R rate of transmissions down before your health system was overwhelmed.
        Germany is now at 1 , if was even went a little higher at 1.2 then even their health system is overwhelmed in the next 4 + months

        Reply
      • Pink David

         /  19th April 2020

        Oddly enough, I’ve done both of those things today.

        Reply
      • Blazer

         /  19th April 2020

        what stoicism you show David…at least you now know what your wife looks like with her hair…dishevelled.

        Reply
  7. duperez

     /  19th April 2020

    Recently Moody’s has been in the news for their latest rankings for banks.

    Is there a ranking system for scientists and modellers? Or is it a system where those scientists you agree with are automatically Aaaaaaa and those you don’t are C? Or zzzzzzzzz …. 🙃

    Reply
    • Duker

       /  19th April 2020

      When the use a desktop app called CovSim ( check) and they have no real idea of epidemiology( check) and this is probably the first time they have looked at infections(check)
      Double points when they are smart arse economic consultants ( check) who get their work published on kiwiblog( check)
      Where was Taildragger Economics own Covid model run 6 weeks ago

      Reply
      • david in aus

         /  19th April 2020

        To be fair to economists. They probably have the best skill-set for modelling. Modelling is mathematics, assumptions, and trade-offs – their specialty. More so, than for doctors like microbiologists. I am not sure how one microbiologist is getting so much attention. It must be the hair.

        Reply
        • Duker

           /  19th April 2020

          It’s the media they always like it when scientists fit an ‘other’ category. Smart but a bit wierd.

          Reply
        • Griff.

           /  19th April 2020

          To be fair to economists.
          Economist are as useful as witch doctors casting a pile of bones .
          They have no idea about even their own discipline.
          How many expert economists predicted the GFC?

          A hand full out of tens of thousands successfully predicted the biggest economic crisis in the entire history of their discipline.

          I read the “robust critique”
          It read like curve fitting, fiddling with parameters, to get the outcome they desired rather than anything approaching analysis using known facts and capability’s.

          “With four parameters I can fit an elephant, and with five I can make him wiggle his trunk.”
          John von Neumann

          Reply
      • david in aus

         /  19th April 2020

        People are talking about ‘experts, experts, experts’. Experts advise; politicians decide. The decision for a shut down involved more than thinking about disease, deaths, and spread. It involves considerations of economics, trade-off, and values.

        That is what public health is about, consideration of health, economics, and politics. For example, some doctors want to ban/tax sugary drinks, the decision, either way, involves more than someone’s medical opinion.

        Reply
        • Duker

           /  19th April 2020

          The sugary drinks sellers aren’t wearing the downstream costs of obesity
          There is a whole list of medical conditions which are slow moving epidemics in waiting, just waiting to overwelhm the medical treatment system…for example kidney dialysis. Sound familiar it’s not only sugary drinks but there is no downside to sweeping them away as much as possible…but we wait till the tsunami is crossing the beach

          Reply
        • Alan Wilkinson

           /  19th April 2020

          The role of experts is to explain options and consequences they expect. Their clients make the choices. Often as in this case the options and consequences impact multiple fields of expertise and therefore require experts in multiple fields.

          And when there are conflicting factors, value judgements must be made in the public interest by politicians reflecting popular opinion.

          Reply
  8. Geoffrey

     /  19th April 2020

    Damn it Blazer, you have wrecked a perfectly good morale booster. Damn your emotionless denigration of a stout chap for no reason other to make a point.

    Reply
  9. Duker

     /  19th April 2020

    No ability to predict the future?
    Well they did no matter how much you didn’t like it and still do. Sure it’s not a crystal ball
    Bizarre that you deny reality…of course the formula used for modelling can be authenticated.
    It’s been so long now I can’t recite the concepts but they underlie a lot of modelling and computers make the huge calculations workable.

    Reply
    • Pink David

       /  19th April 2020

      “Well they did no matter how much you didn’t like it and still do. ”

      Then you must be posting from your island where you are enjoying your billions.

      “It’s been so long now I can’t recite the concepts but they underlie a lot of modelling and computers make the huge calculations workable.”

      I’m sure all the big investment coy are banging down your door for you to consult to them as a quant.

      Reply
  10. rather than making stupid comments, please read and understand the report

    Click to access Corona.pdf


    They have spent many hours of research with expertise. Comments by people who have not properly read and understood the report are not helpful at all in adding to the knowledge base.

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s