More on Covid models

The early Covid-19 models that tried to predict possible death toll from Covid-19 in various countries received a lot of attention because numbers were large and alarming, but the worst case scenarios were based on limited data and nothing being done to stop the virus from spreading.

But a lot has been done to try to limit the death toll, and models have been continually refined, but there are still have quite wide variations due to not being sure how quickly or drastically restrictions will be lifted, and other unknowns.

Modelling is not very important in New Zealand now because we have very few new cases per day and deaths per day have been 0 for a few days and were never more than 4 a day. We still have quite tight restrictions with only gradual easing indicated, so we should be able to keep Covid deaths to not much more than they are now, at least for the next month or two.

Modelling is a bigger deal elsewhere as while the death toll in many countries may have flattened it is still quite high. For a couple of weeks now deaths have averaged around a couple of thousand a day in the US. The situation there is quite complex with different infection rates and different restrictions across various states, and some states are starting to lift restrictions.

FiveThirtyEight takes an interesting look at models, showing wide ranges in single models and differences between models looking ahead only for the next month (May).

Where The Latest COVID-19 Models Think We’re Headed — And Why They Disagree

Models predicting the potential spread of the COVID-19 pandemic have become a fixture of American life. Yet each model tells a different story about the devastation to come, making it hard to know which one is “right.” But COVID-19 models aren’t made to be unquestioned oracles. They’re not trying to tell us one precise future, but rather the range of possibilities given the facts on the ground.

FiveThirtyEight — with the help of the Reich Lab at the University of Massachusetts Amherst — has assembled six models published by infectious disease researchers to illustrate possible trajectories of the pandemic’s death toll.

Forecasts like these are useful because they help us understand the most likely outcomes as well as best- and worst-case possibilities — and they can help policymakers make decisions that can lead us closer to those best-case outcomes.

And looking at multiple models is better than looking at just one because it’s difficult to know which model will match reality the closest. Even when models disagree, understanding why they are different can give us valuable insight.

The article goes on to explain each of the six models and also looks at state by state breakdowns.

What this shows us is how imprecise models are.

But the US models suggest that models from a month or so ago predicting 100-200k or so deaths may have been reasonably on track, From now a lot still depends on the success or otherwise of containing the spreading of the virus, the success in particular in keeping it out of aged care and rest homes, and the time taken to find effective treatments and ultimately a vaccine.

The current official death toll in the US is about 65,000 and if the death rate continues as at present that will reach 130-140k by the end of May. Even if on average the death rate halves it will still be over 100k by then.

Previous Post
Leave a comment

21 Comments

  1. Duker

     /  2nd May 2020

    The reality is ‘the models’ are inherently inaccurate. There was no way they have enough data to make them so and in the time available. What the most use was for having different possible isolation actions to see how they COMPARE. Which was done in NZ as far as I can see which NZ Herald then wrote about ‘predicting 80,000 dead when the results had no such number. There are people still saying Ardern said so … when it was the Herald not the PM.
    The other side of the coin is weather models which we have elaborate data collection around the country and from satellites, massive computing power and yet they are essentially out of date after a day or two. A week or more and they are completely useless.

    Yesterday I checked on NIWAs model based seasonal outlook done at the end of Feb
    https://niwa.co.nz/climate/seasonal-climate-outlook/seasonal-climate-outlook-march-may-2020
    “Rainfall is expected to be near normal in all regions of New Zealand” hahahahaha
    Niwa of course doesnt use the ‘weather models’ but is more orientated to multiple climate model runs, getting a a tangle of results and then just ‘going down the middle’
    The reality is the near term weather over a few months isnt ‘skilled’ in the sense that its close to actual outcome.
    They arent to only ones , Reserve Bank modelling of the economy is equally unskilled. And for a long time was based on economic concepts that have proved to be false . Like levels of unemployment and wages, government borrowing and interest rates and more lately ‘printing money’

    Reply
    • Griff.

       /  2nd May 2020

      All models are wrong some are useful .

      All models are approximations. Assumptions, whether implied or clearly stated, are never exactly true. All models are wrong, but some models are useful. So the question you need to ask is not “Is the model true?” (it never is) but “Is the model good enough for this particular application?”

      Statistical Control By Monitoring and Adjustment p. 61 by George Box.

      Reply
    • Pink David

       /  2nd May 2020

      You are completely right about these models, they have no predictive use, however they can be helpful to compare outcomes. The curious thing is that no one seems have done this at all. Why not use the NZ model to compare to Australia? Ditto Belgium, Sweden and the UK.

      Very little evidence this is being done.

      “Which was done in NZ as far as I can see which NZ Herald then wrote about ‘predicting 80,000 dead when the results had no such number. ”

      This is quite fascinating. You are quite correct the paper did not contain a number of 80,000. The paper did not produce numbers at all. It gave infection and death percentages, which anyone could calculate out to a total number.

      I wonder why you are so fixated on the paper having ‘no such number.’ when it very specifically indicated that amount of death was a scenario it predicted.

      Reply
      • Duker

         /  2nd May 2020

        Its entered folklore the 80,000 deaths .
        Firstly it was after 400 days, we are at about 40 days.
        The single number gives the false story – Trumpian really- that it was the ONLY outcome of the study.
        In any sort of numerical based analysis , if it hasnt been done before , you have to start with broad analysis based on what is it you are looking for . Im sure you done this yourself, maybe even every day of the week.
        They werent interested in how many coffins to order for the undertakers, but was was the effect of different levels of lockdown . The infection rate wasnt known so a range of reasonable estimates was chosen. How hospitals would cope wasnt known , again put in some estimates and see. I think this was one of the principal areas they were interested in. as of course collapse of the health system under the load would send deaths soaring to the worst possible.
        We had 9000 dead in 2 months in 1918..notice the short period. This could be 40,000 in 2 months today. There were no ventilators in those days , probably the hospital system was overwelmed however there were various isolation and quarantine methods used which could and did reduce the spread.
        Either way , 40,000 dead in 2 months ‘could have been possible’ this year if it was as deadly as 1918.

        Reply
        • Pink David

           /  2nd May 2020

          “Firstly it was after 400 days, we are at about 40 days.”

          What was after 400 days? The 80,000 dead that you don’t keep saying the paper does not mention?

          Reply
          • Duker

             /  2nd May 2020

            All model runs comparisons were 400 days and that gave the peak ‘result’
            400 days was an actual number used

            Reply
      • oldlaker

         /  2nd May 2020

        Good analysis of the modelling used to propel NZ into a drastic lockdown by Ian Harrison of Tailrisk economics. He analyses epidemiologists’ claims that “If we hadn’t locked down when we had, it would have just taken off and we would have been way above Australia”.
        As he says, such claims ignore the fact that New Zealand’s epidemic had peaked before the
        lockdown had an effect (which is what I had concluded from looking at graphs of the numbers).
        I suspect there’s going to be an awful lot more arse-covering by modellers…

        Click to access ian-harrison-on-covid-modelling-round-2.pdf

        Reply
        • Duker

           /  2nd May 2020

          Tailrisk is economic and insurance , and knows nothing about epidemiology…most the time he complains about not knowing what the data used is.

          Reply
          • oldlaker

             /  2nd May 2020

            Actually, Harrison also specialises in picking apart dodgy modelling and data. You don’t have to be an epidemiologist to do that.

            Reply
  2. Alan Wilkinson

     /  2nd May 2020

    Prediction is only possible after analysis. Analysis determines sensitivities and impacts of particular conditions. Any prediction is only as good as the preceding and relevant analysis and is confined to the range of conditions the analysis scrutinised.

    Seems to me most of the predictions made from models in this event failed to meet even rudimentary criteria for reliability.

    And their authors simply claim the unknowable – that they avoided the apocalypse. Like all witch doctors.

    Reply
    • Blazer

       /  3rd May 2020

      ‘prediction is difficult…especially about the…future’..Niels Bohr.

      Reply
  3. Alan Wilkinson

     /  2nd May 2020

    I see the Brits are starting to question the ridiculous nonsense of the 2m rule.

    https://www.telegraph.co.uk/politics/2020/05/01/two-metre-rule-reviewed-amid-hope-relaxed-restrictions-could/

    There was so much utter crap in the lockdown rules imposed that the authors of it should face jail for the damage they’ve caused – trillions of dollars of it.

    Reply
    • Gezza

       /  2nd May 2020

      Most people in queues that I’ve seen are standing less than 2m apart. Maybe 1.2 – 1.7m, at a guess.

      Where the 2m positions have been painted or otherwise marked on the ground, people observe it, standing at those markers.

      Reply
      • Alan Wilkinson

         /  2nd May 2020

        Basically public transport is unusable until they scrap that rule.

        Reply
        • The local buses are only taking 15 people, so are not quite unusable…at the moment. If the 2m rule isn’t modified, they will be of very limited use, of course. Anyone who can’t get on will be well and truly stuffed if they haven’t any other form of transport.

          Reply
          • 3 Newa said that Wellington’s public transport will be struggling with that absurd rule.

            Perhaps the Dear Leader could make everyone on it wear a mask for the foreseeable future, although we know that most of these are useless at keeping viruses out. But if it made public transport usable, I suspect that most of us would wear the damned things.

            Reply
            • Duker

               /  2nd May 2020

              So you know what the level 2 rules will be , do you.

          • Alan Wilkinson

             /  2nd May 2020

            You can hardly get on or off without breaching the 2m rule.

            Reply
            • Gezza

               /  3rd May 2020

              Yeah, you can’t help breaching tne 2m rule shopping in a supermarket either.

              I’m very much aware of that when I’m coming up or down an aisle & somebody else is either standing in it, filling shelves or dithering over which item to pick from the choices available, or moving along the aisle in the opposite direction to me..

    • Blazer

       /  3rd May 2020

      yes I tweaked my model and she said the…same.

      Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s