More on Covid models

The early Covid-19 models that tried to predict possible death toll from Covid-19 in various countries received a lot of attention because numbers were large and alarming, but the worst case scenarios were based on limited data and nothing being done to stop the virus from spreading.

But a lot has been done to try to limit the death toll, and models have been continually refined, but there are still have quite wide variations due to not being sure how quickly or drastically restrictions will be lifted, and other unknowns.

Modelling is not very important in New Zealand now because we have very few new cases per day and deaths per day have been 0 for a few days and were never more than 4 a day. We still have quite tight restrictions with only gradual easing indicated, so we should be able to keep Covid deaths to not much more than they are now, at least for the next month or two.

Modelling is a bigger deal elsewhere as while the death toll in many countries may have flattened it is still quite high. For a couple of weeks now deaths have averaged around a couple of thousand a day in the US. The situation there is quite complex with different infection rates and different restrictions across various states, and some states are starting to lift restrictions.

FiveThirtyEight takes an interesting look at models, showing wide ranges in single models and differences between models looking ahead only for the next month (May).

Where The Latest COVID-19 Models Think We’re Headed — And Why They Disagree

Models predicting the potential spread of the COVID-19 pandemic have become a fixture of American life. Yet each model tells a different story about the devastation to come, making it hard to know which one is “right.” But COVID-19 models aren’t made to be unquestioned oracles. They’re not trying to tell us one precise future, but rather the range of possibilities given the facts on the ground.

FiveThirtyEight — with the help of the Reich Lab at the University of Massachusetts Amherst — has assembled six models published by infectious disease researchers to illustrate possible trajectories of the pandemic’s death toll.

Forecasts like these are useful because they help us understand the most likely outcomes as well as best- and worst-case possibilities — and they can help policymakers make decisions that can lead us closer to those best-case outcomes.

And looking at multiple models is better than looking at just one because it’s difficult to know which model will match reality the closest. Even when models disagree, understanding why they are different can give us valuable insight.

The article goes on to explain each of the six models and also looks at state by state breakdowns.

What this shows us is how imprecise models are.

But the US models suggest that models from a month or so ago predicting 100-200k or so deaths may have been reasonably on track, From now a lot still depends on the success or otherwise of containing the spreading of the virus, the success in particular in keeping it out of aged care and rest homes, and the time taken to find effective treatments and ultimately a vaccine.

The current official death toll in the US is about 65,000 and if the death rate continues as at present that will reach 130-140k by the end of May. Even if on average the death rate halves it will still be over 100k by then.

A robust critique of Ministry of Health modeling?

It is important and healthy to have people critiquing official Covid modelling, even if they have available much more information than earlier models were based on.  But critiques are open to criticism too.

David Farrar ar Kiwiblog has posted A robust critique and refers to the “key takeaway” from Ian Harrison at Tailrisk Economics critiques the modelling done by the Ministry of Health:

When we ran the Covidsim model we found credible paths that could reduce the pace of infections to sustainable levels. Deaths in the range of 50 – 500 over a year are more realistic numbers. 500 deaths is around average for the seasonal flu. We found that the higher OCRG numbers were mostly generated by their assumption that tracing and testing would be abandoned.

This OCRG assumption is almost incomprehensible, unless there was a deliberate attempt to blow up the numbers. Whether the Ministry was ‘in on it’, or simply didn’t understand what was being reported to them, we do not know. We have attempted to discuss the issue with the OCRG but have had no response.

Farrar comments “So the figure of potentially 14,000 dead was not at all robust.”

Perhaps that was too high, knowing what we know now. But how robust is “Deaths in the range of 50 – 500 over a year are more realistic numbers”?

New Zealand took drastic action and if we play safe in relaxing restrictions we may come out somewhere near the lower end of that range. But what if we had taken a more relaxed approach?

Sweden has about twice our population and has had 1,511 deaths in about a month. That equates to about 750 deaths in a month here, so we could easily have been outside Harrison’s 50-500 range in a year.

Switzerland has a little more population and in a month has had 1,368 deaths. At a similar rate New Zealand would have had about 800 deaths in about a month. If that rate continued for a year we would have close to 10,000 deaths.

New York has had 17,761 deaths so far, and at that rate we would have had several thousand deaths. New York is quite different to here, but it shows how quickly and deadly Covid-19 can be.

Whether we would ever have got to anywhere near 14,000 deaths in a year remains debatable, but we could easily have gone past 500 in a month let alone a year – and we are yet to find out how the virus and the death rate progresses.

The robustness of the MoH models should certainly be examined, but so should the robustness of any critiques.

There isn’t much coverage of Harrison’s criticque, but it gets some support here at Croaking Cassandra: Coronavirus economics and policy: from the mailbox and it comes up in comments at here.