London School of Hygiene & Tropical Medicine

There is a wealth of statistics on COVID-19 appearing in the media. Major policy decisions and interventions are being based on statistics about COVID-19 frequency and forecasts of what that will become. These numbers are being used to create headlines and make major decisions as to which countries are 'in the lead', if we should 'lockdown' and when, what to do when the lockdown is over, whether herd immunity is an acceptable option, etc. These different policy options are supported by models which use the same data, but produce different forecasts.

One reason for these differences  is that we don’t have the basic information needed to build reliable forecasts. This does not mean that we should not be making decisions – we have to – but we could make much better decisions, or at least understand the options better, if we had better data on which to build forecasts.

Here are some major problems with the data we have:

• Many published graphs just use the number of deaths in each country; small countries like The Netherlands, and Belgium look like they are doing very well, and are ‘behind’ countries such as Italy and the UK, but actually they have very high death rates when you take their small population into account – i.e. we need to calculate the rates (deaths/populations) not just count the deaths
• Most testing is done in people with symptoms, so we have little or no idea how many people are out there who have had COVID-19 but not had symptoms that led to testing.
• The tests have mostly not been validated in the field – a test might look good if you are comparing hospitalised patients with healthy people, but they don’t work so well when you are out in the field and there are many people with mild infections and few symptoms

So what are the solutions? Here are a few:

• Use graphs of rates instead of, or in addition to, graphs of counts
• Test the general population, not just people with symptoms; we need repeated representative sampling of diverse parts of the general population – this is the only way we can determine the true Infection Fatality Rate (the proportion of those infected who die with the disease), which is a crucial input for the various models 
• Each test needs to be fully validated in the field

We will eventually get through this, but in the process the world will have changed. One of the positive changes should be an increased recognition of the need for disease surveillance systems. We need to ensure that good surveillance systems are in place not only for known diseases, but ready to be activated for new diseases such as COVID – disease surveillance and descriptive epidemiology will continue to be the essential foundation for good epidemiology and good public policy. 

Publication

Pearce N, Vandenbroucke J, VanderWeele T, Greenland S. Accurate Statistics on COVID-19 are essential for Policy Guidance and Decisions. American Journal of Public Health. DOI: 10.2105/AJPH.2020.305708