I.In these days of the lock-down caused by the Corona virus, politicians and the public are awaiting the advice of science. Hence the continued unbroken appetite of the media for new numbers. No talk show takes place without a virologist survey, and the results of relevant studies, which may otherwise be of interest to experts, find their way into the main news of all television channels. So also that of the current Heinsberg study: “There should already be 1.8 million infected,” said the head of the study, Bonn’s virologist Hendrik Streeck.
A number of experts doubt this conclusion with good reason. Because it cannot be hit due to the survey. The infection mortality rate of 0.36% calculated in the study was interpreted in numerous comments as a clear sign of the all-clear, according to the motto: everything half as bad, Covid-19, the signal, is not much more dangerous than flu. So it’s worth asking how resilient this number is.
The relevant parameter for assessing the dangerousness
A look at the original work published by the scientists on the website of the University of Bonn helps. They indicate the confidence interval of infection mortality from 0.29 percent to 0.45 percent. The confidence interval is the area of a measurand in which the “true” value of the measurand is likely to be found. The 95 percent confidence interval is usually specified in science, i.e. with a 95 percent probability, the true value lies in the range. Infection mortality is defined as the number of patients who died from an illness divided by the number of those actually infected. The latter number is unknown.
Health authorities only know case mortality. It is defined as the number of deceased patients divided by the number of patients confirmed by tests. The “true” infection mortality can only be reliably determined at the end of the pandemic. However, it is a key factor in assessing the danger of the pandemic. Because it is the only one that can estimate the burdens that threaten the health system.
In the Heinsberg study determines the infection mortality on the basis of only seven (sic!) Deaths reported in the catchment area of the study. The proportion of infected people in the population, the so-called prevalence, was determined in a sample of 919 participants, of whom 15.5 percent tested positive for antibodies against Sars-CoV-2, and thus the disease Covid-19 should already be behind .
The 95 percent confidence interval for prevalence is given as 12.3 to 19 percent. Infection mortality then results from the number of reported deaths divided by the product of prevalence and population in the catchment area of the study.
But those who, like Hendrik Steeck, want to transfer the results to the population of Germany, have to precisely determine the confidence interval of infection mortality. Again, two areas of trust need to be taken into account, those for deaths and prevalence. Even laypeople notice that things are getting unsafe. Maybe that’s why the scientists preferred to let it go or simply forgot it.
To assess the reliability of the results, one has to include the statistical error in the number of deaths. If one uses the necessary – quite complicated – mathematics (one has to calculate the confidence intervals of the binomial distribution), one finds with some surprise that the resulting uncertainty is significantly greater than the authors have stated.
Now the upper limit of the confidence limit for infection mortality is 0.7 percent, almost double the mean value specified by the authors. One could therefore also have titled: Sars-CoV-2 seven times more dangerous than influenza. This could also be derived from the numbers. But this would be negligent. Because the mortality rate of the virus infection in Germany can only be estimated with great uncertainty based on the study in Heinsberg. You can’t know better based on the numbers. You can know that in turn. And then scientists should say it too.
This episode, not the first in the case to cover the mistakes in a study of the current pandemic, is also a lesson on how the media deals with reports from science. Statistics are complicated. But could it not be the habit of television presenters and journalists to ask the scientists the crucial question: Have other scientists already checked their results?
Christoph Sahm is a physicist, worked in a responsible position in the medical industry and leads a hospice group
Stephan Sahm is a medical ethicist and chief physician with a focus on tumor treatment and many years of freelance work in the “Nature and Science” department of the F.A.Z.