Show notes
The unfolding coronavirus pandemic is a story driven by numbers. But how reliable are the numbers we have? What can the data really tell us – and what are the major areas of uncertainty? This is the transcript of a special episode of Risky Talk, recorded on 1st April. Michael Blastland and Professor David Spiegelhalter help us separate the signal from the noise. *** Michael Hello, and welcome to a special edition of a Risky Talk with Professor David Spiegelhalter. Except that many of you will know, I'm not David Spiegelhalter, and not because he's isolated with Covid 19. No, David is here or rather, at one end of a line with me, Michael Blastland, at the other. Plenty of distance between us. David, welcome to your own show! David Grand to be here. Michael We're doing it like this this week simply because the team who made the podcast decided we wanted to hear more from David on Risky Talk, where he's usually asking the questions. This time, he'll be answering them. And what we want to hear about especially at the moment, guess what, given that David's a world leading statistician, are his thoughts about coronavirus or covid 19: Where we are now, what the data we have can tell us and what it can't and how we should think about the risks. Now, we're having this conversation based on data available up to March 31st. So do bear that in mind if you're listening far into the future - new evidence is coming in all the time. David, a more loaded question these days than it usually is. How are you? David Well, I'm actually feeling very fit because this confinement is making me pay a bit more attention to my exercise! Michael Do you have a sense of your personal risk? How risky does it feel for you? David Oh, yeah, not too bad. I wouldn't actually be too worried about getting it - I think it could be very nasty - but we'll come back later to how we can put our personal risks into perspective. Michael Alright, let's begin with how we should approach the problem because there are a lot of numbers kicking around. We've got numbers of deaths, we've got death rates, we've got numbers of cases, hospitalization rates, doubling periods, all of that. Which data would you focus on? David I think this is the first time in an international crisis like this, where there's been immediate access to huge amounts of data. But you know, the data that's most easily available is on cases and on deaths. And there's a problem with both of them. It's one of these very basic things that, before getting down and doing any fancy analysis, we should ask: what are we counting in the first place? And with confirmed cases - those who've actually had a test and been positive - we know that's a very limited view of the actual number of cases out there. People say maybe there's 10 times as many. But of course, it depends on the testing regime. Some countries, like Germany, have done a huge amount of testing and so the confirmed cases there might get closer to the number of people that have actually got it. But other places, like the UK - we've only been testing people in hospital, not even NHS workers really yet, and so actually the number of people that are in this country with the virus is a deeply unknown quantity. With a common testing regime - as long as that's constant - you can use the confirmed cases to monitor what's going on, the trends up and down - provide some information. If the testing regime changes, of course, that all changes as well. I guess you would think what would be a good measure, to put it very crudely, is counting the bodies. But we found out increasingly what a tenuous measure that is of the fatal harm of Covid 19. Each country may be different in what it counts as a death. Some countries are counting deaths as where you can actually say ‘this is what caused the death’. In the UK, we were only just getting clarity from the Department of Health - they made this public just today - of what these deaths actually mean: that the deaths being daily reported are in fact, just deaths in hospital that have occurred for people who have tested positive. So, strictly speaking, if someone tests positive in hospital but actually dies of something else, they'll still be recorded as a Covid 19 death. But those figures don't include any deaths in care homes outside of hospitals. And also the deaths figures are delayed. They're not the deaths that happened today or yesterday - some of them are coming in from two weeks ago. Hospitals have to wait until the family have been informed and given permission. And it seems clear also that they save them up over the weekend and send them all in on Monday. So these numbers are very erratic from day to day. Michael So we're seeing a bit of a spike on Mondays are we? David Oh, yeah, the Monday spike. In fact on Saturday and Sunday, the figures went right down. So we thought this can't be true: this is an artifact. And then suddenly they spiked - in a day they went from 180 to 380. This is completely impossible from a standard statistical perspective. Assuming there’s some sort of smooth process going on - we know what the variability should be - and what we see is vastly more than that. So we've got to be very careful in drawing conclusions about what's going on underneath. Michael Now given all these uncertainties, and they sound pretty large on all the elementary bits of data that we'd expect to have, can you draw any conclusions about the seriousness of the problem? David Both cases and deaths in the early part of the epidemic in almost every country increase by 30% per day. So you get a feeling that at least the rate of increase is representing what's going on, even if the absolute numbers are undercounted or whatever. From the rate of increase we can really get a feeling of how fast something is spreading throughout the population. Of course, the epidemiologists have built mathematical models that are being used to make predictions, and they have been using this kind of data to better inform the models. Michael So, I mean, I've seen one statistician, John Ioannidis, call it a “data fiasco”. Despite that, are there things we can say with some confidence? David Yes, by looking at the rate of change. We can get a lot of information from rates of change. Rates of change, are unaffected by systematic underreporting. They're unaffected by whether it's done per million or just in the whole country, and so on. So all these different ways of measuring make no difference when we monitor the rate of change. And that is the thing everyone is focusing on, because we're so interested in whether that rate is declining. In other words, what we'd love to see both in cases and in deaths is that this rapid increase smooths off into plateaus, into a constant rate, and then falls down as it did in China. And everyone's trying to work out what is going to happen in the Western countries. Michael So that's one good solid bit of evidence. What about the one that I think a lot of people are concerned with - the absolute rate of fatality. I've seen numbers around 0.1% or 0.2%. The Imperial estimate, I believe, is about 0.9%. Is there any way of adjudicating between them? David It's very difficult. It's highly contested. This is the infection mortality rate. The case fatality rate, which is how many people who we know got the virus actually end up dying, that can be estimated a bit more firmly. But the infection mortality rate, which is the percentage of people who get the infection and then die, depends crucially on of course, how many people have the infection - and that's the one thing we do not know! We have to just guess it until we have some population measures of who's had it. So eventually we will know (rather late…). The Imperial group are working on 0.9% as an average, they released some analysis of China a couple of days ago where they lowered that to about 0.6%. But other sources are saying much lower than that - 0.1% and so on - which is more like seasonal flu. Those studies are influenced by places like Iceland, who measured, you know, a lot of people, did a big lot of testing, - they managed to test around testing 3% of the population - and found a very small number of deaths. But that's Iceland! We're assuming that this is some constant number that applies just as much in Iceland, as it would in India or somewhere like that. And that is so deeply unlikely because whether you die or not, depends crucially on the health services available. It will also depend on how serious the illness is that people get. For example, their viral load, how much have they been exposed to? We're getting multiple deaths of healthcare workers and it's really appallingly tragic, and that is sometimes in young healthy people, presumably because they've had a high exposure to the virus. And, in places where people live more distantly, less crowded in, might get very low exposure to the virus, and so actually get a much less severe form of the disease. So I think the idea that there is a fixed infection fatality rate, is pretty inappropriate. There will be big heterogeneity from place to place. Michael So would you feel confident about making a general comparison with things like normal flu or SARS or the 1918 influenza - to say simply ‘is this better or worse?’ David Well, the standard thing that people are saying about it, compared to normal flu, is that it's twice as infectious and ten times as dangerous. And I think there is still reasonably strong evidence for that. It's different to SARS - SARS was far more lethal but actually much less infectious. And so we were actually able to keep it clamped down and controlled. I'm not sure about Spanish flu, I can't remember the numbers for that. But the crucial thing about that, actually, is that it hugely affected younger people. And that is not the case with this disease. Michael When we look at the various efforts by governments around the world to deal with this, there are a lot of lockdowns, but not everybody's locking down, and we're not all locking down in the same way. Can we work out which policies are making a difference yet? David Everyone's interested in this. And I don't want to make any grand claims at all. The plots I find most useful- and I'm looking at Our World in Data at the moment because I like their presentations a lot - is one of a log scale of cumulative deaths by country. And there, the gradient gives you a very powerful idea of the increasing number of deaths. Lines that run in parallel, are running at similar rates, the epidemic is taking a similar form. And we see for example, by that the United Kingdom at the moment is running very similar to Italy was at the same stage. People have been observing that Germany has got a very low mortality rate, but actually, although it's lower at the moment, its track is exactly the same as the UK's. The very interesting controlled experiment in a sense is Sweden vs Norway. They're next door to each other and they've got very similar population structure, very similar climate, but they've taken extremely different policy views. Norway has implemented quite stringent criteria but Sweden, is still really quite relaxed, people going out for meals and no lockdown at all, schools still open and so on. Sweden's progress, although it's lower at the moment, seems to be following the trail of everybody else, of United Kingdom in Germany. Whereas Norway, actually - although it's still really barely started in Norway - is tracking quite low at the moment. And of course, we've got Belarus who have volunteered to be the ultimate control in that they're taking no notice of the virus whatsoever. Michael I assume it will take a little while before we get a clear idea of how those policies have panned out? David Yes, exactly. Michael Okay. Well, we've talked about some of the limitations of these numbers. But let's add a few more complications (as if there weren't enough) and some more uncertainties. So, there's a lot of discussion about whether we're not really preventing many deaths, we're just delaying them and maybe not by very much, maybe because a lot of people are going to die quite soon anyway, if you're already elderly or very ill. What's your view about that? David Yeah, it's an interesting argument, and it's recognized by epidemiologists, they call it "mortality displacement". And it's often said that hot weather will do this. For someone who's frail and vulnerable, hot weather might bring their death forward a little bit. That's known as mortality displacement, or I've got to admit it, the technical term also used is ‘harvesting’! Michael Oh, gosh! David Yeah, exactly! I mean, I suppose it brings up images of the grim reaper, but that is the technical term. So people have said - in fact, Neil Ferguson, the head of the Imperial University modeling team, said in evidence to the parliamentary committee last week that he thought maybe two thirds of deaths were people who would probably have died reasonably soon anyway! So, there's a couple of issues about that. First of all it means, perhaps, that when we come out with this, we'll find that the number of excess deaths in the country over the year might very well not be excessive. It might look just look like even an average flu season in terms of the extra deaths. Michael Really!? Within the kind of normal range that we might expect to see? David Well it does vary very much - and that's to do with the flu. An average flu season will bring it up 17,000 or so extra deaths. A bad flu season would take it up to 25,000 extra deaths over the winter. Many of whom are, to use the technical term, ‘harvested’ from people who would die soon after anyway. Michael And that is assuming lockdown, in saying that it's within what we might expect? David Oh, yes, this is assuming lockdown! This is the crucial thing, this is what I dread at the end of this. If these lockdown measures are effective, and the aim is, as has been made explicit, is to try to keep the deaths below 20,000. If that happens, and people may look back at the end of the year and say, "oh, well, not many extra people died anyway", then the accusation will be made that therefore, we didn't need to do the lockdown, which is not a logical consequence! It's only as low as 20,000 because we have taken these measures. The Imperial group estimated that, even with the less stringent mitigation measures which were in place before the lockdown - there would still have been 200,000 deaths. Now that really would make a dent in the population! We have around 600,000 deaths a year in this country, so we'll use that as the norm. So actually, you know, 20,000 extra deaths is within the margins what can happen in a flu year. Michael But even if we took even if we took two thirds off that 200,000 figure, we’re going to see something that's outside the normal kind of variation? David Yeah, and the crucial thing - and people do forget this, even though it's been emphasized again and again, by the Imperial modellers and everybody - is that in the end it’s not a matter of saving the lives of some old vulnerable people who are going to die soon anyway. It's the fact that if you didn't do that, if you didn't bring in these measures, you know, the NHS would be just totally overwhelmed, particularly as it’s short staffed. It's going to be touch and go, I think, anyway, whether it can deal with it. I mean, amazing measures being taken at the m…
Full show notes at the publisher