"Among the 127 000 cases that we estimated in Wuhan by Feb 20, we estimated that there could have been approximately 11 000 infections (95% CrI 7000–21 000) that met version 5 of the case definition with illness onset by Jan 1, 2020. In the observed data, there were 114 confirmed COVID-19 cases with illness onset by Jan 1, 2020, corresponding to around 1% of our estimated total. Before Jan 23, we estimated that 92% (95% CrI 88–95) of cases were undetected." (My emphasis.)The article was published in the Lancet, so it needs to be taken seriously. Never-the-less, I think they are clearly wrong. In the first instance, I think they are wrong because their model predicts approximately approximately 90 cases with disease onset on December 2nd, six days prior to the onset of symptoms in the first known case. If that had been the case, the disease must have jumped to humans in early November at the latest; and cases would have started to find their way into intensive care units by late November - something clearly contradicted by the testimony of Chinese whistle-blowers.
I have other, more direct criticisms which will be the subject of my post. The most crucial of these criticisms is that when we confine the analysis to Wuhan patients, the logic of the definitions together with known clinical data show Tsang et al to massively over-estimate the effect of changes in the 'case definition' on the number of patients diagnosed with COVID-19.
Before going further, however, I must clarify that the authors of the article do not suggest that any under counting was deliberate, or fraudulent; and nor do I suggest that there was no under counting. Of necessity, in a new disease which can manifest in a mild form, there will be initial under counting of the disease. That is because people with mild symptoms will likely not go to a doctor, and if they do, may dismissed as having an aberrant form of some other disease. Therefore, in the case of COVID-19 there must have been under counting due to ignorance; and the authors do not suggest any other reason for the extent of the under counting. Where we disagree is that I think they radically over estimate the level of under counting; and that they do so in part due to logical flaws in their paper.
Methedology
To begin with, Tsang et al identify seven versions of the "case definition' of COVID-19, ie, the criteria used define whether or not a patient had COVID-19. These criteria were issued by the Chinese national government, with the first set being issued on January 15th, and each successive set being issued sequentially after that. There contention is that in most cases, change to a later criteria would have resulted in a larger cohort of people identified as having COVID-19, and based on their statistical analysis, in the case of the fifth set of criteria, a much larger cohort would have been identified. They make no claims about the sixth and seventh set of criteria because they "... were not able to explore the change after version 5 as [they] only analysed data up to Feb 20, 2020, which included just the first 2 days after the release of version 6." They show the various criteria, and when they applied in their figure 1:
As a side note, Li et al indicate that during the time of version 1 of the 'case definition', patients who had contact with the "...Huanan Seafood Wholesale Market or contact with other patients with similar symptoms" would be diagnosed as having COVID-19. This would decrease the difference in diagnosis rates between version 1 and version 2 of the 'case definition', but only to a minor extent. I will ignore it as an unnecessary complication.
Tsang et al's methodology is to first assign periods on the graph of daily new cases to particular versions of the 'case definition'. They "...estimated that the mean onset-to-reporting delay was 8·6 days (95% CrI 7·4–10·1) and the 95th percentile of this distribution was 15·7 days (13·0–20·1)." To allow for this, and the fact that the data reflects date of onset of symptoms, rather than date on which each case was identified as being a COVID-19 infection; they treat each version of the case definition as applying from 10 days prior to its implimentation (the 'backfill' period). Having done that, they use a statistical model to determine the rate of growth in new cases over each period in which a version of the 'case definition' applies. Their model incorporates January 23, "... the starting date of intervention in China...", as in inflection point, with exponential increase prior to January 23, and exponential decay in new case numbers thereafter. They then treat statistically significant changes in the growth (or decay rates) as showing the effects of changes in the version of the case definition on reporting.
I have several problems with this methodology. Perhaps the most important is that while January 23rd was the date of the lockdown of Wuhan city, measures to controll the spread of infection were imposed prior to that day, and more stringent measures also applied after that date. With the methodology used, the effect of these measures will be interpreted as the consequence of changes in the version of the 'case definition'. In addition, extraneous events such as an upsurge of travel within China associated with the Chinese New Year (January 25), and the ten thousand familly, public banquet in Wuhan on January 19, or the 300,000 people reported to have left Wuhan in the eight hours between the announcement of the lockdown and its implimentation are likely to have significantly influenced the growth rate in daily onset, but will be incorporated as a consequence of the changes version of the 'case definition' by Tsang et al's methodology. (Information in this paragraph about events influencing the spread of COVID-19 are drawn from wikipedia, Associated Press and the South China Morning Post (SCMP). See references for links.)
In addition to unaccounted for influences, the Tsang et al methodology retrodicts growth rates in the prior to the inflection point from decay rates after the inflection point for versions 4 and 5 of the 'case definition. I have considerable doubts as to whether than can be accurately done; but as I do not have the stastical knowledge to argue the case I will assume the proceedure is valid, although point to it as a potential source of errors in their analysis.
Estimating the influence of versions of the case definition from clinical data
If you look at the "Case Definitions" section of the table above, you will see how much of each form of evidence a person needed to satisfy in order to be classified as having been 'medically observed, 'suspected', 'clinically diagnosed' or 'confirmed' as having COVID-19. For the time being, all I want you to notice is that for all levels of classification, under each of the successive 'case definitions' you only ever needed to satisfy one epidemiological criteria, and then provided you satisfied the relevant clinical criteria, you were considered to have COVID-19. I also want you to notice that for all 'case definitions', contact with Wuhan was considered as satisfying an epidemiological criteria.
There is a slight difference between the epidemiological criteria for versions 1 and 2 of the 'case definition' - specifically, in version 1 the criteria was having "Travelled to Wuhan within two weeks before illness onset", while in version 2 it is "travelled to or had lived in Wuhan within two weeks before illness onset" (my emphasis). Tsang et al appear to work on the assumption that no doctor would be foolish enough to believe that only travelers, and not residents, could catch COVID-19 in Wuhan; and I agree. On that assumption, there is no epidemiological difference between versions 1 and 2 of the case definition. As all later versions use the version 2 criteria, that means that any resident of Wuhan satisfied the epidemiological criteria in all versions of the 'case definition'.
In my analysis of Tsang et al, I intend to take advantage of that fact. Specifically, I note that Wuhan cases must logically always satisfy the epidemiological criteria, for (for them to be Wuhan cases) the patients in question must either live in, or have traveled to Wuhan. It follows that any difference in the modeled number of cases between version 1 (ie, the observed data up until the introduction of version 2), or between different version can only be a consequence of differences in the 'clinical manifestations' and 'clinical tests' for the Wuhan data.
The next thing to note is that under all versions of the 'case definition', a 'confirmed case' is a 'medically observed', 'suspected' or 'clinically diagnosed' case that has also passed one of three clinical tests. Of course, in version 1, there is only one available clinical test, while in versions 2 through to 6, there are only two available. The important thing here, however, is that no person under these 'case definitions' can be diagnosed as having a confirmed case of COVID-19 without first being diagnosed as having at least a suspected case of COVID-19 (or equivalently, a medically observed case under version 1). As Tsang et al model total cases, that means that changes in the criteria for a confirmed case has no bearing on their results. Ergo, any difference between the number of modeled cases between different versions of the case definition for Wuhan data depends solely on changes to the criteria for clinical manifestations.
Turning to the clinical manifestations, version 1 of the 'case definition' required a patient satisfy four criteria to be classified as having COVID-19:
- Fever;
- Pneumonia indicated by chest radiograph;
- Reduced or normal white blood cell count, or reduced lymphocyte count during the early stage of the illness; and
- Failure of the patient's condition improving after three days of anti-microbial treatment.
Version 2, and all subsequent versions, dropped the fourth criteria. Versions 3 is identical to version 2 with regard to clinical manifestations. Version 4 requires the patient to show any two of the three clinical manifestations Version 5 uses different criteria within and outside Hubei province. As all patients in Wuhan are necessarily in Hubei province, I will only consider the criteria for inside Hubei province. For all patients, version 5 (and all subsequent versions) change criteria 1 to the patient showing "Fever and/or respiratory symptoms". It also required patients to satisfy both criteria 1 and criteria 3 to be classified as a 'suspected' case, while those patients who were 'suspected' and who also satisfied criteria two were classified as clinically diagnosed.
With these changes in criteria in mind, we can determine logically the situation of any patient excluded by an earlier version of the 'case definition', but included by a later definition, or vice versa. Thus, version 1 would exclude any person not administered a standard microbial treatment over three days, either because their doctors ignored 'the "Guidelines for the Diagnosis and Treatment of Community-acquired Pneumonia in Chinese Adults (2016 Edition)" issued by Chinese Thoracic Society and the "Guidelines for the Diagnosis and Treatment of Community-acquired Pneumonia in Children (2019 Edition)" issued by National Health Commission)' (see the full wording of the criteria in Tsang et al), or because they died prior to the completion of the three days. The former would be unusual, and the later improbable given that the medium period of hospitalization of COVID-19 patients was twelve days (Guan et al, 2020).
We can state this more emphatically. Up until January 29th, 58% of patients with suspected possible COVID-19 received intravenous antibiotic treatment (Guan et al). Version 1 of the 'case definition' only applied up until January 17. From the outbreak of the disease up until January 20, there were 5,417 confirmed cases; while there were a further 26,468 confirmed cases from January 21 through to January 31 . These periods to not precisely overlap with the changes in versions of the 'case definition'. The first period includes three days under version 2, while the second period includes two days beyond the end of the data from Guan et al. Despite this, the fact that the first period includes just one sixth of the total cases. It is therefore highly probable that in the period prior to the introduction of definition 2, nearly all cases had intravenous antibiotic treatment.
 |
Symptoms and Epidemiology of COVID-19 patients in Wuhan and China based on three papers |
We can apply a similar logic to changes to later versions of the 'case definition'. For ease of doing so, I shall use the Guan et al data, and assume various symptoms are independant. This gives a much higher estimate of missed cases under earlier versions of the 'case definition' than does an assumption of strong correlation between symptoms. On the other hand, the assumption of negative correlation (which would give an even larger estimate of missed cases) is simply not plausible.
There were no relevant differences in between version 2 and version 3 of the 'case definition'. Between version 3 and version 4, the criteria for clinical manifestation remained constant, but possession of any two of the criteria for clinical manifestation would result result in the patient being diagnosed with COVID-19. That means that under version 4 of the case definition, a patient diagnosed with COVID-19 must have had either a fever and at least one other of three clinical manifestations, or both of the other clinical manifestations. Using the Guan et al data along with an assumption of statistical independence, this leads to a prediction that cases with at least two of the relevant diagnostic symptoms out number cases with all three by a factor of approximately two (1.88). For comparison, in the Wuhan data Tsang et al show a more than threefold increase between their estimate using versions 2 and 4 of the 'case definition', ie, an over-estimate of around 50%. This is not as bad as their seven-fold over-estimate using versions 1 and 2, but is still very implausible.
Worst of all is Tsang et al's estimate of the effect of a consistent application of version 5 of the 'case definition' from the begining of the outbreak. They fail to notice that logicaly, requiring patients to have both fever and a lowered or normal white blood cell count reduces the proportion of patients diagnosed relave to version 4. It results in the exclusion of patients who had a fever and pneumonia indicated by a chest radiograph, or who had a lowered or normal white blood cell count and pneumonia indicated by a chest radiograph - ie, patients who would have been included under version 4. Indeed, if that were the only change, patients diagnosable under version 5 would be a proper subset of patients diagnosable under version 4, and thus as a matter of logic, it would not be possible for the former group to be larger than the later.
Things are slightly confused because version 5 expands the first criteria to include patients with fever and/or respiratory symptoms. Based on the Guan et al data, and using coughing as a proxy for "respiratory symptoms", that expands the number people meeting the first criterion by eight percent; but that still results in an overall fall of 4% in the total number of people diagnosable under version 5 relative to version 4. Instead, they show a four-fold increase. It is rather stunning to show so large an increase when the strict logic of the case would indicate a fall, or at least no rise in the number of diagnosed cases.
Conclusion:
I believe my logical analysis, along with the clinical data from Guan et al has shown that Tsang et al's estimate of under-reporting of cases of COVID-19 in China over the first two monts of the outbreak borders on the absurd. Their statistical techniques do not compensate for confounding factors, which are almost certainly responsible for the majority of their estimate.
References:
Associated Press: "China didn’t warn public of likely pandemic for 6 key days", 15/April/2020
Guan et al, 2020: "Clinical Characteristics of Coronavirus Disease 2019 in China", New England Journal of Medicine.
Huang: "Coronavirus: 10 new cases confirmed, 30 more suspected in Wuhan after thousands attend Lunar New Year banquet", SCMP, 5/February/2020
Huang et al, 2020: "Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China", The Lancet
Li et at, 2020: "Early Transmission Dynamics in Wuhan, China, of Novel Coronavirus–Infected Pneumonia" New England Journal of Medicine.
Tsang et al, 2020: "Effect of changing case definitions for COVID-19 on the epidemic curve and transmission parameters in mainland China: a modelling study" The Lancet.
Zhang et al, 2020: "The Epidemiological Characteristics of an Outbreak of 2019 NovelCoronavirus Diseases (COVID-19) — China, 2020", China CDC Weekly
No comments:
Post a Comment