Why diagnostic tests are at the heart of the global coronavirus response and why they fail to provide certainty

On 12 February 2020, the number of new cases of coronavirus in Hubei Province, China, rose by 14,840. This sudden rise in numbers was not due to a surge of new infections. Instead, it was caused by changes to the way Chinese authorities officially diagnosed patients with the disease. Since late January, only those cases verified with a nucleic acid laboratory test had been officially tallied. A positive lab result was both a precondition for access to hospital-based treatment and the basis for national and international counts.

But by early February hospitals reported widespread shortages of testing kits and were unable to test all patients presenting with coronavirus-compatible symptoms. For patients and their anxious relatives, this meant that some people who needed hospital treatment were not admitted. For public health authorities, the prospect of large numbers of infected, symptomatic but untested patients heralded a radical intensification of community transmission. New radiology research presented a way forward: respective analysis of CT-scans of symptomatic patients taken at the start of the outbreak showed a distinctive pattern of damage to the lungs of coronavirus patients. The government quickly added clinical CT scan identification as an additional diagnostic method, leading to the rapid spike in case numbers.

How many people have coronavirus, or COVID-19 as it is now officially named? Far from inspiring confidence that all cases can be identified, the change in diagnostic protocol starkly exposed the gap between the number of people infected with the virus and the number of people authorities knowto be infected. How many others might carry the pathogen without having been tested for it? This is a question that was posed in China, but also, even more worryingly, in countries with weak or non-existent diagnostic infrastructures. On 27 February Nigeria reported the first confirmed case in sub-Saharan Africa, but limited laboratory capacity and absence of testing facilities suggests that the actual numbers are already much higher. In the UK, there are reports of inconsistencies in testing protocols for people who have recently returned from high-risk areas, with one GP describing the government’s testing and containment policy   ‘as leaky as a sieve’. In the US, reports that just under 500 suspected cases had been tested for the virus by the end of February were met with disbelief and anger by scientists and public health experts, with one academic telling a New York Times reporter “How come the South Koreans can do 10,000 tests a day and we can’t?”

Despite the authority with which diagnostic numbers are reported – new cases are routinely itemised down to the last digit – the shortage of testing kits in China reveals just how contingent and uncertain diagnosis can be in an outbreak scenario. On 19 February, for example, officials in Hubei Province changed the diagnostic protocol again, reverting back to nucleic acid testing as the sole basis for reporting and precipitating a drop in new cases to 394, from 1,749 the previous day.

Rapid accurate diagnosis has rarely been so important. By the same token, it has rarely ever been so controversial. Over the past few weeks, diagnostic tests for COVID-19, a novel disease, have been developed, authorised and distributed at an unprecedented speed. But questions around the reliability and accuracy of diagnostic data have multiplied. Can tests provide the certainty that publics and public health authorities seek, or are we expecting too much from diagnostic technologies?

 

Who is diagnosis for?

From a clinical perspective, it is not always necessary to use a COVID-19-specific diagnostic kit. A recent report from the Chinese CDC suggests that in 80% of confirmed cases, symptoms are mild, if not non-existent. Even for the minority of patients who develop severe pneumonia, diagnosis of the specific causative agent might be unnecessary since no targeted medicines are available and patients are treated symptomatically. Diagnosis of COVID-19 infection will likely become much more important for case management if and when new antiretroviral treatments become available.

But from a public health perspective, it is vital that everyone infected with the disease is also diagnosed, regardless of the severity of their symptoms. In fact, the mildness of the symptoms in the majority of cases is precisely what makes widespread testing so important for containment. Diagnosis is essential to an outbreak response. It enables authorities to isolate contagious patients and to know when they can safely release them back into the community, and it assists with surveillance and resource planning. Public health experts and governments look to diagnostic data for an indication of how the outbreak is progressing, to which countries it is spreading and what kind of interventions are necessary to halt, or at the very least slow, transmission.

In the consultation room, the traditional role of diagnosis is to establish what is wrong with an individual patient. In an outbreak scenario, the primary purpose of diagnosis is to know ‘who has it’; to distinguish people hosting a specific infectious pathogen from everyone else, sick or healthy. The ulitmate goal is containment; which is why diagnosis often becomes the the instrument of (and pretext for), social exclusion. But who to test? As desirable as it may be, it is not logistically possible to screen whole populations for COVID-19. Instead, resources must be targeted at the most high-risk cases, while mitigating the risk that mild or asymptomatic cases may go undocumented. Specific criteria for coronavirus testing vary by country with ‘suspected cases’ or the ominous sounding ‘persons under investigation’ usually identified on the basis of their symptoms, travel history, and contact with confirmed cases. But as community transmission occurs in more countries, testing protocols are likely to change, with how far governments should cast the net a matter of intense political, ethical and scientific debate.

In China, authorities tested 320,000 people in Guandong Province over a three week period. As of 26 February, a total of 7,690 people had been tested in the UK and 9,462 in Italy, where a cluster of confirmed cases first suggested community transmission in Europe.As community transmission becomes more common and governments prepare to step up mass surveillance operations it is clear that a global testing campaign of this scale has never before been undertaken.

 

The race for diagnostics

By definition, diagnostic tests for novel diseases do not exist before an outbreak occurs. In the case of COVID-19, public health authorities went from zero demand to an urgent need for millions of tests to be made available worldwide in a matter of days. The speed of the response has been extraordinary. Chinese scientists released the genetic sequence of  COVID-19 on 11 January 2020. On 23 January, a group at the Institute of Virology in Berlin Charité University Hospital, published details of a real-time PCR (RT-PCR) diagnostic test. Between 26 January and 30 January, the Chinese National Medical Products Administration approved five nucleic acid testing kits manufactured in China. On 3 February, the CDC announced its development of a rapid laboratory test kit for use with an existing commercially available Real-Time PCR platform.  On 9 February, Public Health England announced they had developed a test for use in specialist laboratories across the UK. The WHO currently lists six molecular assay protocols on its website, sourced from research  groups in China, Germany, Hong Kong, Japan, Thailand and the US. And multiple companies are now pursuing the development of commercial tests, building on the existing commercial automated RT-PCR machines. For example, the US company Cepheid, whose Ebola diagnostic test was used in the outbreaks in West Africa and the Democratic Republic of Congo, has announced that they are developing a new coronavirus test for distribution on their GeneXpert RT-PCR machine.

In comparison with other recent outbreaks, such as Ebola or Zika, the pace at which new molecular diagnostic tests have been developed for COVID-19 is staggering. Lessons seem to have been learnt from those two public health emergencies. Chinese scientists raced to sequence the virus, identified its genetic markers and made the data freely-available. New data-sharing platforms , including those sponsored by for-profit companies like Alibaba, have enabled real-time collaboration between international research groups. The peculiar biology of COVID-19 also played a part. The similarity between COVID-19 and other coronaviruses meant that existing assays only needed minor adaptations to work on the new strain. Last, coronavirus is significantly more contagious than Ebola or Zika and the scale of testing that will be required far outstrips that of both of those previous outbreaks, generating significantly more commercial interest in diagnostics. But while the number of COVID-19 tests has proliferated in the first weeks of the outbreak, this has not led to greater certainty about the number of people infected.

 

Emergency regulation

The WHO declared the latest coronavirus outbreak to be a public health emergency of international concern on 31January. On the same day, the Secretary for Health and Human Services, Alex Azar, declared a national public health emergency in the US, despite a very small number of confirmed cases in the country. On 3 February, the CDC submitted their new rapid RT-PCR test for FDA Emergency Use Authorisation (EUA). The EUA is an FDA mechanism that allows for use of unapproved medical products in the case of a declared emergency when there are no adequate, approved or available alternatives. In essence, the EUA entails a trade-off between evidence of the safety, effectiveness and quality of products on the one hand, and the urgent need to make medical tools accessible in an emergency on the other.

Similar mechanisms were used by the FDA and the WHO during the 2014-2016 Ebola outbreak in West Africa and the 2015-2016 Zika outbreak. In those instances, it took authorities months to release technical specifications for the tests and the majority of authorised tests were only formally listed after the outbreaks began to tail off. By comparison, the FDA gave emergency authorisation to the CDC test on 4 February, one day after the application was submitted, and on the same day that Alex Azar hastily declared the EUA mechanism would come into effect. The emergency authorisation permitted the CDC to immediately begin distributing the test to accredited state laboratories in the US and international referral laboratories identified by the WHO.

While in the Ebola case, emergency authorisation mechanisms came under attack for being introduced too late, in the case of COVID-19 the speed with which the FDA authorised use of the new CDC test came under immediate scrutiny. Within days the CDC had announced that some reagents in the kit were not giving verifiable results, and eventually switched to a new manufacturing platform.There are also reports that RT-PCR tests, including the CDC test, which use swab samples taken from the throat or nasal passage of patients, are not always picking up the virus, which typically resides in the lungs. In some cases, infective patientshave tested negative multiple times before receiving a diagnosis, including many that had COVID-19 identified through pulmonary CT-scans. One possible reason for those ‘false-negative’ results is that there is not enough genetic material for the test to detect in the early stages of the disease. Whatever the reason, the burdens that this diagnostic delay places on hospitals is considerable, as patients need to remain isolated through an extended period of testing, a possible misuse of scarce resources (such as hospital beds) at a critical time. Only adding to mounting anxieties over diagnostic protocol, it has recently emerged that China is excluding from their official tallies patients who despite a positive RT-PCR test result, exhibit no symptoms, with the argument that in such cases, patients are merely carriers and it is unlikely that the virus has entered the cells and started replicating. Needless to say, this claim is hotly contested by scientists.

 

Diagnostic uncertainty 

One possible way of interpreting this enduring diagnostic uncertainty is that the tools currently available are not good enough. But an alternative interpretation is that our expectations of diagnostic tests are too high. No diagnostic test ever provides complete certainty. Results exist within a margin of error, expressed as the percentage of true positive cases that are detected (sensitivity), the percentage of true negative results that are detected (specificity), and the lowest detectable concentration at which 95% of all true positive cases are detected (limit of detection). The notion of a ‘confirmed test’ is therefore a myth – or, one could say, an aspiration. The veracity of diagnosis is always an approximation, reflecting the degree of uncertainty public health authorities, clinicians and, in the case of an emergency, lawmakers, are willing to accept.

Emergency regulatory mechanisms, like the FDA’s Emergency Use Authorisation, or the WHO’s Emergency Use Listing, are premised on the argument that a greater degree of uncertainty can be tolerated in an emergency scenario. As the WHO puts it ‘The EUL is a special procedure for vaccines, medicines and in vitro diagnostics in the event of a public health emergency when the community/public health authorities may be willing to tolerate less certainty about the efficacy and safety of products, given the morbidity and/or mortality of the disease and the lack or paucity of treatment or prevention options.’ In an emergency, however, is precisely the moment when public demands for certainty increase.

Even when a test performs well when evaluated in a laboratory, new uncertainties emerge as soon as the test enters the field. Diagnostic tests are highly sensitive to the way they are used—functionality often requires rigid testing protocols and presume a level of contextual consistency. Those demands gain amplitude when those diagnostic tests are deployed in under-resourced health systems.  Here, the number of scenarios that could impact diagnostic accuracy are, frankly, dizzying: when tests are left on the hot tarmac at an airport while awaiting for customs declarations, for instance, or when a biological sample is not extracted in the correct way due to lack of training. Consider the hurried health worker who does not wait the full number of minutes before reading the result, or a reporting system that has not been  adapted to process the data from new tests. The trouble with all these operational issues is that the margin of error of a result can no longer be presumed. Diagnostic uncertainty no longer exists within a threshold of what is known but rather becomes the province of Rumsfeld’s unknown unknowns. Adrift from any approximation of certainty, diagnostic tests then only serve to deplete the trust that patients, clinicians and public authorities have in their outcomes—a collective scepticism that not only reduces their effectiveness, but also erode faith in a health system and tolerance its containment efforts.

Some research groups and companies are trying to design such operational weaknesses out of their tests to enable them to work in under-resourced health systems. Cartridge-in result-out automated RT-PCR platforms, like the Applied Biosystems Real-Time PCR Instrument on which the CDC test is authorised for use, require little expertise, although they do depend on biosafety systems and electrification infrastructure being in place and acting reliably. Other groups are working on the development of software to enable the direct collection and collation of real-time data from diagnostic devices, thereby bypassing the potential for human error in manual reporting systems.

But the idea that a diagnostic test can be impervious to the health system it is embedded in is also a fiction.  Even self-contained, full automated, highly portable and easy-to-use tests depend on supply chains and some form of training.  During the Ebola outbreak, efforts to develop rapid ‘point-of-care’ tests that could be used in rural health clinics were ultimately abandoned when it became apparent that there were no systems in place to safely dispose of used tests or the personal protective equipment that health workers would need to wear to do the test. Moreover, while targeted single-disease diagnostic tests are important for producing an official count of cases, the availability of a whole suite of basic laboratory tests are also vital for an effective response. From a surveillance perspective it is important to know when patients are suffering from pre-existing conditions, such as hypertension, cardiovascular disease or diabetes. From a clinical perspective, patients need monitoring with CT-scans, temperature ECGs, and testing for secondary infections. Without a fully-functional multi-tier diagnostic system stand-alone tests have only limited utility in an outbreak, putting the spotlight on poor government and donor investment in laboratory systems in sub-Saharan Africa and other under resourced setting.

In the midst of a rapidly evolving emergency, when fears and anxieties thrive, we turn to diagnostic tests to provide some degree of assurance. But what we expect from diagnostic tests is often unreasonable. We want a test that can be immediately scaled up without manufacturing quality being compromised, a test that can provide 100% accuracy, a test that can be used anywhere and by anyone. As the current coronavirus outbreak unfolds, diagnostic testing will continue to play a central role in the public health response, and undoubtedly the accuracy and availability of diagnostic testing will improve as time goes on. But the notion that a diagnostic test can dissipate the inevitable uncertainties that arise during an emergency will only lead to new controversies. Indeed, considering the contagious potential of COVID-19, the salience of testing for the outbreak response might actually decline. At a certain scale, isolation and surveillance, and, under particularly dire circumstances, clinical treatment, no longer becomes a practicable possibility. In the midst of a pandemic, the diagnosis of patients is far less important than ensuring that those who are sick or most vulnerable are provided adequate social support.

---

Alice Street is a Senior Lecturer in Social Anthropology at the University of Edinburgh, where she currently holds a European Research Council Starting Grant for ‘Investigating the Design and Use of Diagnostic Devices in Global Health’ (DiaDev). Her research focuses on hospital ethnography, off-grid health systems and medical innovation. She has carried out ethnographic research in Papua New Guinea and India and is the author of Biomedicine in an Unstable Place: Infrastructure and Personhood in a Papua New Guinean Hospital.

Ann H. Kelly is a Reader in the Department of Global Health and Social Medicine King’s College London. Her ethnographic work focuses on socio-material practices of global health research and innovation in Sub-Saharan Africa. She is currently working on number of transdisciplinary collaborations at the interface of infectious disease control, health system strengthening and global outbreak response and serves on the WHO Strategic Advisory Group of Experts (SAGE) for Ebola Vaccines and Vaccination.