Disarming Frontline Doctors
In its quest to meet the “gold standard,” academic medicine has put Covid patients at risk.
We commend Devorah Goldman on her thoughtful essay, “Disarming Frontline Doctors” (Winter 2021), on the limitations of randomized controlled trials and the tension between different types of knowledge in medicine. As Goldman points out, treatments of Covid-19 patients with hydroxychloroquine and remdesivir have generated heated discussions. But another widely used treatment — convalescent plasma — has also stirred up controversy.
When Covid-19 first struck, some physicians turned to convalescent plasma, a form of immunotherapy dating to the early twentieth century, which had some success in the great flu pandemic of 1918. Plasma — the cell-free part of the blood — can be donated by Covid-19 survivors and administered to people currently ill, who may benefit from the antibodies against the coronavirus found in the survivor’s plasma.
Several randomized controlled trials (RCTs) were mounted to test the efficacy of convalescent plasma. But the RCT proved a weak tool for evaluating a complex therapy in the middle of a pandemic. Its inherent strengths — a pre-determined sample size and a specific clinical outcome to measure — were vulnerabilities in pandemic conditions, particularly because not all convalescent plasma is the same. Doctors have long known that convalescent plasma’s benefit is greatest in people treated early in their illness, and that it is important to use plasma that has high levels of antibodies against the pathogen. But several trials used plasma whose antibody content was unknown and treated patients with very advanced disease. And as early outbreaks ebbed in hard-hit regions, trials were often terminated prematurely as hospitals ran out of patients to enroll. In effect, RCTs of convalescent plasma had been initiated without sufficient understanding of the factors important to success.
But patients were eager to try the treatment, even though it had not yet been approved by the FDA. In April 2020, the agency allowed the use of convalescent plasma as an “investigational” treatment “for patients with serious or immediately life-threatening COVID-19” who could not participate in an RCT. This effort — the FDA’s “Expanded Access Program” (EAP) — was managed by the Mayo Clinic and quickly found itself overwhelmed. Initially established as a demonstration program for up to 5,000 patients, so many frontline doctors embraced the use of convalescent plasma that by August 2020 almost 95,000 patients had been treated.
This widespread use of convalescent plasma led to criticism that it was being used on a massive scale without “gold standard” evidence of safety and efficacy from a randomized controlled trial. But the data collected and analyzed for patients treated under the EAP showed that the treatment was remarkably safe, and, further, that patients treated before mechanical ventilation with plasma rich in antibodies experienced fewer deaths than similar patients treated with plasma with a moderate level of antibodies, and substantially fewer deaths than patients treated with plasma with low levels of antibodies. These findings, which were replicated by the FDA in its own analysis, were important factors in the agency’s decision in August to issue an emergency use authorization for convalescent plasma.
Finding the relationship between dosage and response was important, because showing that as doses of a treatment increase, patient results improve, helps to demonstrate that the association is in fact causal, and not merely coincidental. But for academic physicians wedded to the absolute need for evidence from randomized controlled trials, even this finding was unconvincing. Some accused the FDA of rushing to approve convalescent plasma too soon, and perhaps even under political pressure, a concern amplified by a presidential news conference marred by exaggerations of the treatment’s efficacy.
In the meantime, physicians continued to use plasma outside of randomized controlled trials and accrued a large body of observational experience that rendered obsolete some of the assumptions made in the design of the early RCTs. The observational studies confirmed doctors’ historical understanding that convalescent plasma was most effective when used early in the course of disease and with high antibody content.
A well-conducted randomized trial in elderly outpatients in Argentina supports this understanding. The study reported a halving of progression to severe disease, and, for recipients of plasma with the highest antibody content, a more than 70 percent reduction in progression. Though not statistically significant, the trial showed a halving of mortality and a two-thirds reduction in ICU admission. Importantly, convalescent plasma therapy was administered in the first three days of illness.
Discordant results and lack of nuanced interpretation of various findings of both observational and RCT studies have continued to stoke the controversy about convalescent plasma’s efficacy. In recent weeks, more RCT data has become available, but it too provides conflicting evidence. In the United Kingdom, the large RECOVERY study of convalescent plasma in hospitalized patients treated on average on day nine of illness found no overall effect in reducing mortality. Yet a subgroup analysis suggested that several categories of patients — those who were treated relatively early (on or before the seventh day of symptoms), who did not receive oxygen, did not receive steroids, and had not already developed their own antibodies against the virus — were all less likely to die or require mechanical ventilation if given convalescent plasma. For the patients that did not already have antibodies, the protective effect of plasma was statistically significant, and for early onset and not getting steroids, the effect was very close. According to a newly published preprint of a double-blind RCT in the United States and Brazil, patients treated with plasma had roughly half the mortality of the placebo group.
In hindsight, the totality of the evidence — observational studies, RCTs, mechanistic studies of antibody neutralization, and historical knowledge — provides consistent support for the use of antibody-rich convalescent plasma early in the course of illness, principles that the FDA in its February 2021 update on convalescent plasma now affirms. At the same time, RCTs have found another antibody treatment — monoclonal antibodies — to be most effective when used early in the illness. Since for both monoclonal antibodies and convalescent plasma the active agent is the same, namely antibody to coronavirus, we can consider the findings about each to be supportive of the other.
The convalescent plasma controversy highlights the need to better educate physicians on the knowledge problem in medicine: How do we know what we know, and how do we acquire new knowledge? The usual practice guidelines doctors rely on for the treatment of disease were not available for the treatment of Covid-19 early in the pandemic, since these are usually issued by professional societies only after definitive information is available from RCTs, a luxury we did not have. The convalescent plasma experience supports Devorah Goldman’s plea to consider all available information when making therapeutic decisions.
Fortunately, the availability of rapid communication through pre-print studies, social media, and online conferences have allowed physicians to learn quickly. The experience suggests the value of providing more instruction in medical schools, postgraduate education, and continuing medical education on how best to evaluate evidence — especially preliminary and seemingly contradictory evidence. Just as physicians learn to use clinical judgment in treating individual patients, they must learn how to weigh evidence in treating populations of patients. We also need greater nimbleness and more flexibility from regulators and practice-guideline groups in emergency situations such as pandemics. They should issue interim recommendations that synthesize the best available evidence, as the American Association of Blood Bankers has done for plasma, recognizing that these recommendations may change as new evidence accumulates. Similarly, we all need to make greater efforts to educate the public to understand that all knowledge in medicine and science is provisional, subject to change as new and better studies emerge. Updating and revising recommendations as knowledge advances is not a weakness but a foundational strength of good medicine.
Exhausted by science and tech debates that go nowhere?