COVID-19 vaccines are not comparable in terms of efficacy, according to specialists

The different criteria used in Phase 3 clinical trials of the vaccines approved so far were explained by scientists in a webinar hosted by FAPESP. The impact of delays in vaccinating Brazilians was one of the topics discussed.

By Karina Toledo  |  Agência FAPESP – Which is the best COVID-19 vaccine? For the experts who participated in a webinar held by FAPESP, this question cannot be answered right now. The various vaccines approved and in use around the world were developed using different techniques and tested under different conditions, so the results of the respective Phase 3 clinical trials completed to date are simply not comparable.

“It’s impossible to say if this or that vaccine confers more protection than another. They’ve never been compared in a scientific study, and different clinical outcomes were assessed [in each Phase 3 trial],” said Ricardo Sobhie Diaz, a professor at the Federal University of São Paulo’s Medical School (EPM-UNIFESP), when asked which vaccine he would prefer to take. “There’s no scientific evidence for any answer to that question,” he said.

According to Mirian Dal Ben, an infectious disease specialist at Hospital Sírio-Libanês in São Paulo, people should take “the first vaccine they’re offered”. All the vaccines are being rigorously tested and none will be approved unless they meet the safety and efficacy criteria required for large-scale delivery.

This position was also defended by Eduardo Massad, a professor at the University of São Paulo’s Medical School (FM-USP). “I took part in Butantan Institute’s protocol to assess the immune response in older people, so I’ve taken CoronaVac [developed by the Beijing-based pharmaceutical company Sinovac Life Sciences], but I’d take any vaccine,” he said. “You should take the first one that comes along. They all protect you against the disease, which is what matters.”

Livestreamed in Portuguese only on February 3, the webinar “Designing and interpreting vaccine effectiveness studies” was part of the series FAPESP COVID-19 Research Webinars produced with the support of the Global Research Council (GRC).

During the event, Diaz outlined the various COVID-19 vaccine production platforms in existence. The oldest and most studied is based on inactivated viruses. This technology is used by Sinovac for the CoronaVac vaccine tested and produced in Brazil by Butantan Institute, and by India’s Bharat Biotech for its Covaxin vaccine. A host of well-established vaccines use it, including influenza, cholera, rabies, poliomyelitis, hepatitis A and bubonic plague vaccines. The virus is cultured in a laboratory and inactivated by heat or radiation so that it does not cause the disease when it is injected into the human organism, but triggers an immune response.

Four other types of vaccine are being developed or used in national programs to combat COVID-19. In viral vector vaccines, a different virus is modified so that it cannot replicate in the human organism and used as a vector to deliver a piece of the SARS-CoV-2 spike protein. These include Sputnik V (Gamaleya Research Institute, Russia), ChAdOx1 nCoV-19 (AstraZeneca and Oxford University, UK), and Ad26.COV2.S (Janssen Vaccines, part of Johnson & Johnson, USA). In genetic vaccines, nucleic acids from the novel coronavirus stimulate cells to produce viral proteins, which cannot infect other cells but are foreign enough to trigger the body’s defense systems. Moderna, Pfizer-BioNTech and Fosun Pharma all use this technology. The fourth type, exemplified by Novavax (USA), is known as a protein subunit vaccine, meaning that it uses a laboratory-made fragment of the spike protein.

In her presentation, Dal Ben explained how the preclinical and clinical trials that assess vaccine safety and efficacy are designed. In Phase 1 and 2 trials, scientists focus on what they call laboratory outcomes, especially humoral immunity (involving the development of antibodies that neutralize the virus) and cell-mediated immunity (lymphocytes that recognize and destroy cells infected by the virus). These are also verified in Phase 3 trials, but their main aim is to assess clinical outcomes associated with immunization.

Phase 3 clinical trials can be designed to answer different questions, Dal Ben said, such as how much protection derives from a single or double dose, and how far the vaccine avoids hospitalization, death and asymptomatic transmission of the virus.

The questions to be answered determine key elements of the design, she added, including the number of volunteers, the profile of the study sample (age, ethnicity, degree of exposure, presence of co-morbidities, predominant variants in the region, etc.), and how the volunteers are to be divided into groups – two (vaccine and placebo) or three (single- or double-dose vaccine and placebo), among other formats. It is also necessary to decide whether the study will be single-blind, in which case the volunteers do not know which group they are in, or double-blind, when neither the researchers nor the subjects know; what type of placebo to use (an inert or innocuous substance such as saline, or another vaccine, for example); and how to monitor the volunteers.

“How the volunteers will be monitored and for how long also depends on the questions to be answered,” Dal Ben said. “If infection by SARS-CoV-2 is the outcome to be measured by the trial, monitoring can stop when the patient tests positive using RT-PCR but will have to continue if the goal is to see whether the case progresses to a severe condition or death. And if the aim is to find out how long vaccine-induced immunity lasts, monitoring must be long-term.”

The rarer the event to be detected, the more subjects must be included in the study so that the results are statistically significant, the webinar presenters explained. For example, the death rate among COVID-19 patients is 0.5%-1%, so finding out how well a vaccine prevents death requires a far larger number of volunteers than assessing protection against hospitalization, which is needed in some 20% of cases.

According to Dal Ben, the Phase 3 trials held so far to assess COVID-19 vaccine efficacy have used very different criteria, making comparison impossible. In the CoronaVac trial, for example, volunteers who experienced fever, cough, shortness of breath, fatigue, muscle pain, headache, sore throat, nasal congestion, nausea, vomiting, and diarrhea after the second dose were considered “suspected cases”. AstraZeneca’s trial, in contrast, considered suspected cases to be only volunteers with fever, cough, shortness of breath, loss of smell, and loss of taste.

“Sore throat, which is very common in mild cases of COVID-19, wasn’t covered in the UK or Brazil trial of the Oxford-AstraZeneca vaccine, according to the findings published by AstraZeneca in The Lancet. In the South African trial, however, it was,” Dal Ben said. “Could this be why the vaccine’s efficacy was lower in South Africa?”

Another point that makes comparisons meaningless, Massad said, is that the volunteers were vaccinated at different times and places in each trial, entailing variable degrees of exposure to the virus. There may even be variations in disease incidence within the same clinical trial if the placebo and vaccine are administered to the respective groups at different times, although this can be corrected by mathematical modeling.

The cost of delays

Massad discussed the formulas used in clinical trials to calculate vaccine efficacy and presented a model to estimate the number of deaths that would not have occurred in Brazil if mass vaccination had begun in January.

“If no one were vaccinated, there would be 350,000 deaths by the end of this year,” he said. “The number has now reached a little over 220,000. If vaccination had begun in January, it wouldn’t have risen much by December [considering that 70% of those susceptible would be vaccinated in six months]. A one-month delay will cost 41,000 more lives. Two months will cost 73,000, three months will cost 97,000. If we start vaccinating for real only in May, 111,000 more people will die. The slow start to vaccination has a cost in human lives.”

The calculations assume a vaccine with 90% efficacy delivered to 80% of the population and do not take into account the novel variants of the virus that may be more transmissible.

“Brazil has the capacity to vaccinate at least 10 million people per day. If we had enough doses for everyone, we could get the job done in little more than 20 days,” Massad said.

A recording of the complete webinar is available at www.youtube.com/watch?v=6bNLtyMSIxE.