© ReutersFourth dose for a 90-year-old in Netanya, Israel
I present a striking example, pertaining to a study from Israel. Trying to be methodical, my article is somewhat on the long side, but the implications at the end are radical and broad.
Goldin et al. estimated the effectiveness of the Pfizer vaccine on several Covid-related outcomes, including death, in residents of long-term care facilities in Israel (mean age of 83 years). The large cohort (over 43,000) was heavily skewed towards vaccinated residents (90%). Only about 4,000 residents were not vaccinated.
Using a statistical method called survival analysis, the authors reported two age-adjusted values of vaccine effectiveness (VE) against Covid-related death:
Skipping ten days after the first dose, VE was 72%.Goldin et al. also analyzed all-cause death as an endpoint, which many researchers have omitted. Most important, two of their figures (below) show the cumulative number of Covid deaths and all deaths at several time points — from which we can compute the cumulative number of non-Covid deaths. The latter data has been consistently hidden in studies of vaccine effectiveness.
Skipping about seven days after the second dose, VE was 85%.
Moreover, we have mortality data since an "index date", the date on which the first dose was injected. We can analyze the data the way it should have been analyzed. No skipping.
My busy tables below show the cumulative number of deaths (Covid, non-Covid) in vaccinated residents and unvaccinated residents by the end of the follow-up (5 months), and by three interim time points. Using simple analysis, formally called "cumulative incidence", I computed the risk of the two types of death in vaccinated (blue) and unvaccinated (red) residents.
Does the Pfizer vaccine protect against death from non-Covid causes?
We have yet to hear someone making the claim.
If not, what is the explanation?
It is simple and not astonishing at all. The decision of whom to not vaccinate was not random. It must have been based on reasonable medical considerations, particularly life expectancy. For instance, what is the merit of vaccinating a 90-year-old who suffers from advanced dementia and metastatic cancer?
Those 4,114 unvaccinated residents were sicker to begin with. Their life expectancy was shorter, regardless of possible SARS-Cov-2 infection, and that's why their non-Covid mortality was several fold higher.
Stated differently, belonging to the unvaccinated group was a general marker of poorer health. Or vice versa — belonging to the vaccinated group was a marker of better health. That's on average, of course.
The phenomenon we observe here is called the "healthy vaccinee" bias, and it is well documented in the research literature, dating back to flu vaccines. The bias is very strong in frail elderly residents of nursing homes, but it is seen in all age groups of the general population.
The implication of the "healthy vaccinee" phenomenon — when estimating vaccine effectiveness — is called confounding bias. A naïve comparison of Covid mortality in vaccinated people and unvaccinated people, even if age-adjusted, is grossly misleading because the latter have higher risk of death to begin with. At least part of their higher Covid mortality, if not all, has nothing to do with not being vaccinated. They are simply sicker people.
Even Goldin et al. are aware of the bias, to which they devote one sentence at the very end of the article:
"The unvaccinated group might have suffered from more comorbidities, leading them to be more susceptible to SARS-CoV-2 infection and death, thus making the vaccine's effectiveness seem higher than it actually is." [my italics]Some researchers think that the bias operates in the opposite direction (called confounding-by-indication bias), whereby the unhealthy person is more likely to be vaccinated because they are at risk. Regardless, the net effect of the healthy vaccinee bias and confounding-by-indication bias, if the latter exists, is shown in the bottom table (above): those who were vaccinated had substantially lower non-Covid mortality. They must have been healthier, on average, not the other way around.
The table below shows the risk ratio and VE against Covid death, as computed from the data in the previous top table. VE is around 80% when calculated at various time points, and my simplified computation for the entire follow-up (82%) is similar to the main result of Goldin et al. (85%). Recall that all these estimates are distorted (biased) versions of the truth due to the healthy vaccinee bias (and naively assuming no additional sources of bias.)
Suppose the apparent risk of Covid death is twice as high in unvaccinated than in vaccinated, which means a biased risk ratio of 0.5 in favor of the vaccinated, and biased VE of 50%. Suppose we find out that the risk of death from non-Covid causes isalsotwice as high in unvaccinated. What does that imply?
Vaccination has made no difference. It had no effect on Covid death. A two-fold risk of Covid death is the expected, "baseline" risk of death in unvaccinated because they are generally sicker. Vaccinated or not, they would have had a two-fold risk of death from Covid than their vaccinated counterparts — just like their two-fold risk of death from non-Covid causes. The biased risk ratio of 0.5 (VE=50%) should be corrected to 1 (VE=0%).
To get a risk ratio of 1, from a biased risk ratio of 0.5, we need to multiply 0.5 by 2, which may be called the bias factor. The bias factor captures the underlying higher risk of death in those who were not vaccinated. It may be estimated by the risk ratio of non-Covid mortality, comparing the unvaccinated to their vaccinated counterparts.
In my simple example, the correction method nullified the presumed effect of a vaccine. As we'll see next, the outcome could be anything from attenuated VE to negative VE, where a supposedly beneficial vaccine is actually harmful.
The table below shows the bias factor in the study of Goldin et al. by follow-up time, along with the corrected risk ratio, and the corrected VE. For example, over the entire follow-up unvaccinated residents of nursing homes in Israel were 3.5 times more likely to die from non-Covid causes than vaccinated residents (bias factor of 3.5). Multiplying the biased risk ratio of 0.18 by 3.5 changed the risk ratio to 0.63 and reduced VE from 82% to 37%.
Almost all Covid deaths have accumulated by the third month (888 out of 899). Indeed, the biased VE was essentially the same (81%). Since the bias factor was higher (4.1), the corrected VE is now 22%.
Whether VE was 22% or 37% — that's a mediocre vaccine. And worse results are forthcoming.
By the end of the first month the bias factor was 7.3 and by the end of the second month it was 5.2, whereas the biased risk ratio was similar. As a result, we observe a harmful effect of the Pfizer vaccine in the first month, and overall null effect by the end of the second month. That's negative and zero VE, respectively, against Covid death.
When the inference strongly depends on the amount of data — no effectiveness by the second month versus 22% to 37% effectiveness with longer follow-up — we have a rule of thumb:
The inference is stronger where we have most of the data, not after adding a few more observations. About 95% of all Covid deaths have occurred in the first two months (first row in the table above).The correction method is not perfect, and the result depends on the value of the bias factor (an estimate by itself). Nonetheless, an increased risk of Covid death during an early post-vaccination hazard period is compatible with other data. Indeed, news outlets in Israel reported outbreaks of Covid infection in nursing homes shortly after the initiation of the vaccination campaign.
Below are two translated paragraphs from a news report, dated January 14, 2021, about three weeks into the campaign:
"Once again, a failure in nursing homes: At the same time as the second dose of COVID-19 vaccines is being distributed, the pandemic is hitting hard in the institutions where the elderly live. In the past two weeks, outbreaks have been recorded in no fewer than 160 geriatric institutions, and 1,098 new confirmed cases have been detected among residents of institutions licensed by the Ministry of Health alone.Why did they stop reporting? Have they also seen a surge in Covid death of vaccinated residents of nursing homes during the first month of the campaign?
"In parallel to the surge in the number of patients in nursing homes and assisted living centers, in the past two weeks "Senior Shield" [a task force for Covid management in nursing homes] stopped publishing the daily report on Covid morbidity data in geriatric institutions on the Ministry of Health website."
Whether the Pfizer vaccine had time-dependent negative effectiveness, no effectiveness, or mediocre effectiveness — the excellent effectiveness against Covid death, as reported by Goldin et al., was false. Assuming this conclusion is not challenged, what are the implications?
Some readers might think that refutation of one study does not mean a lot. Goldin et al. are wrong, but there are other studies that support the narrative of a "highly effective vaccine" in the vulnerable population. We did not show that the results from those studies were also false.
That's not how deductive inference works. If VE against Covid death is shown to be far from "highly effective" in one study of the frail elderly, we must deduce that all other studies that reported similar or better VE are false as well — likewise distorted by the healthy vaccinee bias. Otherwise, we have to make an implausible assumption: Despite severe bias, the play of chance miraculously generated the true VE in the study of Goldin et al.
What should happen next?
First, the paper by Goldin et al. should be retracted.Such trials are scientifically justified because residents of nursing homes, the most vulnerable population, were excluded from the original trials (in which death was not an endpoint). Moreover, randomized trials in this unique population become ethically obligatory when corrected VE against Covid death from observational data ranges from mediocre to negative, and there are vaccine-related fatalities.
Second, vaccination of the frail elderly with updated Covid vaccines should be halted.
Third, public health agencies should initiate a Request for Applications (RFA) for placebo-controlled randomized trials of Covid vaccines in nursing homes — with Covid and all-cause mortality as endpoints.
Of course, all of the above is relevant and applicable in another universe.
About the Author:
Eyal Shahar is a Professor Emeritus of Public Health (University of Arizona); MD (Tel-Aviv University, Israel); MPH, Epidemiology (University of Minnesota)
Comment: Correct interpretation is the key. This is one example coming to light.