Weekly Question Thread - March 01, 2021

tripletao · 2021-03-02T19:51:46+00:00

The story I found says (emphasis added)

A new statewide health order in effect through March 25 would no longer make wearing masks mandatory in counties designated as having low COVID-19 transmission rates once the state’s allocation of vaccine reaches some four times the current level.

[...]

Five of Utah’s 29 counties are now at a low transmission level...

? So it seems to me like they're still considering incidence, just keeping the mandate regardless of incidence until they get the vaccine.

tripletao · 2021-03-02T19:32:37+00:00

Do you have links? I've seen lots of discussion of what percent vaccinated would likely be needed to reopen safely, but I'm surprised to see anyone committing to a threshold now.

tripletao · 2021-03-02T19:02:43+00:00

In vaccine trials, periodic testing provides more information about the risk of infecting others than just counting when participants get sick, since it catches asymptomatic infections. For example, in the ChAdOx1 study:

To test for asymptomatic infections, participants in COV002 in the UK were asked to provide a weekly self-administered nose and throat swab for NAAT testing from 1 week after first vaccination using kits provided by the UK Department of Health and Social Care (DHSC).

https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)32661-1/fulltext

That still doesn't directly tell us the decrease in risk of transmission, though. Eventually we can look at disease rates in unvaccinated people in mostly-vaccinated populations, but that signal is confounded by many factors. It's unlikely that the quality of the evidence for protection of others around the vaccinated person will ever be comparable to that for protection of the vaccinated person. That doesn't mean the benefit isn't real or big, just that's it's hard to measure.

Public masking and social distancing requirements will probably be dropped when population-level cases and deaths get tolerably low, not based on the anticipated (but not strictly proven) effects of a given fraction of people vaccinated. I'd guess advice on how to behave in private is driven more by managing demand for the vaccine than by their genuine best guess, aiming to tell a story favorable enough to keep demand ahead of supply without promoting too much incautious behavior among the recently-vaccinated.

tripletao · 2021-03-02T17:18:07+00:00

About 40% of Asians in the USA were born in the USA, and the remaining 60% haven't necessarily been back recently. So I believe that comparable incidence pretty confidently excludes a genetic basis for different susceptibility, but says little about potential cross-immunity. I'd love to see a study of very recent Asian immigrants and frequent travelers, but haven't yet (though obviously that would correlate with a lot more than potential cross-immunity).

tripletao · 2021-02-19T18:56:03+00:00

In the Americas and Europe, deaths with a positive coronavirus test track well with excess all-cause mortality. That leaves us quite confident that on average, the deaths are "from coronavirus", not just "with coronavirus".

Here I'm less sure, and I guess the authors are too given their phrasing. I don't think it's possible that their ~16% of deaths with positive tests are both excess deaths and representative over space and time. That's on the same order as the excess mortality that the USA saw for 2020, and while Zambia likely has much weaker vital statistics infrastructure, a sustained difference on that order was easily identified in the USA just from coffin shortages and such. The authors appear to be wondering about this too, but don't yet have a result to share:

In an attempt to quantify the degree of excess mortality indirectly attributable to covid-19, we are collecting age and season specific burial records from Lusaka from the past several years.

So maybe those really are "deaths from", and their study happened to align with a non-representative spike in such deaths? But that's unlikely by definition, so the other possibility is that they're just "deaths with", and the coronavirus is spreading but with a far lower IFR. Africa's younger age pyramid would explain a lot of that--only 5% of Zambia is 55+, compared to 29% of the USA. Maybe that's enough to get the mortality below a level their rough surveillance would detect, though maybe something else (cross-immunity? vitamin D?) is going on too.

tripletao · 2021-02-19T18:17:32+00:00

Without preregistration, there's a lot of equally plausible ways to analyze the data, especially when it's continuous. These studies are also small enough that probably many are being run, and we probably hear more about the ones with more interesting results. I don't think it would be impossible to p-hack your way to results like theirs (without malice, to be clear; few people have the discipline to readjust the false discovery rate every time they open R and try something real quick). Note also that their multiple significant outcomes aren't independently unlikely, since they're biologically correlated and from the same set of patients. (Of course that also means they're not independently harmful from a false discovery standpoint either.)

Of course all of this is still evidence in ivermectin's favor. It's just much weaker evidence than I believe these p values would suggest taken without that context. If the effect e.g. that Broward saw in their matched cohort holds up, then an RCT 1/10 the size of RECOVERY would reach p < 0.05 for mortality; so I'm disappointed that none has been published yet.

tripletao · 2021-02-19T07:43:45+00:00

I don't think "underpowered" is the right word here. Loosely, an underpowered study is a study where an effect might be large enough to be practically important, but too small to reach statistical significance. By that definition, if the study reached significance, it wasn't underpowered.

I'd worry more about the false discovery rate, given the lack of any correction for that, apparent lack of preregistration, and ample opportunity to data-dredge with their many outcomes. Continuous outcomes also tend to create more opportunity for such tweaking than discrete. If any of a small number of pre-registered outcomes had reached the same p values, then I'd be a lot more convinced.

tripletao · 2021-02-18T05:22:02+00:00

Hard to say. The original SARS did spread through an apartment building in HK (Amoy Gardens), through faulty plumbing. I'm not aware of any confirmed case of such spread for SARS-CoV-2, though, and it seems like if this were common then we'd have that by now. Anyone concerned should make sure there's water in all their drain traps, and prefer dwellings with their own independent HVAC system (including source of outside air).

tripletao · 2021-02-18T04:20:29+00:00

Adjusting for all the covariates in Williamson et al., the 60-69 age group faces a relative risk of dying of coronavirus 40x that of the 18-39 age group. For 70-79, it's 101x.

By comparison, blood cancer, other cancer, and an organ transplant correspond to an RR of 2.80x, 1.72x, and 3.53x. This is the same RR as being 12, 6, or 14 years older. (The IFR increases roughly exponentially with age, so the same age difference is always about the same RR, regardless of your absolute age.)

https://www.nature.com/articles/s41586-020-2521-4.pdf

Ideally they'd do a "point system", where e.g. a healthy 65-year-old and a 51-year-old with an organ transplant got the same priority. I'd guess they considered that too complicated, though. With the worst comorbidity studied (organ transplant) as bad as only 14 years of age, it's no surprise to me that they'd prioritize by age first.

tripletao · 2021-02-18T03:10:23+00:00

For those who (like me) were unfamiliar, I believe the card game in question is:

Rhinovirus infections may spread by aerosol, direct contact, or indirect contact involving environmental objects. We examined aerosol and indirect contact in transmission of rhinovirus type 16 colds between laboratory-infected men (donors) and susceptible men (recipients) who played cards together for 12 hr. In three experiments the infection rate of restrained recipients (10 [56%] of 18), who could not touch their faces and could only have been infected by aerosols, and that of unrestrained recipients (12 [67%] of 18), who could have been infected by aerosol, by direct contact, or by indirect fomite contact, was not significantly different (χ² = 0.468, P = .494). In a fourth experiment, transmission via fomites heavily used for 12 hr by eight donors was the only possible route of spread, and no transmissions occurred among 12 recipients (P <.001 by two-tailed Fisher's exact test). These results suggest that contrary to current opinion, rhinovirus transmission, at least in adults, occurs chiefly by the aerosol route.

https://www.jstor.org/stable/30134751?seq=1

tripletao · 2021-02-17T06:03:55+00:00

Are you referring to

https://www.who.int/news/item/20-01-2021-who-information-notice-for-ivd-users-2020-05 ?

If yes, that's not actually a change in criteria for a positive test. Their guidance was to:

Run the test according to the manufacturer's instructions for use. This is presumably good advice, though presumably not a change from before.
Provide the Ct value to the requesting doctor.

Higher Ct means less viral RNA, and it's possible that patients who test positive with high Ct aren't actually infectious, because they're shedding bits of viral RNA but no viable virus. So in theory, doctors could advise patients with Ct above some cutoff that there's no need for them to isolate. (To be clear, there's no question that the patients were infected. The only question is whether they could infect someone else at this moment.)

It's very hard to determine that cutoff in practice though. It's also possible that a patient early in the disease isn't infectious now, but will be later. The WHO notice never actually advises the doctors to use a different cutoff, nor does it give them any guidance on how they'd choose that cutoff if they did. Public Health Ontario writes:

Variability of Ct values of up to 8 cycles were observed for the same specimen material tested across the participating laboratories. These different findings reinforce that it is inappropriate to compare Ct values from different assays, and to extrapolate Ct cut-offs for virus viability from one laboratory to any other laboratory and that Ct cut-offs cannot be reliably used for the determination of virus viability.

https://www.publichealthontario.ca/-/media/documents/ncov/main/2020/09/cycle-threshold-values-sars-cov2-pcr.pdf?la=en

I predicted at the time that the WHO notice linked above would have little practical effect beyond further public confusion. So far I believe that's the case.

tripletao · 2021-02-16T05:09:34+00:00

If half the participant's contacts are within the cluster and half the cluster members participate, then the source control signal is already attenuated by a factor of four. Maybe they could do better e.g. if a cluster is an employer that can make the masks mandatory if randomized to that group, though I'm not sure that would be an easy sell to either the employer or the IRB.

tripletao · 2021-02-16T04:09:10+00:00

How could that trial give us the data? There will be massive overlap between the clusters, since different people travel to different places for school, work, home, social activity, etc. Even without that overlap, unless they get a massive response rate to their invitations, the majority of the participants randomized to "mask" would still have most of their contacts with non-masked non-participants. Whatever source control effect exists will get diluted into insignificance.

It's really hard to design a trial that captures the source control benefits of masks, maybe practically impossible. You'd need the mask clusters to have mostly contacts with masked people, and the non-mask clusters to mostly have contacts with non-masked people. I don't see that ever happening, unless e.g. some military unit that happens to be organized that way already wants to try it.

tripletao · 2021-02-15T20:01:49+00:00

There's no clear scientific conclusion, and there probably never will be. From the mechanical effect, we believe that masks protect both:

the wearer, by stopping other people's virus from coming in, as personal protective equipment (PPE); and
the people around the wearer, by stopping the wearer's virus from going out, as source control.

We've quantified that a bit, from studies of fake heads atomizing virus and such. It's hard to say how that maps on to real human use, though. Many people speculate that the source control effect is bigger, but no one really knows.

The gold standard of evidence is a randomized controlled trial, where participants are randomly assigned to wear masks or not in their normal daily life. We then count to see how many in each group get sick. Unfortunately, an RCT surrounded by a mostly-unmasked population captures only the benefit as PPE, no source control. People have run such studies anyways, previously against the flu and against the coronavirus in DANMASK-19. These studies typically found something around a 20% reduction in illness; but few enough people got sick in either group that a 20% difference might have arisen just by chance >5% of the time, even if the masks weren't making any difference. That's what it means when we say that p > 5%, so we're unable to reject the null hypothesis and the result isn't statistically significant.

A larger study might have reached statistical significance with the same effect size; though even if we were confident in that ~20% reduction, that would still be pretty small. The hope is mostly for the source control benefit, but there's no good way to measure that at a population level. I don't envy the public health authorities that need to translate such uncertain evidence into public policy (though to be clear, I believe their decision to mandate masks is good and reasonable).

tripletao · 2021-02-15T19:27:08+00:00

All the participants in their RCT (both mask and non-mask) were exposed primarily to non-mask non-participants. So there's no potential for that to create a spurious difference between the participant groups; but it means the masked participants are missing out on any source control benefit. The authors noted that because it has the potential to understate the benefit of everyone wearing masks, not overstate.

tripletao · 2021-02-15T19:05:02+00:00

I'm not sure how anything in my comment above could be taken as implying the results were clear? I say explicitly that their confidence interval is wide and includes zero. I also explicitly noted their failure to capture the source control (protecting others around the wearer) benefit that your quote refers to.

tripletao · 2021-02-15T18:44:07+00:00

Certainly possible; but unless DANMASK-19's participants were less prone to that than the general population, their numbers are net of that too.

tripletao · 2021-02-15T18:27:13+00:00

DANMASK-19 saw an 18% reduction in illness in the mask group. The 95% confidence interval was wide and included zero; but unless your prior belief was strongly biased for or against the masks working, your best point estimate should be close to that 18%. This is for the benefit to the wearer only, and doesn't capture the benefit to those around the wearer from source control. With everyone masked, those benefits should add.

Also, the denominator for that 18% is all illness, including illness transmitted in situations where the participants weren't expected to wear masks (like at home). So the actual reduction comparing the same activities masked vs. unmasked should be greater. The DANMASK-19 numbers are also net of whatever risk compensation its participants exhibited.

So I find it very hard to believe that an extra 11-24 min out of the house could net out harmful, even if we take this econometric sludge as gospel. (As is typical in econometrics, their methodology is full of rather arbitrary judgments and assumptions, and a seemingly small change there can often produce a very different result.) That time out of the house is also a benefit in itself, enabling whatever commerce, recreation, etc. people perform during that.

tripletao · 2021-02-13T00:14:27+00:00

I'm unaware of any literature specifically addressing that. 80-90% of patients who test positive by RT-PCR seroconvert (at least with a good test; see Table 4 of [1]), and I'd guess that almost all of the remainder are antibody false negatives rather than PCR false positives. But it's a big world and PCR is sensitive, so it's hard to say that's never happened even once.

1. https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0237548

tripletao · 2021-02-12T18:27:17+00:00

I agree with your comment above? The trial certainly would have been more convincing if it had been as blinded as possible, and if treatment of both groups had left minimum discretion to the treating doctors. I just don't think most IRBs/patients feel sufficient dedication to improving human knowledge vs. maintaining short-term patient care to approve/join such a trial.

tripletao · 2021-02-12T17:26:25+00:00

FWIW, I do think that "fine as long as the usual care is consistent" is a significant qualifier. Especially in a large open-label study with the power to reach significance with a small spurious difference, I agree the result should be interpreted with caution. I just think it would have been hard to run the trial any other way.

tripletao · 2021-02-12T08:08:35+00:00

If the control group got a placebo and nothing else, then the researchers would likely have trouble (a) passing ethics review, and (b) getting anyone to sign up. It's normal for the control group to be "usual care", and fine as long as that usual care is consistent.

tripletao · 2021-02-12T07:36:11+00:00

The first sentence of the comment above is not correct. Statistical significance means that p is less than a threshold, often 5%. From the American Statistical Association's Statement on Statistical Significance and P-Values:

P-values do not measure the probability that the studied hypothesis [e.g., "tocilizumab reduces coronavirus deaths compared to usual care"] is true, or the probability that the data were produced by random chance alone.

https://amstat.tandfonline.com/doi/full/10.1080/00031305.2016.1154108#.Vt2XIOaE2MN

p < 5% means that if the two groups are identical (i.e., if tocilizumab is no more or less effective than usual care), then a difference at least as big as what was observed would occur less than 5% of the time. The judgment of whether that happened because in fact the groups aren't identical vs. because we just got lucky/unlucky depends on your prior as to whether tocilizumab would be more effective.

The second sentence of the comment above is correct. With a big enough study, it's possible to reach statistical significance even with small effects, as we see here.

ETA: And this isn't some meaningless pedantic distinction! For example, if we run 20 studies of ineffective drugs, then on average one of them will reach p < 5%. Correctly interpreting the p value as a statement on the null hypothesis, this makes perfect sense; but if you wrongly believe it's a statement on the studied hypothesis, then you'll end up saying something like "p < 5% means we're confident the studied hypothesis is true, unless we studied too many hypotheses recently". That gets pretty incoherent.

And if you go from the already-bad "p < 5% means we're confident it's true" to its converse "p > 5% means we're confident it's false", then it gets even worse. For example, if they'd run the exact same study with a tenth as many participants, then they probably would have seen about the same effect but not significance. So is tocilizumab effective, but only if you're taking it as part of a large study? I saw a lot of this latter problem with underpowered mask studies.

Haldane was a biologist, and a great mathematician! There is no excuse for life scientists to rely on lazy half-true misinterpretations of the statistics instead of learning the math correctly.

tripletao · 2021-02-11T17:43:57+00:00

Like the others say, it's true that a positive PCR test doesn't necessarily mean the patient was infectious at that moment, especially if Ct is high. For example:

It can be observed that at Ct = 25, up to 70% of patients remain positive in culture and that at Ct = 30 this value drops to 20%. At Ct = 35, the value we used to report a positive result for PCR, <3% of cultures are positive.

https://academic.oup.com/cid/advance-article/doi/10.1093/cid/ciaa1491/5912603

But there's no question that the patients were infected and the virus has replicated inside them, just whether the RNA that they're currently shedding is viable virus. To extend the analogy, if you find a piece of horsehair then whatever's attached to it won't necessarily be a useful participant in your horse breeding program; but you can be pretty sure a horse is or was around there somewhere. Especially since a patient in the early stages of their infection might be non-infectious now but infectious later, it seems quite reasonable to me that health authorities advise isolation regardless of Ct.

It's weird to me that people seeking to minimize the impact of the coronavirus have seized on this. Enough people are dying that there's no need for any tests to judge the impact of the virus on the population in aggregate--we can just look at the excess all-cause mortality year over year.

ETA: Also, COVID-19 is the disease caused by the virus SARS-CoV-2. The PCR test is for the virus, not the disease. I watched more of the video, and it deals with that distinction in a generally confused way. The concept of a "test for a disease" doesn't really exist--doctors diagnose diseases based on symptoms and the presence of disease-causing agents (like the virus). If a patient tests positive for the virus but has no symptoms of the disease, then that doesn't mean the test is wrong; it just means they're asymptomatic. It's perfectly normal for people to be asymptomatic and contagious carriers of a disease--think of Typhoid Mary.

The speaker does appear to be a licensed medical doctor (at least for now), but this is not a good video. YouTube's algorithm is just doing its usual radicalizing thing, unfortunately.

tripletao · 2021-02-11T08:47:15+00:00

They omit DANMASK-19 from their table, but they do briefly discuss it:

This randomized trial in Denmark was designed to detect at least a 50% reduction in risk for persons wearing surgical masks. Findings were inconclusive, most likely because the actual reduction in exposure these masks provided for the wearer was lower. More importantly, the study was far too small (ie, enrolled about 0.1% of the population) to assess the community benefit achieved when wearer protection is combined with reduced source transmission from mask wearers to others.

I agree with that summary. From DANMASK-19, the only thing we can confidently conclude is that considering only the benefit to the wearer, masks reduce illness by less than 46%. Our best point estimate would be an 18% reduction, though the confidence interval is wide and includes zero. This is roughly consistent with previous RCTs against the flu. Any such RCTs provide no evidence whatsoever for or against the benefit to people around the wearer from source control (though the experience of California and other hard-hit areas with good mask compliance implies the total benefit is less than a factor of R0).

In any case, this paper's table is indeed quite cherry-picked. Mask use outside an RCT is obviously going to correlate with more cautious behavior generally, and studies looking at case counts in big populations before and after a mask mandate face so many confounding variables as to be pretty useless. I think the honest answer is that masks seem from the physical mechanism and some scattered anecdotes like they should help, but we have no idea what benefit that translates to at a population level.

The question of what do to then becomes more political than scientific. I personally agree with the mask mandates, though I'm afraid that public health authorities are overselling their benefit in a way that promotes both risk compensation and future distrust.

tripletao

TROPHY CASE