I don’t write much about e-cigarette junk science because there is too much of it and it is just too depressing. But I just came across this effort, which I missed earlier in the year, because I saw someone tweet about it. It’s worth looking at as a cautionary tale about statistics.
It’s the first study I’ve come across that purports to show that vaping gives you cancer. The headline claim is:
The e-cigarette users have lower prevalence of cancer compared to traditional smoking (2.3% vs. 16.8%; P < 0.0001), but they were diagnosed with cancer at a younger age.
Even the suggestion that vapers have a (much) lower rate of cancer than smokers seemed doubtful after they did a regression analysis…
Our regression analysis showed that e-cigarette users have 2.2 times higher risk of having cancer compared to non-smokers (odds ratio (OR): 2.2; 95% confidence interval (CI): 2.2 - 2.3; P < 0.0001). Similarly, traditional smokers have 1.96 higher odds of having cancer compared to non-smokers (OR: 1.96; 95% CI: 1.96 - 1.97; P < 0.0001).
See that? Vapers have a higher cancer risk than smokers, according to this piece of research.
The authors looked at data from the USA’s National Health and Nutrition Examination Survey (NHANES) between 2015 and 2018. They found 154,856 participants, of whom 5% were e-cigarette users, 63.6% were nonsmokers, and 31.4% were ‘traditional smokers’ (the authors insist on calling smokers ‘traditional smokers’, presumably to imply that vaping is a form of smoking). The survey has a question asking whether the participant has ever had cancer.
None of the vapers in the study would have been vaping for more than a decade and the majority would have only been vaping for a few years. Since ‘traditional smoking’ typically takes several decades to cause cancer, the idea that a few years of vaping would have the same - or bigger - effect is deeply implausible, even if you ignore the fact that vapour doesn’t contain smoke and has far fewer, if any, potential carcinogens.
Since the vast majority of vapers are former smokers, you would expect them to have a higher cancer rate than lifelong nonsmokers, all things being equal. But as it happens, none of the vapers had ever had lung cancer.
Looking at all cancers combined, the rate was lowest among the vapers (2.3%), highest among the smokers (16.8%), with the nonsmokers in between (9.5%).
If it seems a little odd that the nonsmokers didn’t have the lowest rate, you need to consider that the average age of the nonsmokers was 50, the average age of the smokers was 62, but the average age of the vapers was just 25. Age is the biggest risk factor for the vast majority of cancers.
Once the researchers adjusted the data for confounding variables, the cancer rates were as mentioned at the start of this post: a relative risk of 2.2 (2.2-2.3) for vapers and 1.96 (1.96-97) for smokers, as compared with nonsmokers.
The authors don’t say which confounding factors they adjusted for. The obvious one would be former smoking, but it is not obvious that they did adjust for it. The fact that vapers were more than twice as likely to get cancer than nonsmokers in the adjusted analysis - and were even more likely to get cancer than the ‘traditional smokers’ - suggests that they didn’t (or that they didn’t do a very good job if they did). There is no credible biological mechanism for a few years of vaping being more dangerous than a lifetime of smoking.
The authors say they had data on age, sex, race, annual household income and various comorbidities so presumably some of these were adjusted for. But the only questions about smoking were “[Have you s]moked at least 100 cigarettes in life” and “Do you now smoke cigarettes?”, so maybe there was no question about whether they used to smoke? If so, this is an absolute howler.
The researchers may have been aware of this shortcoming and were slightly embarrassed about the extraordinary finding that vaping is worse than smoking because they don’t lead with it in the abstract or the conclusion. Instead they focus on the arguably less insane finding that vapers who developed cancer got it at a younger age than nonsmokers.
But which cancers were vapers more likely to get? Not lung cancer. None of them got that. Nor did any of them get cancer of the bladder, colon, prostate, uterus or kidney. Of the vapers who got cancer, they were less likely to have had breast cancer and melanoma than were the nonsmokers.
The only cancers that were more common among the vapers - not in absolute terms, but as a proportion of cancers among the people who had ever had cancer - were cervical cancer, thyroid cancer, leukemia and one form of skin cancer. What these have in common is that they are either not ‘smoking-related’ (or not strongly associated with smoking) and have a tendency to affect younger people more than most cancers.
Smoking is associated with a lower risk of thyroid cancer and two-thirds of all thyroid cancer cases are found in people aged between 20 and 55. The highest rates of cervical cancer are among women aged between 30 and 35 and the disease is caused by the human papillomavirus. It’s not clear from the study what type of skin cancer vapers had a greater risk of, but smoking is associated with reduced risk for two types and an increased risk for one. The only outlier is leukemia which was weirdly high among the e-cigarette users who had cancer, accounting for 33 cases (8.5%) as opposed to 1-2% of the smokers’ and nonsmokers’ cases. Childhood leukemia is obviously associated with younger people and has a high survival rate, but it still seems odd.
I can’t explain that bit, but how likely is it that vaping not only causes cancers that smoking doesn’t cause, but also causes them at a younger-than-average age?
The answer is that it almost certainly doesn’t. The authors have fallen into the survivorship bias trap. They say:
Interestingly, cancer respondents had a lower prevalence of e-cigarette use than traditional smoking (2.3% vs. 16.8%), and e-cigarette users were diagnosed with cancer at a younger age than respondents with traditional smoking (median age of 45 years vs. 63 years).
These two statements might seem hard to reconcile on the face of it, but they can both be explained by the e-cigarette users being much younger than the other two groups. The vapers had much lower rates of diabetes, heart disease and stroke for the same reason.
The authors could have pointed out that the ‘cancer respondents’ were more likely to be nonsmokers than vapers, but this has the same explanation: people with cancer are disproportionately old and old people are less likely to vape.
Very few of the vapers had experience of cancer because they were younger than the other groups, but because they were younger than the other groups the ones who got cancer were more likely to have developed it when they were young! A 30 year old vaper can hardly claim to have developed prostate cancer when he was 80.
This is a classic case of immortal time bias. You need to wait until these people are very old, or preferably dead, before you can draw any conclusions from data like this.
You may recall the study which claimed that rock stars die younger than ordinary people and rap stars die youngest of all. This is probably true, but we don’t have the data to prove it until all the rock and rap artists dead. The only rap stars who have died are the ones who died relatively young. No rock star has yet lived to 100 but one of them probably will.
We know the average age of death of the ones who died. We have no idea when the ones who are still alive will die. Keith Richard is still going strong but people like him - i.e. the living - were excluded from the study by design.
In The Spirit Level Delusion, I discussed the study which claimed that actors who win Oscars live longer than actors who don’t. And so they do - but not for the reasons given by the authors (clue: you have to live long enough to win an Oscar).
It’s an easy mistake to make, I guess, but the authors of the vaping study should have been aware of it. Their research tells us absolutely nothing. The fact that vapers with an average age of 25 are less likely to have had cancer (as of 2018) than nonsmokers who are twice their age is entirely unsurprising, as is the news that the relative handful of vapers who have had cancer developed the disease at a lower-than-average age.
Don’t these things get peer reviewed?
Update: 3 January 2023
I’m pleased to say that this study has now been retracted.
After publication of this article, concerns have been raised regarding the article’s methodology, source data processing including statistical analysis, and reliability of conclusions, the authors failed to provide justified explanations and evidence for the inquires, subsequently this article has been retracted at the request of Editor-in-Chief.
Thanks for reading The Snowdon Substack! Subscribe for free to receive new posts and support my work.
More crap being portrayed as "science" by the king of junk science. We've all been "Glantzed" again...
Hi, Chris. I am late to this, but found it and quite liked it. Nice work.
I am struggling a bit with where you think you saw saw immortal person-time errors. There is clearly survivorship bias (people who would have been current smokers with leukemia or cervical cancer often did not live long enough to be included, whereas the vapers with those diseases are likely still alive even if their prognosis is poor). There might also be some reverse causation there (people with cancer at an early age are probably more likely to avoid smoking). But there is no person-time measure, right? So there cannot really be immortal time in the denominator. Survivorship bias is a lot like immortal person-time bias, and they are close to interchangeable in some cases, but this is not really one of those cases.
Also, you let them off too easy on the "controlling for age" thing. It is really almost as bad as the failure to control for former smoking (which, as you say, is the clear central fatal flaw in the analysis). There is no possible way you can make a comparison of age-dominated diseases between a group who average 25yo and groups that measure c.60yo without doing a stratified analysis. Any other method of "controlling" is just going to produce model-driven nonsense.