The minimum pricing salvage job

After years of disappointment, has a new study saved the SNP's blushes?

In January, I mentioned that minimum unit pricing (MUP) in Scotland has a sunset clause and the SNP was running out of time in its search for evidence that the policy works. I made a prediction…

I wouldn’t be surprised if activist-academics pull a dodgy counterfactual out of their hat at the last minute and claim success, as has just happened with the sugar tax. In fact, I'd put money on it.

That day has come with this study published in the Lancet. The headline claim is that minimum pricing ‘led to’ a 13.4 per cent decline in deaths wholly attributable to alcohol consumption and a 4.1 per cent reduction in hospitalisations for the same, although the latter is not statistically significant.

Unusually, the media have given a better account of what the study shows than its authors did. The term ‘activist-academics’ might be a bit strong, but several of them have been advocating for minimum pricing for a decade (e.g. here and here). They are the core of the team that has been evaluating minimum pricing for Public Health Scotland for the last few years and have been unable to find much cause for celebration. MUP seems to have had no effect on A & E admissions or crime. There is ‘no clear evidence that MUP led to an overall reduction in alcohol consumption among people drinking at harmful levels. On the contrary, there is evidence that the heaviest drinking men, in particular, are drinking more.

It may or may not a ‘dodgy’ counterfactual, but today’s study is certainly based on a counterfactual. Nobody is claiming that the number of deaths actually fell by 13.4%. The claim is that the number of deaths is 13.4% lower than it would have been, if we assume the trend in Scotland would have been the same as in England.

In the Herald, minimum pricing modeller Petra Meier (who was not involved in this study but is a big fan of the policy) says:

“If changes happen in both countries, they cannot be caused by the policy which only affected Scotland, but Scotland-only changes are highly likely to be caused by Minimum Unit Pricing, given there were no other major alcohol-related changes that occurred in only one of the countries in the meantime.”

This is a huge exaggeration. Trends often go in opposite directions in England and Scotland for reasons that ‘public health’ researchers neither understand nor usually care about. Most notably, the alcohol-specific death rate fell sharply in Scotland between 2006 and 2012 while remaining broadly flat in England (there were no policy changes in Scotland to explain this). In 2017, the year before MUP began, rates fell in Scotland but rose in England. In 2018, they went up in Scotland and down in England. As a control group, England is very unreliable.

Nevertheless, it is clear that in 2019 - the first full year of minimum pricing - the alcohol-specific death rate went down by about 10 per cent in Scotland while it rose slightly in England. As I said when these figures were first published, this is ‘consistent with the view that minimum pricing reduces alcohol-related mortality’. But it is also consistent with normal year-to-year fluctuation. A fall of this magnitude is hardly unprecedented. The rate fell by 10 per cent in both 2007 and in 2009, and by 15 per cent in 2012. Nobody paid much attention to any of these declines because no one was looking for a policy to pin them on.

The authors of the Lancet study are able to give a more detailed view by showing the monthly figures, but the story remains much the same. The death rate was a little lower in 2019 than in the most of the previous few years. Interestingly, there were an unusually high number of alcohol-specific deaths in Scotland in the months leading up to MUP, for reasons unknown. This makes the subsequent decline look a little more dramatic.

Still, it’s not very dramatic, is it? If the authors hadn’t highlighted when MUP began, you would be hard pushed to put your finger on it.

The graphs showing alcohol-related hospital admissions are even less impressive (n.b. the sharp fall in spring 2020 was due to Covid)…

If you took this data to a health minister in another country and told them that this is what they could achieve if they introduced minimum pricing, I can’t imagine them getting too excited.

COVID-19 is considered an exogenous shock which makes evaluating MUP in 2020 and 2021 very difficult. The authors do not look at 2021 at all, saying only that ‘the increase in the rate in Scotland from 2020 to 2021 (4%) was lower than in England (7%).’ This is misleading. In percentage terms there was a slightly larger rise in England during the pandemic, but in absolute terms there were an extra 3.8 alcohol deaths per 100,000 in Scotland in 2021 compared to the pre-pandemic year of 2019 whereas there were only an extra 3.0 alcohol deaths per 100,000 in England.

Since MUP only affects the off-trade, one might expect MUP to have had a greater effect during lockdown when alcohol was only available from the off-trade, but there is no indication of this in the data.

I will leave it to statisticians to decide whether the model is dodgy. But if we accept that there is a temporal association, the question is about causation. To decide whether or not MUP ‘worked’, we need to look at the totality of the evidence. Is it plausible that a decline in alcohol sales of three per cent would lead to a 13 per cent reduction in deaths? (Even the three per cent reduction in sales is based on a counterfactual; the real figure is 1.1 per cent, and that doesn’t include cross-border sales.) Is it plausible that MUP would reduce the number of alcohol-related deaths without reducing the number of alcohol-related A & E admissions? Or the number of alcohol-related crimes, for that matter? The claim in this study doesn’t fit with the rest of the evidence.

We also have evidence from Wales which introduced MUP in March 2020, a few weeks before the first lockdown. Like the rest of the UK, and much of the world, Wales saw a sharp rise in alcohol-related deaths in 2020 and 2021. There is no indication that MUP softened this rise. In fact, the number of alcohol-related deaths in Wales went up at exactly the same rate between 2019 and 2021 as it did England in percentage terms (27 per cent) and by slightly more in absolute terms (an extra 3.2 deaths per 100,000 in Wales versus an extra 3.0 per 100,000 in England).

The authors of the study nevertheless assume causality:

Our study reports on the final intended outcome and finds that this reduction in sales led to a 13% reduction in deaths and a 4% reduction in hospitalisations. The methods used suggest plausibility that these effects can be causally attributed to MUP.

As Kevin McConway says, the real problem with this quote is the phrase ‘led to’, which clearly assumes causality.

This is where you would be better off getting your information from the media. As the Guardian rightly says…

The research was observational, so cannot prove conclusively that the significant fall in deaths was due to the minimum unit pricing policy.

While the BBC adds the helpful reminder that…

The actual recorded number of alcohol-specific deaths in Scotland has generally been rising since 2012.

The BBC also recognises that the supposed decline in hospitalisations (which actually rose in both 2018 and 2019) is not significant.

But they didn't quite see a difference in hospital admissions or in deaths partially due to alcohol, like liver cirrhosis.

And so this study is evidence for the effect of MUP, rather than proof.

It will, I suspect, be seen as proof by the SNP. This is the study they have been waiting for. All the evidence suggesting that the policy has been a flop will be disregarded and the minimum unit price will be raised from 50p to 65p. Job done.