Dodgy gambling statistics incoming

The Gambling Commission warns people against taking its own data too seriously

Media reporting of gambling statistics is woeful in the UK. Problem gambling is routinely referred to as ‘gambling addiction’, which it isn’t, and the number of people attending NHS problem gambling clinics is taken as a proxy for problem gambling rates (which it isn’t; we just keep opening more clinics). Loyal readers will know that claims about the number of gambling-related suicides are fictitious.

Newspapers often report statistically meaningless increases in problem gambling estimates but do not report the equally common - and equally meaningless - declines, leaving readers with the impression that problem gambling rates are always doubling. For example, in 2013, the Independent reported that the number of problem gamblers had ‘doubled in six years to almost 500,000’. In 2016, the Times reported that the number of problem gamblers had ‘almost doubled in three years from 0.4 per cent of the population to 0.7 per cent, the equivalent of 336,000 people’. Such claims are mutually incompatible. The reality is that problem gambling rates have been hovering around 0.5% ever since we started measuring them in 1999.¹

The Gambling Commission occasionally tries to get such errors corrected. In the last year, it has written to the Guardian, Gambling with Lives and the House of Lords (among others) about the misuse of statistics. For example, when the Guardian’s Rob Davies wrote:

Revenue from online casino products, which have a much higher rate of addiction than sports betting, according to a 2018 NHS survey, reached a high of £4.01bn in the Covid-affected year to the end of March 2021. It has now exceeded that total, reaching £4.03bn.

The Commission explained that:

There are no official statistics on the rate of gambling addiction across the population or in relation to individual products. Data on rates of problem gambling cannot automatically be equated to rates of addiction. It would be a misuse of Officials [sic] Statistics to claim rates of addiction.

(They could also have mentioned that the amount spent fell in real terms but perhaps that’s beyond their remit.)

Correcting such misinformation is a Sisyphean task and it is about to get much harder thanks to the, er, Gambling Commission. Last year, I mentioned a pilot survey conducted by the Commission which produced unfeasibly large problem gambling estimates…

Official statistics show that the rate of problem gambling in England is 0.3 per cent. This is low by both historical and international standards and so the Gambling Commission has decided to hike it up. New, experimental statistics released last week report a problem gambling rate of 2.5 per cent. As the Guardian eagerly reported, this means that “UK problem gambling rates may be eight times higher than thought”.

This is thanks to a new methodology in which a randomly selected group of people are asked whether they’d like to complete a survey about gambling. If they say yes, they are asked to answer up to 200 questions, not just about their gambling but about their general lifestyle and attitudes.

The problem is that most people do not say yes, and those who do are more likely to be gamblers than average and are even more likely to be problem gamblers. People who are interested in a subject are obviously more likely to be interested in a survey about it.

Previously, the gambling questions were just one part of a broader health survey and the survey was conducted face-to-face, with respondent self-completion for sensitive questions. This got a higher response rate and there was no way it could have a particular appeal to gamblers.

The flaws in the new methodology are well known, as Patrick Sturgis said in his report for the Commission:

Until there is a better understanding of the errors affecting the new survey’s estimates of the prevalence of gambling and gambling harm, policy-makers must treat them with due caution, being mindful to the fact there is a non-negligible risk that they substantially over-state the true level of gambling and gambling harm in the population.

The 2020 pilot got a response rate of just 21%, compared with around 50% for the earlier health surveys and it came up with a problem gambling rate three or four times higher. And yet next week, a survey with a similar methodology will be published as the Gambling Survey for Great Britain (GSGB) and its statistics will become official. Why? Basically, because it’s cheap. So from now on, this is how the problem gambling rate will be measured.

Furthermore, because the survey also includes questions about mental health, crime, advertising, drinking, drug use and much more, it will be easy for people to draw crude statistical associations and treat it like an epidemiological study.

You don’t need to be too cynical about the media to foresee what is going to happen. We will be told that the rate of ‘gambling addiction’ has soared, that millions of people are gambling ‘addicts’ and that X, Y and Z ‘cause’ gambling ‘addiction’.

The Gambling Commission can see this coming, but rather than use a better methodology, it has issued some lame guidance in advance of next Thursday’s release. Noting that the response rate in the new survey is just 19% and reminding us that Patrick Sturgis warned that it could “substantially over-state the true level of gambling and gambling harm”, the Commission has slipped out some advice, saying:

There is a risk that the GSGB may overstate some gambling behaviours and therefore estimates should be used with some caution.

Good luck with that.

The GSGB can be used with some caution (until further work is completed):

• to provide estimates of Problem Gambling Severity Index (PGSI) scores amongst adults (aged 18 and over) in Great Britain

• to provide estimates of the prevalence of consequences of gambling amongst adults (aged 18 and over) in Great Britain.

What does this even mean? Either the stats will give an accurate reflection of the number of problem gamblers in Britain (in which case no ‘caution’ is required), or they will not (in which case they shouldn’t be used at all). You can’t put out dodgy statistics and then ask the media to exercise restraint. You know they won’t and, frankly, why should they? They’re your statistics. Take some responsibility.

1. This graph uses data from three different surveys and after 2010 relates to England rather than Great Britain. That can’t be helped as the government keeps hanging the methodology. There appears to be a decline in recent years, but that is not certain. The confidence intervals are very wide. We can, however, be pretty confident that there has not been a rise.