Browse > Home / pseudo science / Lies, damn lies, statistics & meta-analysis – their contribution to the weak case for minimum pricing

| Subcribe via RSS



Lies, damn lies, statistics & meta-analysis – their contribution to the weak case for minimum pricing

October 3rd, 2012 Posted in pseudo science by

Few days go by without the public being subjected to some health scare or miracle cure delivered to them by an ill-informed but very enthusiastic media. Generally, these stories refer to “a new study” or “latest research” implying that “scientists”, “doctors” or “experts” have actually performed a novel experiment that has scientifically demonstrated something new and potentially useful.

In the vast majority of cases, this isn’t true. What has often happened is that a special interest group has reviewed some historical data, re-analysed it, applied a bit of spin in support of their case, published it somewhere not over insistent on scientific rigour such as a medical journal and issued a press release usually full of impressive sounding numbers.

These reviews appear in many formats but all suffer from the fundamental problem that they tend to conclude whatever the authors want them to. The most advanced form of this data manipulation epidemic is the meta-analysis, which can be viewed as a sort of amplifier. The idea is to take a number of studies that may be individually insignificant or even contradictory and combine them in a way that enhances consistencies.

Two major limitations of the approach as identified by numerous academic sources are publication bias and agenda bias. These factors are especially problematic in public health which is observably doctrinaire.

Publication bias normally refers to the tendency for positive results to be more likely to be published than those that support the null hypothesis thus distorting collective analysis of outcomes. Public health publications are often policy driven rather than objective or evidence based and the dogmatic nature of this approach fuels a more extreme form of publication bias caused by suppression of non-conformist ideas.

The depth of this problem was exposed in 2003 when the BMJ published Enstrom and Kabat whose work suggested that passive smoking appeared to be less lethal than previously claimed. The authors were set upon by the public health industry and The BMJ itself was subject to attack for its heretical challenge to public health orthodoxy. The vast majority of critics didn’t even address the content of the paper. The unsavoury incident led Ungar and Bray to write Silencing Science in which they conclude that an intelligent debate on the effects of passive smoke has become impossible. Irrespective of the debate over its content, the reception of the BMJ paper serves to illustrate the extreme extent of publication bias in public health.

Public health also suffers from agenda bias. The bedrock of science is sceptical objectivity and this is particularly important with meta-analysis because freedom to choose which studies to include, how to weight them and how to interpret the results introduces a degree of subjectivity.  In science the tendency to use this freedom to engineer favourable outcomes is usually offset by the value placed on scientific /academic integrity. Such ethical considerations are less restraining in public health where the discourse is dominated by policy driven orthodoxy rather than a desire for genuine discovery and the adversarial exploration of competing hypotheses.

In public health the authors of meta-analyses are all too often wishful thinking medics or public health activists who exhibit a depressing tendency to make the data fit the theory. Attempts to justify smoking bans by claiming dramatic post-ban falls in heart attacks have unsurprisingly produced some of the most unconvincing meta-analyses including this from Stanton Glantz a man obsessed by his personal war against tobacco and this produced by a cardiologist from Kansas

The outcomes of these meta-analyses are pre-determined by biased study selection. The authors chose to ignore the poor quality, methodological weaknesses and arguably fraudulent nature of the selected studies highlighting another weakness of meta-analysis. The authors should have been exposed by the “success” of their efforts which imply that 15-20% of heart attacks are caused by passive smoking. This is implausible to the point of being ridiculous but amazingly, their output is still referenced.

The dubious use of meta-analysis is not confined to tobacco control. Many of us have been left scratching our heads by claims made for minimum alcohol pricing. The notion that a modest financial measure that would not inconvenience the majority could have a significant impact on problem drinking and youth drinking appears counterintuitive and depends on some odd assumptions about price elasticity and behaviour.

The politician’s claims are based on the work of Petra Meier who derives much of her theory from meta-analyses. Based on Gallet’s 2007 interpretation of 132 studies dating back to 1945 she concludes “if the price of beer is raised by 10%, beer consumption would fall by 3.5%; if the price of wine was increased by 10%, wine consumption would fall by 6.8%; and if the price of spirits increased by 10%, spirits consumption would fall by 9.8%.”

In some ivory towered fairyland perhaps but in real life here in the UK, the leap from Gallet’s findings to “A 50p limit should cut alcohol consumption among moderate drinkers by about 3.5%, or half a unit for women and two-thirds of a unit for men” together with rest of the hyperbolic nonsense in this fabulously biased BBC article is hard to understand or justify.

I can imagine the Scottish government falling for this on the basis that any data however ridiculous is an improvement when you are used to your health secretary simply making up the numbers to support her agenda, but surely David Cameron should be better advised than this?

Reading Craig Gallet’s 2007 paper on which Meier relies heavily and being mindful of her claims with respect to targeting young drinkers, I was struck by a line in the conclusion:

“… if we are particularly concerned with teenage drinking, since we find that teens are least responsive to price, then perhaps the best approach to reducing teen alcohol consumption should involve alternatives to taxation, such as education campaigns.”

This is not the only inconsistency in this classic case of torturing the numbers to fit pre-determined policy.

By Chris Oakley. Chris has previously posted on Liberal Vision:  Smokers-State Aprroved hate and Intolerance is UK policy,   Alcohol is Old News – Minimum Pricing for Digestives is the “Next Logical Step” , Soviet Style Alcohol Suppression Campaign Called for By Public Health Activists , Alcohol Taxation: The truth, the whole truth and nothing but the truth , A Liberal Tolerant nation?What hope is there for liberty if truth becomes the plaything of political lobbyists and Public Health Success?

Note from Editor

You might also find the following posts interesting:

BBC And Guardian Played Like Fools On Minimum Alcohol Pricing by Dick Puddlecote 3/10/2012

A black market in booze fearmongering by Chris Snowdon for Spiked! 3/10/12012

Lies, Damn lies and Sheffield University by The Pub Curmudgeon 3/10/2012

10 Responses to “Lies, damn lies, statistics & meta-analysis – their contribution to the weak case for minimum pricing”

  1. Sally Says:

    Dear Chris

    Call me naive but I am totally shocked by this.

    What, apart from people like you exposing what is effectively fraud, can be done about it? Do you have any ideas?

    Thanks for raising it and explaining it without too much techie speak,


  2. Eric Crampton Says:

    I’m no fan of minimum prices. But the best estimates we currently have of the elasticity of demand with respect to changes in minimum prices say that increasing the minimum price of alcohol by 10% reduces aggregate consumption by about 3.5%. Because the group consuming the lowest priced product includes both the more harmful drinkers and poorer moderate drinkers, I really don’t like minimum pricing: it does too much harm to poorer sensible consumers. But it would be odd if prices didn’t affect consumption, wouldn’t it?


  3. Dave Atherton Says:

    Chris you maybe interested in Rand Corporation’s review of 217,000 heart attack deaths, 2 million heart attacks covering all 50 states and 468 counties over an 8 year period in the USA. It even goes onto suggest that those studies which purport to show a reduction are no more than publication bias like Professor Jill Pell’s heart attack study.

    The Abstract says it all: “In contrast with smaller regional studies, we find that smoking bans are not associated with statistically significant short-term declines in mortality or hospital admissions for myocardial infarction or other diseases. An analysis simulating smaller studies using subsamples reveals that large short-term increases in myocardial infarction incidence following a smoking ban are as common as the large decreases reported in the published literature. “

    http://onlinelibrary.wiley.com/doi/10.1002/pam.20548/full


  4. Dave Atherton Says:

    The backwash of the Enstrom/Kabat study was even more evil than you can imagine. Dr. John Seffrin CEO of the American Cancer Society, salary $2,220,00 a year and Dr. Michael Thun wrote to Wyatt Hume the Provost of University of California, Los Angeles in 2006 trying to get him fired.

    Here are some of the letters.

    http://daveatherton.wordpress.com/2011/07/05/the-american-cancer-society-let-it-all-hang-out/


  5. Dave Atherton Says:

    The Enstrom/Kabat paper is not unique. There are over 80 studies done into passive smoking, lung cancer and heart attacks, 85% are the null hypothesis, some like Wu-Williams’ and Neuberger papers suggest protection.

    Here they are.

    http://www.pnlee.co.uk/documents/refs/lee2010B.pdf


  6. Dave Atherton Says:

    Without wishing to hog the comments perhaps the most damning evidence is from the Department of Health (DH). The DH set up the Scientific Committee on Smoking and Health (SCOTH)which first reported in 1998 and its report in November 2004 was the “scientific” basis for banning smoking in pubs. Made up of DH funded “scientists,” Big Pharma paid shills and Labour nannies it is an embarrassment.

    Under a Freedom of Information request I have obtained the minutes of their meetings. The relative risk (RR)of the meta-analysis of studies is 1.25 and epidemiologically is statistical insignificance.

    More importantantly the writer of the SCOTH Committe report was Professor Martin Jarvis said in the meeting that ”Most of the studies were not felt to be sufficiently conclusive in their findings.”

    We have been lied to on a Biblical scale.

    http://daveatherton.wordpress.com/2011/12/22/the-scientific-committee-on-tobacco-and-health-scoth-and-the-freedom-of-information-act-most-of-the-studies-were-not-felt-to-be-sufficiently-conclusive-in-their-findings/


  7. Chris Oakley Says:

    Thank you Sally.

    It is hard to stop the publication of dubious studies without harming academic freedom. In that sense it is a similar dilemma to that of protecting privacy whilst maintaining the freedom of the press. One problem these days is that it has become relatively easy to claim peer reviewed publication without your work being too rigorously examined. I do not have space to explain in full here but I might write a short piece on how to produce peer reviewed junk science.

    Another problem is the sheer amount of money poured into public health, which bearing in mind the quality of the output is surprising. A lot of government money goes to pressure groups that produce less than objective pseudo-science in order to lobby the government.
    The problem with the Sheffield study is that it is not truly independent. It was directly commissioned by the DoH after politicians involved with that department had already expressed a preference for the results that they would like to see. It would be a good policy to stop the practice of government directly funding research and introduce more objectivity into the system.

    Much public health research is unfortunately policy led and quite frequently the principle aim of a publication is to gain publicity for some cause or other. So it would help if the mainstream media stopped regurgitating press releases linked to shoddy or fraudulent science. The main culprit when it comes to giving huge publicity to dubious science here in the UK is the BBC. I have spent hours of my life patiently and mostly politely trying to work with the BBC but it is establishment biased so improbably difficult to deal with if you are an ordinary license payer. Despite my best efforts and those of others the BBC continues its uncritical coverage of all things medical and especially public health. I have no doubt that the BBC does have agendas.


  8. Chris Oakley Says:

    @Eric

    The elasticities are calculated using data from the general population or gross consumption figures. The claim that pricing will have a disproportionate effect on target groups because they currently drink lower priced products is pure speculation and according to Gallet (above) may be unfounded.

    I agree that there is a point at which price will impact consumption but I believe that it is a long way beyond the 50p /unit that the politicians are touting. Why stop at 50p when your calculations show how many lives £1.50 could “save”. I have little faith in price elasticities produced by meta-analyses of policy driven publications.

    I do believe that minimum pricing is likely to impact poor responsible adults more than wealthy ones. It is a regressive tax.


  9. Chris Oakley Says:

    @Dave

    I have read the Rand report and most of the literature on passive smoking.

    As I say above, the link between heart attacks and smoking bans is implausible and is based entirely on dubious evidence produced by people who are far from objective. I covered the fraud that is the Scottish heart attack miracle in a previous post. That the perpetrator is still a professor at a UK university speaks volumes about modern academic standards.


  10. Dave Atherton Says:

    My most recent take on plain packs and minimum priced alcohol.

    Plain packs consultation rigging, Andrew Black and the minimum pricing of alcohol. How government lies and manipulates.

    http://www.thecommentator.com/article/1746/government_pouring_our_money_down_the_drain_and_rigging_health_consultations_


Leave a Reply