Epidemiology – Splitting the atom with a breadknife?
“If your experiment needs statistics, you ought to have done a better experiment” – Ernest Rutherford
Statistics are so ingrained in so many aspects of our lives that it is sometimes hard to believe that the statistical approach is a relatively new phenomenon which is far from beyond criticism. It has dominated medical research and practice only since the mid-20th century but is now the principle means by which new pharmaceuticals are judged and the basis for many public health claims.
One major problem with over reliance on the statistical approach is that it fails to adequately take into account our individuality and diversity. Human beings vary enormously in both nature and biology. Our individual biology dictates how we respond to drugs and our individual nature determines how we respond to public health initiatives.
Liberals question the statistical approach to public health because it often seems clumsy and heavy handed, offering poorly defined benefits to the few at the expense of the freedoms of the many.
Medics question the value of new drugs whose efficacy is judged by small statistical effects in large scale clinical trials because whilst these drugs may benefit some patients, the chances are that in many cases they will have no significant effect.
In recent years, growing investment in personalized medicine has begun to challenge the dominance of the statistical approach. The idea is to treat patients as individuals based on their unique genetic make-up and thus offer “the right treatment to the right person at the right time”. As we move further down the personalized medicine path it is likely that medics will further challenge the value of the statistics based approach to their profession. One medic who certainly does is James Penston, author of stats.con
Penston believes that the statistical approach to medicine is flawed and uses both epidemiology and randomized control trials to illustrate his point. His chapter on epidemiology subtitled The Study of Scare Stories is particularly damning. He concludes:
“People are right to be sceptical about medical research. Cohort and case-control studies, and those who promote them, deserve the scorn heaped upon them. A sensible approach to epidemiological studies would be to ignore the results altogether.”
A bit extreme perhaps but he does make a coherent case based on essential weaknesses in the approach and the tendency to place far too much emphasis on very small effects derived using imprecise methods. The reality is that aside from establishing a link between smoking and lung cancer 60 years ago, epidemiology can claim very few successes but this has not deterred the media or the public health industry from bombarding us with statistics and exaggerated claims. It is difficult to take seriously risk factors calculated to x decimal places that are based on individuals responses to questionnaires. Do people really tell the truth about their lifestyles? Can they remember what they did 10 years before their illness began? Are these precise claims really justifiable?
The frequent misuse of the epidemiologist’s efforts by others contributes to epidemiology’s poor reputation as does its coverage in the media. For
example, this BBC article claims to report Tim Key’s study on lifestyle factors and circulating sex hormones in post-menopausal women. Key observed that elevated hormone levels, which are one risk factor for breast cancer, are more closely associated with obesity than other lifestyle factors such as smoking and alcohol. The BBC article is so poorly written that the reader is likely to be misled into believing that obesity is the major cause of breast cancer “shortly followed” by alcohol and smoking. That is not what Key studied but many more people will have read the BBC article than will have read the original paper. This is unfortunate because the contrast between Key’s carefully worded conclusions and the journalist’s interpretation is important.
Bearing in mind the large sums of research funding at stake, the constant meddling of politicians, the evangelical fervour of pressure groups and the ignorance of the media, it is perhaps not surprising that the importance of small epidemiological effects is often exaggerated for public consumption.
One very common method used to achieve impact far greater than the actual significance the data merit is to emphasize changes in relative rather than absolute risk. Using relative risk makes headlines but is often fairly meaningless at the individual level. For example, if a person’s risk of contracting cancer A is 0.01% and a study shows that eating bacon increases that risk by 20%, then the bacon eater increases his actual risk of developing cancer A to 0.012%. The 20% headline is eye catching but most people would not give up bacon based on these odds.
Penston highlights cohort size as another factor misused by statistics adherents. It is an indication of just how dominant the statistical approach has become that we tend to be impressed by studies involving large numbers of people when the real benefit of such numbers is to achieve statistical significance for effects that are otherwise too small to be meaningful.
In a society that allegedly values individual freedoms should we really be basing policy on data that is all but meaningless to the individual and effects that require tens of thousands of subjects to achieve even the smallest significance?
Although I respect the work of some epidemiologists I have to ultimately agree with Dr Penston’s criticism of their collective output because the exaggerated, over-precise claims made by the media and public health industry contrast so sharply with the bluntness of the tools at the epidemiologist’s disposal that they can be reasonably equated to claiming to have split the atom with a breadknife.
By Chris Oakley. Chris has previously posted on Liberal Vision: Smokers-State Aprroved hate and Intolerance is UK policy, Alcohol is Old News – Minimum Pricing for Digestives is the “Next Logical Step” , Soviet Style Alcohol Suppression Campaign Called for By Public Health Activists , Alcohol Taxation: The truth, the whole truth and nothing but the truth , A Liberal Tolerant nation? , What hope is there for liberty if truth becomes the plaything of political lobbyists, Public Health Success? and Could the increasing popularity of harm reduction products impact cigarette consumption?
Tags: Epidemiology, James Penston, stats.con