Lies, damned lies and statistics: How do you turn 61% into 95%?

Image from "Working together for a stronger NHS" Crown Copyright

Edit: 13/05/11 An analysis of the BSA 2007 dataset by Siobhan Farmer, Mark Hawker and myself has been published today in the new publication Lancet UK Policy Matters. You can find it here. We conclude that it is not possible to conclude that 95% of respondents wished for more choice. Using the kind of suppositions given below it may be possible to infer that between 61% and 72% may have thought there should be more choice, but the survey was not designed to answer this question. 

..........................................................................................................................................................

Edit: 17/4/11 I and a colleague have independently tried to verify Mark's analysis. We have reached similar conclusions but they don't corroborate Mark's results. Unfortunately he is currently outside the UK. We will update with our own results in the UK. We are in agreement that there is still no justification for claiming that "95% of people want more choice in healthcare". 

..................................................................................................................................................................

Today many of you will have read Ben Goldacre's excellent  analysis of the leaflet which the Department of Health issued last week to help the public understand why they were pursuing reforms of the NHS which are facing widespread opposition. 

Page 11 of the leaflet contains the graphic above stating that 95% of those surveyed in the 25th British Social Attitudes Survey  wanted MORE choice in the NHS. When Ben looked at the published reports he found that 'Do you want more choice in the NHS?' was not a question in the survey. Instead respondents were asked 'How much choice do you think you should have?' and 'How much choice do you actually have?'  But if you could establish how many people thought they should have choice but who currently think that they don't have choice then you might be able to say how many people think that they should have more choice. As Ben points out to answer this you would need to have individual level data. When he asked the DOH for the dataset they unfortunately pointed him to a book chapter.

But fortunately the day this leaflet was published, April 6th, 2011, Mark Hawker  started wondering if he could find out more about this dataset. And he did find more. The individual level data is available to download. So Mark did that. Then he analysed it. And what did he find?  Well, you can read a lot more in the blog post that he published on April 7th but here is a summary.

13% thought they had more choice that they thought they should have.

46% thought they had just the right amount of choice.

41% thought they should have more choice that they had.

So how did the DOH manage to get this so wrong? How did they confuse 41% with 95%? Why weren't they able to direct Ben Goldacre to the correct data source? And why have they decided not to fund this survey in the future?

Hopefully someone can help make the correct data look just as pretty as the incorrect infographic in the leaflet. In the meantime I think I'd like to thank Mark for his work, and to agree with this tweet:

#1care : crowd sourcing a hashtag for primary care

hashtags, a photo by halfbrown on Flickr.

Don't Think Websites, think data

What the non-geeks including me, need to understand. Check out this SlideShare Presentation:

Don't Think Websites, think data

View more presentations from Mike Ellis.

A medical student's thoughts on empathy and #meded

Patient + Wife a photo by polaroid667 on Flickr.

Today during the twitter conversation Gautam went back to one of my old blog posts on empathy and left a comment that I think deserves its own post, so I have posted it below. Much of what he writes strikes me as very true and accurate. What is the solution?

"I think the crux of the problem is the practitioners that medical students train with. I'm a final year in Sheffield and even though we have 12 weeks of general practice over two separate sessions, that leaves around 2.75 YEARS of training in hospital.

My own opinion is that in-hospital practitioners are less empathic because the prevailing attitude is that patients are problems to be solved. The 'House MD' way of looking at things still prevails amongst many practitioners - particularly surgeons but equally amongst medical physicians. Students are 'taught' empathy but equally, are (not overtly) dissuaded from feeling it, talking about it and dealing with it. The pressure is to deal with the 'real' problems - the broken leg, the tweaking of medication doses - and ignore the 'BS' - the trauma of losing a child or partner (unless they can be referred for CBT).

Contrasting hospital care to general practice, the existence of a lasting relationship between doctor and patient means that these concerns have to be taken more seriously simply because the patient is a recurring figure in the doctor's professional life. In other words, it makes sense to deal with these patients as people, rather than as problems.

Now, this theory of mine (as weakly-backed by evidence as it is!) holds some water, I feel, since patients who are seen in clinic regularly are treated differently by consultants. These patients' problems are listened to, their concerns are heard and dealt with as much as is possible. Time constraints exist with patients on the ward, as well, but for some reason, (perhaps worries about confidentiality and privacy?) they are not covered as completely.

Medical students can be taught to reflect and encouraged to empathise by the medical schools as much as possible. But while they are taught as apprentices by overworked and jaded physicians and surgeons who may not have time to empathise as much as they'd like to, true empathy remains out of reach."

Thank you, Gautam.

A twitter conversation with UK medical students about empathy

Antidepressant prescribing in England: variation may not be as great as Guardian map suggests.

The Guardian published a story on March 5th, 2011, showing that the crude rate of anti-depressant prescribing in some areas of England was more than 3 times higher than in others.The highest rate of prescriptions was in Blackpool, and the lowest in Kensington and Chelsea.
The data was from publicly available sources and is linked to from the Guardian website. The journalists give this description of their method :
"How did we arrive at our figures? First, we gathered prescription data from the online database managed by the NHS Information Centre.

This quarterly information was compiled to get annual numbers covering 1 April 2009 to 31 March 2010 – the most recent full year with available data. In order to make the numbers comparable, we then linked the raw prescription numbers to the ONS mid-year population estimates.

This allowed us to calculate the prescriptions per 100,000 figure in the data below, which controls for the different sizes of PCTs, if not their different levels of wealth, employment and general illness."

The data is also not age standardised. In the comments section some have suggested that in areas with low numbers of prescriptions doctors may be prescribing several months prescriptions at one time. I have looked at the most recent data set available (July-August 2010). Data is available on the total cost of the prescriptions as well as the number of prescriptions and these are plotted below.


This shows a clear correlation between number of prescriptions and cost. However when one looks at individual PCTs the average cost of an antidepressant prescription in Kensington and Chelsea PCT is £7.01, whilst in Blackpool PCT it is £3.48. This does suggest that either more expensive antidepressants are being prescribed in Kensington and Chelsea, or more months prescriptions are being given at one time. Since the absolute number of prescriptions in Kensington and Chelsea is so much lower than in Blackpool, it may be that more antidepressants are being prescribed in each prescription. 


When looking at datasets it's good to make use of all that is available.


You can find my spreadsheet here.


EDIT 4.20pm 6/3/11 I've calculated a rate of prescribing of antidepressants per 1000 of population over 19. (This is not ideal as some teenagers may be prescribed antidepressants, but more accurate than using the total population including children). This has been plotted against average cost of antidepressant prescription. This shows that Blackpool and Kensington and Chelsea are outliers. 

There is not a strong relationship between cost of prescriptions and number of prescriptions. This may explain some of the variation between north and south of England, but the long-established relationships between deprivation and depression are likely to have greater explanatory power.


Here is a link to some papers on that relationship and a very interesting report (via @coxar) on the relationship between antidepressant prescribing, poisoning by antidepressants and deprivation.

If something is shared do you value it less?

Image: Sharing the juice by zummersweet

If you listened only to those I know online, who are interested in learning, you would think that the world believed in trying to make and share educational content as easily as possible. But that is not always true.  I have participated in discussions where it has been suggested that students might think that a course has less value if the content is freely available. The usual response is that  a student should sense more value in participating in a course,  than that which can be derived from easily shared online content. There should be more to it than a few lectures.

This morning I saw a tweet about a lecture on health inequities in the London School of Hygiene and Tropical Medicine. My immediate thought, which I tweeted back, was that it was a pity that the lecture wasn't more widely available (recorded, shared and able to be embedded in a blog) since the topic is so important. The things is that I've started getting a little spoiled. Last week I was able to watch sessions from the Foundation for Informed Medical Decision Making (FIMDM) conference in Washington DC here, live as they were being streamed, and for free. Last year, I watched many of the sessions in a 2 day conference on "Innovation in the Age of Reform" and you can too now, because those sessions are still available here. That conference was organised by Swedish . I had never heard of them before but I have now and so have you.

Lecture capture is becoming so easy to do that most universities have started already. And if you have the lecture then why not share it? Many are. iTunesU is one way of  sharing your content freely with others. Cambridge, Oxford, Harvard and Yale are some of the universities already sharing with iTunesU. MIT have a whole website devoted to sharing 2000 of their courses. So all of these institutions have decided that there is something to be gained from making some of their content freely available. Maybe they believe that it is morally right to share, or that is will garner them positive coverage, or attract future students.

Anyway, the doctor I was speaking to thought that it wouldn't be appropriate to share material from a post-graduate course that someone was paying for. I asked if she thought it would devalue the course and she said yes. So, away from the echo chamber that I seem to inhabit more often I wanted to ask how you would feel about paying to attend a course where some of the materials were made freely available to others. Would it put you off? Would you feel proud that others could see the high standard of teaching you were receiving? Would you feel glad that your fees were helping to share knowledge around the world? Or would you feel cheated?