If you want to be trusted more: claim less

January 9th, 2010 by Ben Goldacre in bad science, politics, spin, statistics | 55 Comments »

Ben Goldacre, The Guardian, Saturday 8 January 2009

“Public sector pay races ahead in a recession” shouted the front page of this week’s Sunday Times. “Public sector workers earn 7% more on average than their peers in the private sector — a pay gulf that has more than doubled since the recession began.” The Telegraph followed up with a copycat story a few hours later.

In reality, this is one of those interesting areas where anybody who makes a firm statement is wrong, because there is not sufficient evidence to make a confident assertion in either direction.

The Sunday Times have identified a difference in the median pay of all public sector employees in the country, when compared with all the private sector employees in the country, and overextrapolated from there to claim that – job for job –public sector employees are paid more than their peers in the private sector.

We will discuss whether that figure is worse than useless in a moment.

But first, some interesting details. For their analysis the Times use “annual salary” instead of “hourly pay”, although the latter is clearly more meaningful, especially since the Times quote the annual salary figures for part-time and full-time employees, all mixed together, but 31% of public sector jobs are part-time, against 23% of private sector jobs. In fact, quoting “hourly salary” would also have made the difference between the public and private sector median wages look even bigger. So why did the Times and the Telegraph use annual pay?

image Perhaps because this figure makes the difference in medians look like a new phenomenon under the present government. Using the hourly figures, you can see that public sector median pay has been higher than private sector hourly pay for years. If you go to the “Annual Survey of Hours and Earnings” data on the ONS website which the Times used, you can see for yourself. It was £7.98 vs £6.72 in 1997 under the previous government, a difference of almost 20%, and £8.56 vs £7.32 in 1999. Meanwhile the “annual salary” difference which the Times chose to use was negligible in 1999 (the first year ONS gave this figure), at £15,002 vs £14,963, a difference of 0.3%, allowing them to create this illusion of a brand new phenomenon (for example, see their graphic, here).

More than that, using the “annual salary” figure allows the Times to claim dramatically that the difference has doubled in 2 years: the difference in medians for annual pay has gone from 3.8% to 6.8% since 2007, while the difference in hourly pay has gone from 25.1% to 28.7%, which is much less eye-catching.

“By a whole range of measures,” the Times continues: “public sector employees are also enjoying better working conditions. Last year the average public sector worker laboured for 35 hours a week… 2 hours less than the typical private sector worker.”

Is this really down to laziness, and better working conditions? No. Again, this is simply due to the greater number of part time jobs in the public sector – 31% vs 23% – which is a longstanding phenomenon.

But there is a deeper problem with the analysis in the Sunday Times and the Telegraph. The longstanding difference in median wage for all jobs in each sector is hardly informative on the question of whether someone is paid more or less than their peer in the other sector. Firstly, it’s hard to decide what the comparison job is for a policeman, a fireman, a teacher, and so on.

Secondly, to make that comparison between medians meaningful you’d need data showing the breakdown of what kinds of jobs are done in each sector. Because it’s possible, after all, that the state employs more people in more senior or middling roles, and fewer people in the kinds of jobs you find at the absolute bottom of the employment ladder.

If you like, for an illustration, we can poke around the ONS ASHE data again. The national median hourly wage is £11.03. If you take table 14_5a of the ASHE 2009 data, re-order it by wage, and look at the bottom 3 categories with over a million people in them, as a rough illustration, we have: 1,126,000 sales and retail assistants on a median hourly wage of £6.36; 1,355,000 cashiers at £6.40; 1,430,000 in sales at £6.45.

None of these are jobs you find in the public sector, although there are also cleaners at the low wage end of this table. If someone here was quoting data comparing public/private wages for the same kind of cleaning jobs, say, then that would be interesting. There’s no such data on offer. But as the Times says: “our reports today show, the public sector has become so big and such a generous employer that it is sucking workers out of private companies”. I don’t see how they can justify this, other than with their laughable case studies, and if it’s true, it should be an longstanding trend, not a new one.

I could go on. It’s not surprising if public sector pay increased from what it used to be, under this government: improving recruitment for teachers and the like was a manifesto promise. But as for a comparison, I don’t know if the public sector pays more than the private sector for the same work, or less: nobody does, from a difference in median wages. Meanwhile I do know that this was one of the most statistically misleading front page stories I have seen in a long time. It’s going to be a fun election.

If you like what I do, and you want me to do more, you can: buy my books Bad Science and Bad Pharma, give them to your friends, put them on your reading list, employ me to do a talk, or tweet this article to your friends. Thanks! ++++++++++++++++++++++++++++++++++++++++++

55 Responses

  1. Deasun said,

    January 19, 2010 at 12:29 pm

    And if you want to see some classic pseudo-scientific right-wing propaganda masquerading as a serious study, try the following:

    This relates to the Republic of Ireland, but I’m sure that there will be something similar for the UK. As always, skip to the methodology section. You may notice, for example, that the ‘paper’ compares only remuneration in the months of March 2003 and 2006 so, presumably, bonuses are not included! There are also some odd assumption made but not explained such as corrections for demographics. Naturally, the study concludes that the public sector is the land of milk and honey. Funny that!

    Anyway, why not compare jobs at various levels which do exist in both the sectors (e.g. lawyer, IT worker, press officer…). Surely, that would be more meaningful. Let me know what you think.

  2. Heuristic said,

    February 2, 2010 at 9:55 am

    There has been a lot of research conducted into pay variation, in particular “pay gaps” between men’s and women’s pay, between different industries, etc.

    The main issue, as has been pointed out above, is the number of important covariates that need to be considered when comparing, say, public sector and private sector pay.

    As has been addressed, the public sector employs a relatively large proportion of graduates (in a number of countries, a large volume of low paid men’s work – for example – was removed from the public service during 1980s public service structural reforms). Also, in some countries, public servants tend to be older on average, which is another good explanation for higher pay – older workers tend to be paid more than younger ones for social capital reasons.

    The main complexity of talking about private sector versus public sector pay is that often the comparisons are between apples and oranges with no attempt to control for differences between these two employer types. Good research on pay (which goes back to the 1960s) has tended to try to control for industry, and there have been some attempts to control for occupation. There are standardised classification of industry and occupation codes that can be used, to assist this process.

    Another key explanatory factor is job size, which is used by many employers to set the pay banding for a job. The translation of job size into pay is moderated by industry and occupation, so that jobs of similar size can pay quite differently depending on the particular role/scarcity of applicants for the particular occupation.

    Finally, employers actively check their pay scales against “market” remuneration information for occupations, and different employers pitch their pay scales to different levels of the market remuneration. For example, some public sector organisations peg their remuneration at around the 50% mark. Clearly, employer decisions around where to set their remuneration rates are going to directly affect employee pay.

    Any analysis of pay gaps that does not take account of these differences is simplistic and “bad statistics” let alone bad science.

    If anyone is interested, there is a large volume of peer reviewed research on pay gap analysis, in particular the Blinder-Oaxaca decomposition (multiple regression technique) has been used.

    cheers all.

  3. orangelavaglow said,

    February 20, 2010 at 10:10 pm

    As the story is also about economics and politics, why does it matter so much that there are separate ‘public’ and ‘private’ sectors?

    How does the mixing of public finance and private finance (such as introducing private finance initiatives to the NHS, or managemant consultancies to every aspect of the public sector)(or publically financing the entire banking sector) affect the legitamacy of any of these comparisons?

    Why do economists and ‘market analysts’ decide one sector is legitimate and the other not?

    but maybe the ‘science’ is a good place to start…

  4. qingimiss said,

    February 23, 2010 at 3:28 am

    Other wedding rituals such wedding invitations as the Highland

  5. kimekime said,

    April 1, 2010 at 5:53 am

    cheap wedding invitations
    cheap prom dresses
    cheap bridesmaid dresses
    lace wigs