“Exams are getting easier”

August 21st, 2010 by Ben Goldacre in bad science, evidence, numerical context, politics, schools | 123 Comments »

Ben Goldacre, The Guardian, Saturday 21 August 2010

Pass rates are at 98%. A quarter of grades are higher than an A. This week every newspaper in the country was filled with people asserting that exams are definitely getting easier, and then other people asserting that exams are definitely not getting easier. The question for me is always simple: how do you know?

Firstly, the idea of kids getting cleverer is not ludicrous. The Flynn Effect is a term coined to describe the gradual improvement in IQ scores. This has been an important problem for IQ researchers, since IQ tests are peer referenced: that is, your performance is compared against everyone else, and the scores are rejigged so that the average IQ is always 100. Because of the trend to greater scores, year on year, you have to be careful not to use older tests on current populations, or their scores come out spuriously high, by the standards of the weaker average population of the past. Regardless of what you think about IQ tests, the tasks in them are at least relatively consistent. That said, there’s also some evidence that the Flynn effect has slowed in developed countries more recently.

But ideally, we want to address the exams directly. One research approach would be to measure current kids’ performance on the exams of the past. This is what the Royal Society of Chemistry did in their report “The Five Decade Challenge” in 2008, running the project as a competition for 16 year olds, which netted them 1,300 self-selecting higher-ability kids. They sat tests taken from the numerical and analytical components of O-level and GCSE exams over the past half century, and performance against each decade rose over time: the average score for the 1960s questions was 15%, rising to 35% for the current exams (though with a giant leap around the introduction of GCSEs, after which score remained fairly stable).

There are often many possible explanations for a finding. Their result could mean that exams have got easier, but it could be that syllabuses have changed, and so modern kids are less prepared for old style questions. When the researchers delved into specific questions, they do say they found some things that were removed from the GCSE syllabus because they’d moved up to A level, but that’s drifting – unwelcomely – into anecdote.

Another approach would be to compare performance on a consistent test, over the years, against performance on A levels. Robert Coe at Durham University produced a study of just this for the Office of National Statistics in 2007. Every year since 1988 they’ve given a few thousand children the Test of Developed Abilities, a consistent test (with a blip in 2002) of general maths and verbal reasoning skills. Scores on this saw a modest decline over the 1990s, and have been fairly flat for the past decade. But the clever thing is what they did next: they worked out the A level scores for children, accounting for their TDA scores, and found that children with the same TDA score were getting higher and higher exam results. From 1988 to 2006, for the same TDA score, levels rose by an average of 2 grades in each subject.

It could be that exams are easier. It could be that teaching and learning have improved, or teaching is more exam focused, so kids at the same TDA level do better in A levels: this is hard to measure. It could be that TDA scores are as irrelevant as shoe size, so the finding is spurious.

Alternatively, it could be that exams are different, and so easier with respect to verbal reasoning and maths, but harder with respect to something else: this, again, is hard to quantify. If the content and goals of your exams change, then that poses difficulties for measuring their consistency over time, and it might be something to declare loudly (or consult employers and the public over).

Our last study thinks more along those lines: some people do have clear goals from education, and they can measure students against this yardstick over time. “Measuring the Mathematics Problem” is a report done for the Engineering Council and other august bodies in 2000, analysing data from 60 departments of maths, physics and engineering who gave diagnostic tests on basic maths skills to their new undergraduates each year. They found strong evidence of a steady decline in scores on these tests, over the preceeing decade, among students accepted onto degree courses where they would naturally need good maths.

Sadly they didn’t control for A level grade, so we can’t be sure how far they were comparing like with like, but there are a few explanations. Maybe maths syllabuses changed and were less useful for maths and engineering degrees. Maybe the clever kids were doing English to become lawyers instead. Or maybe exams got easier.

If you know of more research, I’d be interested to see it, but the main thing that strikes me here is the paucity of work in the field. There’s a man called Rupert Sheldrake who believes that pets are psychic: they know when their owners are coming home, that sort of thing. Obviously we disagree on a lot, but we chat, and are friendly, and once when we were talking he came out with an excellent suggestion: maybe 1%, maybe 0.01% of the total UK research budget could be given to the public, so that they could decide what their research obsessions were. Obviously most of the money would get spent on psychic pets, and which vegetables cure cancer, but I’d like to think that some of it, possibly, might get spent on good quality robust research to find out whether exams are getting easier.


++++++++++++++++++++++++++++++++++++++++++
If you like what I do, and you want me to do more, you can: buy my books Bad Science and Bad Pharma, give them to your friends, put them on your reading list, employ me to do a talk, or tweet this article to your friends. Thanks! ++++++++++++++++++++++++++++++++++++++++++

123 Responses



  1. smowton said,

    August 21, 2010 at 1:13 am

    Odds on yes they are, but for other reasons: a lot of people are politically motivated to make exam grades increase (it makes a nice statistic); practically nobody is motivated to make them go down.

    Thankfully this doesn’t matter, as, as you point out, universities (and employers) already know the true meaning of Christmas^H^H^H^H^H A-level grades, and correct appropriately.

  2. rptb1 said,

    August 21, 2010 at 1:36 am

    As an employer, I want exams to discriminate between people. If 25% get an A then they’re next to useless. I was under the impression exams used to be moderated so that roughly the same proportion of As occurred each year. Was that never really true? If it was, how did the system break down? I’d be very happy for it to be shown that people are getting smarter, but it would still be more important to know who is in the top 5% among them.

  3. ellieban said,

    August 21, 2010 at 2:21 am

    This is, of course, anecdote but when I started my degree (my fairly middle of the road biology degree at a middle of the road uni, so no super high achieving course to keep up with)we were given a “basic” maths skills test because the university were concerned we didn’t know enough of the basics to do the course. It was a train wreck.

    I also have a German GCSE at grade C – I sat it in 1996 so it is now old but I gained a C grade just by answering as many questions as I could in English with a German spelling. That is a very worrying indictment on our languages training.

    It would take a lot of evidence of a very high standard to override my personal experience and convince me that the improved grades can be accounted for by something such as improved teaching.

  4. EvanHarris said,

    August 21, 2010 at 2:55 am

    Based on this review, the evidence – such as it – suggests that “exams are getting easier” in other words there has been grade inflation.

    Does this matter? Yes it does – for several reasons

    1) It allows politicians to disguise a failure to improve teaching/performance, and more importantly to fail to grasp the need to improve these.
    2) It allows politicians to claim credit for what turns out not to be progress
    3) It makes it difficult for employers to judge qualifications
    4) It means that some courses at some Universities need to do more “remedial” work with students before the degree syllabus is tackled.

    One further potential cause of grade inflation, not covered by Ben, is that there is a MARKET in exam papers. I have never understood whether any gain from having different exam boards (providers) compete to sell exam papers to schools is worth the risk of schools skill-fully shopping around for the papers that more students will pass or pass with higher grades to improve the school (groan) league-table position.

    A second potential cause of overall grade inflation is the growth in the range and choice of A levels so that students can pick out subjects which are easier to pass (eg Business Studies vs Economics) or which they have an aptitude for (which is not such a bad thing – even an increase in language exams being taken by students where that is their first language)

    This area needs more research and then tackling. After all Universities are finally partly dealing with degree result inflation.

    While papers may, as Ben asserts in line 2, be full of people saying exams are getting easier (no doubt squeezed in alongside pictures of attractive girls celebrating), none of them are politicians. It is remarkable that even opposition politicians are reluctant to call this out for fear of being accused of dismissing student/teacher achievement, and that parties – who at least cough and look embarrassed about the onward march when in opposition – suddenly become really defensive when in Government.

    There is no political incentive in the short term to recalibrate because of fear that opposition politicians or papers will misrepresent re-calibration as real sudden decline. But it should be done – perhaps after a commission achieves consensus on that.

  5. 6821isnotmyPIN said,

    August 21, 2010 at 3:17 am

    The last 2 years I have been studying for my A-levels, and I didn’t even realise they counted for anything outside of getting to university.
    Many of my teachers have actually said that they feel like they aren’t teaching the subject, just teaching the exam.
    I’m beginning to get the impression that degrees might just become the new GCSE’s for the future.

  6. Jeremy Nel said,

    August 21, 2010 at 5:33 am

    I think comment #2 is particularly shrewd. From a certain perspective, it simply doesn’t matter whether exams are getting easier. What they should do is discriminate between the top, middle and bottom students effectively.

  7. dredjah said,

    August 21, 2010 at 5:46 am

    I think Dr Michael de Podesta (physicist and Science Ambassador at the UK’s National Physical Laboratory) has a point when he says :

    ‘….publisher/awarding body conglomerates submit schemes of work which minimally satisfy QCDA guidelines. Despite the input of many talented and creative individuals in schools and awarding bodies alike, this structure is guaranteed to lower standards. It is a race to the bottom: the awarding body which can produce the minimum specification with the easiest examinations will win ‘market share’ from their competitors and exam pass rates will rise.’

  8. clt47 said,

    August 21, 2010 at 7:23 am

    There was an interesting discussion on the writing and marking of GCSEs Word of Mouth on Radio 4 last week, which probably applies to A-levels as well. (Disclosure: youngest child sat A-levels 3 years ago, now going on to an MA.) The idea, according to a senior examiner from Edexel, was to make the exams more standardised, to eliminate subjective responses by the markers, and so make them more reliable. The effect has actually been to reduce confidence in the exams, partly because so many essay questions have effectively been reduced to learning formulaic answers.

    Others have suggested that as the number of candidates has risen, the skill level for marking has declined and that, again, formulaic answers are preferred because these are easier to spot and mark.

    When I took the US College Boards and GRE exams many years ago, you got two scores for the different sections of the exams: a raw numerical score and a series of percentile scores (all candidates, candidates going on to university in previous years, and so on.) As far as I know, this is still the case. Including either of these would help.

    Another question is whether any of these exams are actually good predictors of academic or employment success, in the absence of a control group.

  9. Kertz said,

    August 21, 2010 at 7:30 am

    All of this would be moot if the government made exam grades norm referenced. This way the top 5% or so would get the best grade, the next 10% the next grade and so on to the bottom 5% getting the lowest grade. This would allow everyone to differentiate the best students from the worst. However it would never show any improvement from all the education spending and so obviously can’t be allowed.

  10. juliang said,

    August 21, 2010 at 7:37 am

    Either exams are getting easier or students are getting better at them. Whichever the case, we are left with a top grade which does not distinguish sufficiently the brightest students. If students were given a mark percentile as well as a grade for each exam, we would have a system where those needing to use the results could decide for themselves. The top universities might use percentiles and employers might use grades.

  11. muso-tim said,

    August 21, 2010 at 8:25 am

    To comment #2 yes there used to be a proportional system until the 1980s, when it was removed. I suppose the idea is that a big subject like English will have a statistically similar group of students every year but a minority subject like Electronics which might only get 900 entries may vary widely. Anyway, ever since then the pass rate has increased every year. Spooky.

    As a teacher, I have heard every reason under the sun. I’m a musician and don’t need to worry about evidence :) so I can’t contribute much. All I will suggest is that questions are easier to mark (1 and 2 mark answers), specifications are lengthy documents (used to be one side of A5), examiners often training for teachers (only examiners knew how to get the best grades) and teachers have a lot of pressure on them to get ever higher grades. Make of that what you will.

  12. Boilingtube said,

    August 21, 2010 at 8:43 am

    There are two other factors to consider in this minefield. Students are examined on more subjects at AS level (your old lower 6th or Y12) and then drop one or two which they feel they have not done well in. So at A2 level (your old upper 6th or Y13!), they are studying subjects in which they are likely to succeed. this immediately skews the marks upwards.
    Secondly, in the lower 6th, they take module examinations so that if they fail a section of the subject they can retake it.
    Now for the anecdotal bit: there are some topics in chemistry (my subject) which I taught in the 1970s and 1980s which are not in the syllabus now, eg, conductivity of solutions. And there are topics which are not taught in the detail that we did, eg reaction kinetics has lost much of the mathematical base, ie calculus, organic reaction mechanisms and solubility products in equilibrium.
    There have been a few additions but not many. I started teaching free and entropy at the end of the 80s.
    Comparison of A levels is now to A levels in the 1960s, 70s and 80s is probably a waste of money as there are just too many variables to cope with.
    More important is putting money into having teachers teaching their specialism with great enthusiasm for their subject. On the Times Educational Supplement science forum is a plea for extra training from a newly Qualified Teacher who specialises in Physics having to teach A-level Chemistry. That training does exist at our Science Learning Centres. I hope the school allows the teacher to attend! But it should not happen in the first place!

  13. oldandrew said,

    August 21, 2010 at 8:49 am

    Education isn’t medicine. Anecdotal stories about medicines are unreliable. Studying two exam papers to see which is easier is probably quite reliable as a method, except where people have obvious agendas or they are very, very similar papers. The only problem is how willingly you accept “they are testing different things” as an excuse.

  14. Ian said,

    August 21, 2010 at 8:57 am

    I keep seeing statistics showing that the percentage of candidates getting an A has risen and the percentage of candidates who sit the exam and pass has risen. But most schools/teachers don’t enter students for an exam they’re unlikely to pass.

    I teach biology in Scotland and I was very pleased this year as my 4th year (the equivalent of GCSE students) achieved a 100% passrate. They do every year, because they all sit exams at different levels (Intermediate 1 or intermediate 2. Many schools use Standard grades at Credit, General or foundation level).

    Norm referenced would be great – but not as a percentage of students sitting the exam. So an A would have to be 10% (or whatever) of that year’s student population, not 10% of those sitting at that level.

  15. YJ99 said,

    August 21, 2010 at 8:59 am

    My son got his A-level results on Thursday. I got mine 33 years earlier. In terms of the percentage of entries gaining the top grades his results roughly equate to mine.(A*A*A in 2010, AAB in 1977). I do not feel it was any easier for him to get those grades. The difference seems to be that savvy pupils and teachers know how to obtain them and we didn’t. When I sat exams I had no concept of such things as a ‘marking plan’. Today pupils are told exactly how many marks they can get for each part of their answer. To obtain the highest grades they must ensure that as many of these as possible are ticked. The corollary of this is that there is no point in showing any real flair or putting forward an alternative view; no extra marks will be awarded. In my day, we tried to provide an interesting and informed answer and hope the marker liked it. A-levels today are no easier but top grades can be obtained without a candidate ever having to show even an ounce of initiative or original thought which does not bode well for some when it comes to going on to further academic study.

  16. maddy2010 said,

    August 21, 2010 at 9:35 am

    A little bit tangential to this debate. I was recently watching the YouTube “Einstein for the Masses” by Prof Shankar at Yale. It struck me that most people with a GCSE equivalent education and 45 minutes to spare would probably grasp the basics of Special and General Theories of Relativity after watching this. Yet when it was published only about 100 or so highly trained professional physicists would have been able to get their heads around it.

    Perhaps there is a parallel to exams – I doubt the material they examine has changed enormously over the last 50 years.

  17. giessen said,

    August 21, 2010 at 9:35 am

    You have relatively little to worry about. In the Netherlands, exams got easier without any rise in grades. Imagine that!

  18. bagpuss7 said,

    August 21, 2010 at 9:54 am

    I’ve three children at a “Specialist Technical College” studying for what are GCSE’s (and then onto A Level). The quality of the worksheets they bring home in general is poor. They consistently have basic English errors and frequently present poorly worded or misleading questions and the scientific rigour of the questions is “open to debate”. It’s extraordinary. The staff I’ve met mirror this with the IT teachers unaware of the next year’s syllabus and unable to explain the grading scheme, the science teachers also unable to really explain the grading scheme, also admitting that it “was a numbers game” and the Electronics teacher having less electronics knowledge than I did pre-degree. I’m truly appalled.

    The English and Maths and Music teachers were however sharp and focussed, which is a relief. BUT 3/11 is not good enough though. If I had staff meeting their customers that ill prepared I’d put their botties in a sling. Likewise if they prepared reports and documents to these standards I’d be pretty cross. I don’t know for sure about exam standards (although I’m inclined to believe something is amiss), but the rigour and standards earlier in the academic program leave me unimpressed and searching for a 6th form college with better standards and looking for tutors in the interim.

  19. David Mingay said,

    August 21, 2010 at 9:57 am

    One of the roots of the problem may be this: Just before this year’s results were released, one of my teacher ex-colleagues (I’ve given up teaching, since the new A level specifications have made psychology boring and embarrassing to teach) posted a Facebook status which read “I’m bricking it – my results are out tomorrow”. The “my” is the salient part. I’ve always thought of them as my students’ results, but teachers have been made to feel increasingly over recent years that they are entirely responsible for their students’ performance, and so naturally they want exams to be as easy as possible. At an exam board workshop on the latest round of dumbing-down, when it was announced that about a third of the specification was being removed along with assessed experimental work, two teachers next to me said, “Great! More A grades!”

    On the subject of writing answers which show some flair, two anecdotes: At a meeting hosted by the chief examiner a couple of years ago, while he was going through the new detailed marking scheme, he said that in the old days he and his colleagues gave A grades for answers “just if they thought they were interesting”, and then laughed. “What’s wrong with that?” I asked. A tumbleweed blew across the room, a distant train whistle sounded, and we went back to poring over the intricacies of the mark scheme. At another meeting this year between HE representatives and the A level exam boards, one board officer started by apologising that the new specifications “weren’t education; they’re just box-ticking”. “You’re the exam board – why don’t you do something about it?” I asked. “The government makes us,” she replied.

    I don’t think norm referencing is the answer. The results should be criterion referenced – it’s just that the criteria should be a lot higher.

  20. scarynige said,

    August 21, 2010 at 10:03 am

    There’s a possible confounding variable that should be taken into account when looking at A-level results, i.e. that schools are now under pressure to get better results too and one of the ways they do that is by being more selective over which pupils can take which exams. One would expect this to improve results over time without saying anything about how the exam itself changes in an objective way.

  21. osipacmeist said,

    August 21, 2010 at 10:06 am

    It is interesting and more than or little scary that good research on this would seem to be so scarce. For as long as I can remember it has been a given of our politics that standards should be improved, and no one seems to have asked “how shall we know”. In this as in many things our public discourse becomes just a lot of silly shouting, usually reflecting various prejudices. It’s all very depressing.

  22. RussPitcher said,

    August 21, 2010 at 10:13 am

    @Kertz & rptb1
    Whilst I can see the reasoning behind your desires for normalised grading (as my O-levels were marked), where only the top x% get and A-Grade, there would seem to be an obvious problem with this. As an example, if 2010 graduates perform unusually well and 2009 graduates perform unusually badly, you will end up with some less able 2009 graduates with A-grades, and some more able 2010 graduates with B-grades.

    Surely as an employer, or other party concerned with ability, you need to be able to judge all candidates reasonably equitably, rather than only those from within the same year of graduation?

    Having said all that, if 25% of the graduates get the same top-level grade, then the grades need to be re-assesed or new sub-divisions need to be applied.

  23. bagpuss7 said,

    August 21, 2010 at 10:15 am

    As scarynige says, My children’s school also push pupils in the direction that suits them. I’ve had to battle to keep my daughter on more rigorous subjects which she is capable of, when the school would prefer easier subjects of less use (in my opinion) in the real world. The worst example of this was that on learning that she is currently working towards Grade 3 piano and Grade 4 flute they pushed far too hard for a music GCSE instead of a language, an easy high grade in music suiting the school. However a grade 6 is generally considered GCSE equivalent, so it made no sense for my daughter to do a GCSE as well…

    (no marks to anyone who senses the possibility of a bee in my bonnet on this subject)

  24. Spudtater said,

    August 21, 2010 at 10:43 am

    @RussPitcher: For the more common subjects, you’d expect a large enough population that the difference between mean scores from year to year will be vanishingly small. As muso-tim points out, normalised grading only falls down in rarer subjects.

    Which raises the question: do we really need to uniformly normalise every subject, or can we just do the big ones?

  25. zeno said,

    August 21, 2010 at 10:50 am

    In Scotland, where I went to school, there is one set of exams for everyone. What I don’t understand is why, in England, there are different exam boards – although there are significantly fewer than there used to be. What benefit do they bring? Do they ‘compete’ against each other – with that competition possibly affecting the difficulty of the course and exams?

  26. depjac said,

    August 21, 2010 at 10:53 am

    The rise in the overall pass rate is easily explained and not really part of the possible change in standards debate. When students sit AS level, any student who fails this, or gets a grade E will be advised not to continue with the subject as, other than in exceptional circumstances, they stand little chance of getting a high enough grade on the harder A2 modules to achieve an overall A level pass. Therefore most of these students will move onto different courses, more suited to their ability. These students do not appear in the final total. Anybody who fails an A level has either received bad advice at the end of year 12 or refused to accept it.

  27. FelixO said,

    August 21, 2010 at 11:27 am

    @Depjec #20 +1 Insightful

    It could of course be true that:
    * Exams are getting easier
    * Pupils are getting brighter
    * Schools are better at teaching
    * Schools are more selective over which pupils can take which exams
    * Schools push pupils to do easier subjects in order to bolster the schools statistics
    * Exam boards are competing in a market where being easier pays.
    * Only better students actually complete the full A-level

    AFAICT all of these issues would be avoided by basing grades on percentiles.

    As for @RussPitcher who suggests that an entire year of students could be dumb enough to render the system unfair – that’s just the opposite of what we have now, exams could getter harder in one year so that everybody scores a lower mark but the percentile system still tells us who the best students are.

  28. FelixO said,

    August 21, 2010 at 11:28 am

    In fact, it would be fab because it would allow the curriculum to respond to the needs of employers or universities without affecting the grades.

  29. MrSimonWood said,

    August 21, 2010 at 12:29 pm

    @SpudTater The point made by RussPitcher in #22 is important if we are getting a significant variation in cohorts even over a longer time, eg. more students *are* performing better (and is norm-referencing the best solution if we’re not?). That’s assuming an employer might want to compare the A-levels of students who had achieved their qualifications several years apart.

    The suggestion made by juliang is #10 is genius. We should be making available as much information as can be usefully used.

  30. Hilary said,

    August 21, 2010 at 12:46 pm

    I am sure that it is more the marking than the exam itself (based on my own and husbans’s teaching experience), especially after course assessment arrived and more people’s marking had to be standardised. Check-boxes may produce a high mark from a garbled answer that nonetheless happens to include the criterion words for instance. Formula examples of what ‘counts’ as an acceptable answer got more flexible – it’s really hard to define what counts as ‘understanding’ without relying on subjective teacher reaction (& we cant’t have that!).

    I often think of the old joke: middleaged person revisits university & asks to see the finals papers. “Good grief these are the same questions I had!” “Ah, but the answers have changed.”

  31. hleith said,

    August 21, 2010 at 1:11 pm

    This whole debate about whether exams are getting easier or not stems, I think, from the inbuilt inequality and sheer utter snobbery of the English education system. I am Scottish (and like Zeno am used to only one exam board) and I am a teacher working in Kent. I cannot believe that such an archaic system still survives. People in Kent are quite happy for their children to mix at primary school but that is where it ends – then begins the cruel and humiliating 11 Plus experience with no guarantee of a Grammar School place even with a pass. High School, Grammar Schools, Faith Schools and Private Schools; all with their own ‘reputations’ on whether they are ‘good’ or ‘bad’ schools and with their own admission policies. Then of course you have to choose your subjects but for goodness sake don’t choose ‘soft’ subjects that you like and might actually do well in because they might not be seen as good enough. Also, were they from the right examination board because we all know that some boards are ‘better’ than others… And so it goes on into Universities.

    Are exams getting easier? Who knows, but what is important is the outcome. I have heard anecdotal evidence that Universities and employers complain about the standards (usually about literacy) of school leavers but where is the research for this? Do we have a shortage of intelligent, skilled, flexible citizens?

    Unless England sees education in a completely new light we will be stuck with a system that does not get the best out of every child in this country. If Gove has anything to do with it that inequality will become even worse. We need a fair and open education system but I do not see that happening anytime soon.

  32. J. Wibble said,

    August 21, 2010 at 1:13 pm

    @bagpuss7: Grade 6 is definitely not GCSE equivalent. Grade 6 is the ‘standard’ difficulty level for performance at A2 (as set by the exam board) and the theoretical knowledge required for A2 is also about Grade 6 (based on personal experience and asking friends).

    Re the original question: The expert group report used to allocate UCAS points to music exam grades, published in 2002, explicitly states that the standard of performance required has been lowered.

    “Before the implementation of Curriculum 2000 the standard of performance required at GCE A-level was Grade 8 and/or Grade 6, depending on the syllabus offered. In order to make the GCE Music more accessible to more students, the standard difficulty level at AS is now Grade 5, and at A2 Grade 6 (although many candidates perform to higher levels in both years).” (p.25)

    The QCA also published a paper entitled “Review of standards in A level and GCSE music: 1985-2005″ (QCA, 2007). They concluded that the syllabus had been extended, mostly to include areas of study outside the Western classical tradition, but at the expense of depth of study. At A level,

    “They considered that, over the review period, the demand of history and analysis papers had reduced. In some cases they noted that AS papers had more in common with earlier O level and GCSE papers, particularly with regard to style of questioning and the greater emphasis on breadth rather than on depth and extended writing. In the case of aural perception and musical techniques, there was general agreement that traditional skills were no longer being tested at such high levels as in the past or, in the case of harmony and counterpoint, at all.” (p.70)

    Obviously this is only one subject and a very limited body of evidence, but what is most interesting is that the number of A-B grades in A level music has actually decreased over the past decade. There could be a number of reasons why this is the case; my personal hypothesis is that the lowering of the required standard has meant that more pupils are sitting the exam who might previously have not even considered doing so, and thus higher grade percentages are decreasing because the same, relatively small core number of advanced-level music students are sitting the exam, and the rest are not aiming for As and Bs. This could also explain why the proportion of As (and now A*s) for Further Maths is ridiculously high, as only students who are looking to get the top grades would even consider taking it.

  33. sciencerocks said,

    August 21, 2010 at 1:20 pm

    Nice article Ben.

    I can’t talk from the point of A Level teaching; nor am I familiar with the relevant educational evidence on issue. I therefore can only offer views and experiences of a lecturer at University who was witnessed (recently) what happens when governments get involved with setting curricula and frameworks for educational programmes at BSc and MSc level. Perhaps these are relevant to political interference in A Levels too.

    My subjective impression of the methodology used when government leads development of frameworks for University programmes is something like this: start with the conclusion and seek evidence to support it; seek out “yes” people and other people who will tell you what you want to hear for advisers, whilst ignoring those who might be critical (ignoring at best – but don’t rule denigrating if necessary although with plausible deny-ability of course); when justifying a decision, favour who the advisers are (e.g. “Professor Dame so-and-so thinks…”) rather than the merit’s of the advisers’ arguments; favour personal experiences of education over high-quality research evidence; be highly self-congratulatory at every possibly opportunity; consult because it looks good but announce action plans before publishing the responses to consultation; when reporting responses to consultation or working groups, cherry-pick views and evidence that fits with pre-determined decisions; award OBEs and CBEs to the advisers that have stayed ‘on message’… You know, all the stuff that proper educationalists and scientists attempt to avoid. Basic reasoning errors. Anyone remember Modernising Medical Careers? Take a look into Modernising Scientific Careers too, as a recent example. There are other examples. I therefore see Evan’s points (#4) about political motivations as being particularly pertinent.

    Returning to A Levels, I am tempted to believe that we know a lot more about education and learning these days and that at least some of this has filtered into the class room and into exam papers. One of the key exceptions to this is that I suspect (from my work with undergraduates) that there is still too strong an emphasis in the “filling the vessel” analogy to education in A Levels (but also, to be fair, at University to an extent too). Again, my experience of government-led frameworks is that they love the idea of “knowing” as opposed to development of higher-level skills (e.g. reasoning skills, critical analysis, reflection, communication). For example, most first year undergrads in science degrees (and occasion high flying A Level students I sometimes work with) I have experienced over last 10 years who have done science A Levels (sometimes three science A Levels) generally have little idea and experience of how to write a scientific report e.g. of an experiment. Some barely know how to write, or communication more generally, their ideas critically. I think we should be worried about this.

    I wonder how long it will be until A** will be available as an A Level grade.

  34. Bishop Gillian Wakefield said,

    August 21, 2010 at 1:37 pm

    What would happen if, say, next year’s students did worse than this year’s?

    (I know, a ridiculous idea)

  35. bagpuss7 said,

    August 21, 2010 at 1:55 pm

    @J.Wibble. Thanks for that, I’ll research a bit further.

  36. Guy Chapman said,

    August 21, 2010 at 2:05 pm

    The decline in maths standards may also be attributable to the difficulty in recruiting students to do degrees in science and engineering; in the 1980s I know my department was accepting lower grades in order to fill spaces and this is definitely still an issue for engineering courses. Ben says it’s a shame these results are not matched with A level grades, and I agree.

  37. YJ99 said,

    August 21, 2010 at 2:38 pm

    @hleith Very few parts of the country have the 11-plus and grammar school system. The vast majority of state education in England is fully comprehensive and non-selective. Indeed the 11-plus is so out of date that I did not even have to sit it back in 1970!

    On the subject of ‘soft’ options. I saw the breakdown of grades by subject in (I think) the Telegraph yesterday and noticed that it was often the so-called ‘soft’ subjects where there was the lowest percentage of A* grades. This probably reflects the fact that it is towards these subjects that the weaker students are guided. They may be more likely to get an A-level in the subject but are as unlikely to get a top grade in a soft subject as they would in a harder one. If this is the case it would appear that ‘soft’ subjects could be seen as self-regulating when it comes to meeting Russell Group criteria.

  38. bagpuss7 said,

    August 21, 2010 at 3:10 pm

    What is most interesting about this debate is how hard it is to make any kind of quantitative measurement of education standards and in any overhaul of an education system, it would be nice to think that a system of empirically measuring the effects of that change would be devised first, so you can check that you got it right.

  39. CoralBloom said,

    August 21, 2010 at 3:35 pm

    Good article Ben.

    I knew they were dumbing down education when I watched my elder brother open a tin of tuna as instructed by the local school teacher. It said on the paper list of instructions that their kid wasn’t to do it unless supervised by a parent because it was dangerous. He was cooking a full meal from scratch for us at that age because if he didn’t we’d be waiting until mum came home from work at 7:30 pm every night!

    @hleith

    Agreed! I was educated in Scotland and find the annual ‘news’ from England bizarre, and, well, immature. It goes round in circles, and gets whipped up into hysterics. No one bothers to design a decent system and test for consistency (Evan Harris rightly points to the political benefits/costs) preferring hocus pocus instead. I suspect it suits the adults (media jounralists in particular) – we can say it was harder for us oldies, and carry on insulting the kids telling them they aren’t as bright, don’t work as hard, and their grades aren’t as worthy as ours! Multiple exam boards! For goodness sake, why? What next? Will there be exam boards opening up ‘advice shops’ in all of the shopping malls (with the increasing number of vacant spots these days, at least the malls will look busy and collect some rent!).

    It’s education, not a competition between mummy’s at the school gate!

    It needs to be sorted out. Properly. It’s getting boring after all these years.

    @FelixO

    Sorry, but education is not for the benefit of employers. Education is about providing you me, our children, everyone with a sense of wonder, their personal development. Not once was I ever given any advice on what subjects would help me find a job (though maybe that was really because where I grew up the factories were closing down at the time and there were no jobs).

    I was advised to think about two things: what I enjoyed and what I was good at. I certainly wasn’t impatient to learn to read or to write my name because that would get me a job. It wasn’t the job prospects that made photosynthesis, or nitrogen fixation mindblowing! It wasn’t job prospects that made the double-helix the makes beautiful thing I can imagine, the production of yet another crappy drawing that made my teachers search harder for the unearthed art talent and me decide the sooner I could give up secondary school art the better! And it certainly wasn’t the job prospects that made me see Dulce et Decorum Est as meaningful, moving and still important to anyone interested in current affairs.

    ‘We’re not rats. We’re human beings.’

  40. Hadenuff said,

    August 21, 2010 at 3:52 pm

    The belief in objectivity in educational statistics of any kind is wishful thinking. Only mathematics offers any kind of objective, standardised measurement of attainment. In all other subjects, the variables are too many and the level of subjectivity too great. Lies, damned lies and statistics – and then educational statistics. I have been teaching for 30 years, and have marked A level papers for most of that time. If only you knew how inconsistent and unreliable marking and grading systems are! And that’s for A levels; don’t even think about how reliable National Currculum levels or GCSE grades are. As has been pointed out by others, educational research can be used to justify almost any conclusion to suit politicians’ needs or prejudices. The world is imperfect; education is imperfect; educational statistics are extremely imperfect.

  41. hleith said,

    August 21, 2010 at 4:02 pm

    @YJ99 Yes I do realise that Kent is ‘special’! However, Kent is a large and influential county (how has it managed to hold onto the 11 Plus so long?) and I just fear that if Gove and the Con-dem Coalition get their way we may see more of the country return to this system.

    As for the possible self-regulation of the ‘soft’ options for establishments with hidden criteria – I think you are right.

  42. Zamzara said,

    August 21, 2010 at 5:23 pm

    When I took my exams in the late 90s I was told by at least two different teachers that the pass rate is set in advance, with the pass mark then set retrospectively to achieve that rate.

    I am not sure if this is true, and have been unable to confirm it, but it cerainly would explain a consistent yearly rise of just the right amount.

  43. pandateam said,

    August 21, 2010 at 5:26 pm

    I’m an A Level teacher and I think the following things have contributed to the rise in pass rates and A grades:

    – the increase in range and availability of vocational qualifications has taken some less academically able students out of the A Level system. I’ve worked in both an FE and a 6th form college now, and if FE courses like Hair & Beauty or Motor Vehicle didn’t exist some (obviously not all – some would go into employment and a smaller group would succeed at A Level) of those students would be struggling on and failing A Level courses

    – as depjac says, students failing AS rarely progress to A2. Where I work, students with an E at AS have to negotiate progression to A2. When A2s lasted 2 years, students couldn’t fail and fall off the course before the final exams. Pressure on the college also means that students are increasingly streamed away from courses they’re likely to fail. GCSE is slightly different, obviously, but at my college GCSE retake is only available to students with a D first time round, and there is also an I-GCSE for non-native English speakers. This sort of attention to who’s likely to succeed on what, and gatekeepers to make the decision, increases pass rates (and happens at A Level too, you need a C in English GCSE to do an English A Level, for example)

    – teachers and students are just more exam focused than they were (I did my A Levels in 98 and there was barely any talk of Assessment Objectives. We were more or less expected to sort ourselves out, and the teachers weren’t going to be beaten up by management if one year’s results were down)

    – possibly the increase in the number of different A levels offered means students are better able to take subjects that suit their skills/interests (this is speculative, but I’ve seen people succeed on film studies who would struggle badly if they had to do economics/law/maths/hard science/english lang.)

    – some sort of analysis of A Level results measured against numbers taking BTEC/NVQs etc.. would be interesting, I think

    From my subjective position A levels are not by any means easy, and lots of students at AS are unpleasantly surprised how hard it is after they’ve cruised GCSE. Anyone getting an A in either English Lit or Lang does properly understand the subject, and they have to be properly forensic and sensitive in their reading. The quality of work/understanding/insight some A level students produce is exceptionally impressive.

    Finally, this is by the bye, but is there any work at all on standardisation of degree marking between different universities? I would love to see some, because they don’t seem to even pretend to be doing it :/

  44. pandateam said,

    August 21, 2010 at 5:38 pm

    Oh, and also:

    – A Levels being made modular when ASs came in means retaking single units is possible, and this raises A Level achievement. Lots of students mess up AS modules but then improve in retakes the following Jan or summer (sometimes both). Some of the modules making up final A Level grades will have been sat three times (I inherited a student last year whose unit one grade had gone E, D, B on her 3 attempts). Before the AS and modules, students didn’t know they were falling short until they messed up at the end of the second year.

    – There’s probably also a small benefit from students knowing where they stand before taking final exams (because they already know at least 50% of their A Level marks by then, more if teachers share final coursework marks)

  45. MrFred said,

    August 21, 2010 at 6:30 pm

    @pandateam

    “…is there any work at all on standardisation of degree marking between different universities? I would love to see some…”

    Why?

    In my experience, degree courses tend to reflect the research interests of the staff delivering them, especially towards the later stages. That way (ideally at least) students get to learn from experts in the relevant field. Standardisation is not compatible with this.

  46. twaza said,

    August 21, 2010 at 6:54 pm

    If the results were published as both grades and ranks, employers and universities could use the ranks to select employees and students, and schools and politicians could use grades to show how clever they have been.

  47. depjac said,

    August 21, 2010 at 7:00 pm

    Following on from Pandateams’s comment about standardisation between universities. I find it difficult to reconcile the reports about falling standards of students received from schools and their ever increasing proportion of first class and 2.1 degrees. When I took my degree at a red brick university in 1984 there were 3 firsts from a group of over 100. The same course now delivers more than 20% at first class. How can this be if the students are weaker?
    @ Mr Fred As am employer trying to shortlist to an interview list from over 50 candidates it is a juggling act to try and decide how degrees on application forms from different institutions compare and we can’t possibly know whether they have been taught by experts- all we have is the grade and the name and “reputation” of the university which may of course be incorrect and prejudiced.

  48. MrFred said,

    August 21, 2010 at 7:21 pm

    @depjac

    “When I took my degree at a red brick university in 1984 there were 3 firsts from a group of over 100. The same course now delivers more than 20% at first class. How can this be if the students are weaker?”

    Perhaps standards at your alma mater have not been maintained.

    “As am employer trying to shortlist to an interview list from over 50 candidates it is a juggling act to try and decide how degrees on application forms from different institutions compare and we can’t possibly know whether they have been taught by experts- all we have is the grade and the name and “reputation” of the university which may of course be incorrect and prejudiced.”

    Tough. Standardisation might help employers with the process of shortlisting, but in my opinion it would be detrimental to the actual purpose of degrees, which is to provide the best possible education for students.

  49. depjac said,

    August 21, 2010 at 7:28 pm

    @Mr Fred. I agree completely “standards MIGHT not have been maintained.” That is my point. Who can tell?

  50. CoralBloom said,

    August 21, 2010 at 7:30 pm

    @MrFred

    Agreed. They expect students and education to bend over backwards to provide the candidates they want. Not too many employers are bending overbackwards to pay for the education they want or for the effort and time put in by the students. There is a word for that: freeloaders!

  51. Jon d said,

    August 21, 2010 at 8:08 pm

    possibly it’s now easier to produce 1st class degree standard work than it used to be by using the internet to help your research?

    anyway the A level system as explained to me circa 1987, when I was taking them, was that they asked a bunch of questions get the marks in (hopefully they’d be a lovely bell curve) and set the A grade mark at the 90th percentile – the absolute ‘hardness’ of the paper didn’t really matter very much as long as you didn’t get too much bunching at the extremes – i.e. set a paper where everyone gets everything right or no one can get any marks. Not much chance of showing ‘grade improvement’ with fixed bands though – it’d always be as close to 10% of candidates getting grade A as it could be. The exam boards were run by the universities too IIRC so presumably there wasn’t so much griping about the unsuitability of exam grades for selecting undergrad students back then.

    Seems that now the governments have, wittingly or not, set up a pseudo market system where the exam companies are rewarded for giving ever greater numbers of higher grades, with predictable results.

  52. sciencerocks said,

    August 21, 2010 at 8:49 pm

    Very interesting discussions. Interpreting the increase in proportion of students achieving high grades seems like a very difficult thing to unpick.

    @depjac

    While I sympathise with your problem in short-listing, I don’t think standardisation is the answer for some of the reasons described by others. In fact, there are arguments for ignoring final degree classification altogether (except perhaps for the lowest classifications, e.g. third). I think part of the answer is to expect candidates to draw on evidence that they have accumulated from their degree to make a good argument in a personal statement etc as to why they meet the job description (e.g. particularly relevant final year project, particularly good performance in particularly relevant module, particularly relevant work experience etc). No it doesn’t make decision making easy for you, but I am inclined to agree with MrFred’s “tough”. It’s a similar problem Universities have with A Levels and we have had to learn (in some cases a very hard way) that a single (or three+) grade doesn’t give one the measure of a person.

    @PandaTeam

    “Anyone getting an A in either English Lit or Lang does properly understand the subject, and they have to be properly forensic and sensitive in their reading. The quality of work/understanding/insight some A level students produce is exceptionally impressive.”

    May be for Eng Lit and Lang, these aren’t my areas of experience. I would question whether A Level science students (e.g. physics, biology, chemistry, psychology) *understand* much. They probably will have *known* a lot at the exam but I’m not convinced it goes much deeper in the main. I get the impression that A Level maths students doing really well in mechanics have a deeper understanding of this area than physics students. Not a criticism of them but rather curricula and, more importantly, exams that seem to me to emphasise “knowing” leading to the “filling the vessel” analogy to learning I mentioned above.

  53. thomaswp said,

    August 21, 2010 at 9:24 pm

    I like pandateam’s theories. But I also think that the system of independent exam boards competing for candidates is a cute way of politicians all but guaranteeing that grades will rise. At one school I worked at a new Head of English improved grades (admittedly at GCSE) overnight by moving to the Welsh Board’s exams.
    I think that norm referencing is a noble idea but that would not help as schools would still chase the “weaker” boards. Maybe we need to have fewer boards to increase the sample size before norm referencing?
    P.S. I’m a Science teacher. Science is two subjects now, not three so of course 1/3 of the subject is now in A Level and the paper now has old ‘O’ Level content. How could it not?

  54. SimonW said,

    August 21, 2010 at 11:04 pm

    Re: Comment 15

    I knew the marking plan for my A level physics in the mid 80’s due to the chief examiner being my teacher. I can imagine that being a big help, but you still need subject knowledge to put down the points that score those marks.

    When I did O level maths we finished the syllabus early, and did many old O level papers. What struck me most was not how hard they were, but that you could pretty much get a passing grade by knowing how the pre-decimal currency system worked.

    This I think brings home to me that if you change the material covered, the type or style of exams, then any comparison is apples to oranges. O level maths pre-1971 is not readily comparable to O level maths post-1971. Employers & Universities pretty much only need comparison with their peer group, absolute values are of interest only to educators and even then only rarely.

  55. MrNick said,

    August 21, 2010 at 11:31 pm

    My oldest has just achieved 2A*s in History and Geograpy and a B in Psychology and is pretty pleased. His A1 exams were lower and he did some resits and it seems has worked really effectively this year.

    The thing that I notice from when I did my A levels in 1973 is that they know so much more about how to study than I ever did. That coupled with downloadable past papers (with mark schemes) and probably more directed teaching means that they can go into the A2 exams much better prepared.

    My youngest sat Maths GCSE final papers and triple Science first year GCSE papers this summer. I was shocked at how little science was in the science papers. Perhaps there will me more next year? He attends a specialist Science College which makes it even more shocking to me.

    My children tell me that they know several people with GCSE A and A* in maths who then bomb at AS. So it seems that, at least in some subjects, the step between GCSE and AS/A2 is really big.
    It remains to be seen if the same is true with science subjects.

    Nick

  56. maloushe said,

    August 21, 2010 at 11:45 pm

    I’m more interested in something that’s slightly off-topic but very well showcased by this article.

    Why is it that so many people are willing to assert their opinion on whether exams getting easier or otherwise? How do they know? Have they sat multiple papers from different years? Why does no-one have the balls to simply say “I don’t know”. And perhaps postscript “let’s do a study”.

    I find that the same happens across all kinds of topics. I get accused of sitting on the fence, when in reality, it’s just the plain truth. I really don’t know, so why express a pointless opinion? Sure; discuss it, but why arrive at a conclusion when you know next to nothing about the subject?

    In response to comments #2 & #6, I think it does matter whether exams are getting easier from an employers perspective. If I have 2 CVs on my desk from candidates with an A grade in A-Level Maths, but one is 20 years old and the other is 30 – how do I compare them if an A grade is a different level of achievement at different times?

    Of course, I’m not allowing for other possible or probable differences between the candidates but the point is; I should be able to discriminate between the top middle and bottom performers consistently over time.

  57. David Millar said,

    August 22, 2010 at 12:12 am

    This is a classic case of a topic where the “I think you’ll find it’s a bit more complicated than that” tag fits perfectly. I work in the area of educational research in Ireland, where this is becoming more and more talked about (Unfortunately it seems to me that the level of public debate is where the greatest amount “dumbing down” has occurred). Year-on-year standards are also closely tied in with the area of between subject standards, e.g. is it easier to get an A grade in Art or Home Economics than in Maths or Physics? And what does it mean to make this comparison anyway? (BTW, there is lots of lovely stuff involved in summing scores from different components – when 1 plus 1 is slightly less than 2 (Millar, D., Kellaghan, T., & Mac Aogáin, E. (2006). A study of the intended and achieved weights of components in the Leaving Certificate Examination. Dublin: Educational Research Centre.))

    Getting back on subject – another reason why you get an improvement of grades across the last three decades (say) is because proportionally more of the age cohort are staying in school past compulsory schooling (which is probably a good thing). “A major objective will be that the percentage of the sixteen-to-eighteen-year-old age-group completing senior cycle will increase to at least 90 per cent by the year 2000″, according to education White Paper, Charting our Education Future (Dept. Education, 1995)(www.education.ie/servlet/blobservlet/dept_education_wp.doc). (Actually we are still a little short on that). Now, keeping more of the cohort in school for longer means that you will very likely keep more of the less academically minded in the system. They are likely to find the work more difficult and, all things being equal, are more likely to fail the terminal examinations. As keeping them in the education system only to fail them at the end of it seems an unnecessary cruelty you could make it easier to pass the examination. Is this dumbing-down? I would argue not.

  58. David Millar said,

    August 22, 2010 at 12:26 am

    Oh, and in response to #56 – if you are an employer deciding on the CVs of two candidates who sat A-level maths 20 years ago and 30 years ago; if the other information doesn’t swing the decision either way, you might as well toss a coin. Or better yet, consider exactly how large measurement error is in examinations (Dylan Wiliam “Reliability, validity, and all that jazz” www.informaworld.com/smpp/content~db=all~content=a780933579), or ponder on what sort of reference Einstein’s secondary school maths teacher might have given him.

  59. CoralBloom said,

    August 22, 2010 at 12:30 am

    @maloushe
    You need wisdom as an employer.
    If you were looking at a CV from someone educated in Italy, Scotland, Ireland, or France…
    I thought bosses were paid a whack for being willing to take risks, for uderstanding you have to make decisions with imperfect incomplete data and that making mistakes is all part of business life. When you hire people you hire people, it’s not quite like comparing washing machines or cars, no matter if when/how exams are standardised. It never has been and it never will be.

  60. Twm said,

    August 22, 2010 at 1:33 am

    All anecdotal:
    As a mature student, I recently(2 years ago) sat A-level chemistry and biology and it was interesting to compare the experience to my original exams 15 years ago.
    Overall I thought the syllabus was well balanced and the material interesting. I wasn’t expecting a ‘dead poets society’ experience, but yes it was disappointing that questions/discussions which veered off syllabus were discouraged during class time.

    The teacher was clearly under pressure to churn out good grades. Whenever we finished a subject, he would give us a wodge of paper with all the related past paper exam questions from the last ten years or so. We were coached into writing bullet point answers that ticked all the boxes in the marking scheme and it paid off as the exam questions were predominantly variations on past papers.
    Although it sounds like an awful way to learn, I have to say that doing the past papers really helped me explore the limits of my understanding with enough time to correct before the exam which is no bad thing.

    After the exam, I remember a lot of students complaining that they had no idea how to begin answering the question “Explain why transpiration is an inevitable consequence of gaseous exchange in plants.”. This was not something that had been taught explicitly and the students lack of confidence in tackling the unfamiliar question from first principles is perhaps a manifestation of the much feared ‘dumming down’. But, being around 5% of that particular paper, teachers can live with all students skipping that particular question if they can regurgitate the rest.

    The difficulty of a paper is hard to judge in isolation. I think that difficulty might be related to familiarity with the question structure/style.

    If anyone fancies a sample. here is a website that carries past papers and marking schemes for OCR.
    e.g Biology foundation
    www.thepaperbank.co.uk/viewpapers.php?dir=papers/AS-A%20Levels/Biology/Foundation%20Biology/

  61. Sqk said,

    August 22, 2010 at 1:48 am

    @ jon d

    When I was at university around the turn of the millennium my department had a simple way of stopping excessive use of internet sources: you weren’t allowed to use or refer to them. You could use them for pointers of where to start your research but you did the rest of the work in the library.

    As time moved on you could use them but you had to be able to defend (and prove) the use of each one and they were to be used sparingly. Before I left university, you’re right, there were problems with internet use, in particular the ease with which it allowed plagiarism or the sudden production of a First Class piece of work that the student couldn’t have come up with on their own. There were ways to spot it though and it was frightening just how easily a lot were caught out just because they thought they were the first to try it or hadn’t thought it through.

  62. cjames said,

    August 22, 2010 at 10:22 am

    I’m a prof. teaching a maths-based topic at an RG university, and setting exams is one of the toughest parts of the whole job (not just the teaching bit). Apart from making questions perfectly clear and unambiguous you need to make sure that the best students have to work to ‘earn’ (and be proud of) the best marks but the worst can have a chance to pass if they make the effort and pay attention. Looking at the exam statistics against standard norms (x% average, y% standard deviation) and also the cohort norms give you an idea how (your) students did and how you did (with the exam writing). Lower than ‘norm’ marks cause (you) problems and raise eyerows much more than higher than norm marks. So there’s no pressure to make exams harder, and masses of it to make the exams (and the examiners life) easier. I bet there’s an element of this in A-levels even with a greater degree of separation from consequences: so it would be no surprise if they actually are easier.
    Students are now very exam-smart too compared to my era, but that probably does not explain a steady and ongoing up trend.
    Despite this I don’t observe student quality decline: this is because university entrance requirement (can) effectively reimpose the norm-refereincing by upping grade requirements. So no great disaster if it’s true.
    By the way what happened to S-levels? They seem to have vanished without a trace.
    And finally, there’s a comment here about deteriorating report writing (i.e. communication) skills with which I agree. If that’s a result of easier English exams then that’s a bigger worry to me given students will spend half their working lives dealing with written communication.

  63. Jon d said,

    August 22, 2010 at 12:59 pm

    sqk: I was really trying to say that the internet has made it easier to do a proper job – there’s tons of respactable journals and books online and it’s easier to find relevant information in them than with the old paper system. I’ve had two bites at the undergraduate cherry and it’s a lot easier now. You used to trudge down to the library through the rain and find someone had sliced out the best pages of some crucial journal or expensive ref book with a razor blade and taken them off.

    cjames: some academic I was listening to on radio 5 about this time last year was saying the problem for setting entry grades had changed from grade inflation to grade compression,an A grade didn’t convey the useful information it used to cos the range of abilities it covered was now too great – they’d run out of headroom for upping the entry requirements, having gone iirc from ABC 20 years ago to AAA at his place. Dispite ‘raising the bar’ they were in his view getting more dstudents that went on to struggle with the course than ever clearing that bar. Interesting that your experience differs from his but the destination of a system where the number of A grades keeps going up in small increments every year for long enough would be what he describes wouldn’t it?

  64. killary said,

    August 22, 2010 at 7:17 pm

    The increasing pass rate in A-levels is an example of the Law of Unintended Consequences.

    When Sir Keith Joseph was in charge of education in 1980s he noticed that because of comprehensive schools more and more people were taking A-levels. The quota system meant that 10% of the examined were awarded a grade A and this number rose each year. Joseph assumed that this meant that the standard must be dropping for how could 2,000 do what only 1,000 did ten years before. He therefore changed the system so that to achieve a grade A you had to reach a certain pre-determined standard. He assumed that the percentage gaining an A would drop as the numbers taking A-levels rose. Of course this had the exact opposite effect because each year more and more students reached this level.

  65. thermalCat said,

    August 22, 2010 at 10:34 pm

    Please excuse me for skim reading the above before posting.

    @maloushe asks a very reasonable question, to which I can answer yes for maths & physics papers over the last 10 years (except for just a few of the easier statistics & discrete/decision maths papers); Also, papers from each of the English exam boards.

    With regard to maths, I agree with John Dunford’s comment in a recent BBC article (hastily paraphrased by me):
    “For the past 10 years (or so) questions would lead students through the problem in a number of stages, whereas now they have to decide how to tackle the problem as well as carry out the calculations. It’s required a different way of teaching.”

    I would call this additional skill ‘problem solving’. If the current generation of A-level school leavers are well practised at it, then I expect that universities & (later on)industry will be delighted with their new recruits.

    www.bbc.co.uk/news/education-11037928

  66. Timbo said,

    August 22, 2010 at 11:21 pm

    I am surprised at how many of the comments relate to views about harder or softer. Personal, not testable and desperately full of un-testable opinions.

    Wasn’t Ben’s initial article about the difficulty or indeed lack of a proven method (gosh I hate “methodology”, the twaddle word)

    So where is the evidence, not opinion, backed by testable numbers?

  67. Sqk said,

    August 23, 2010 at 1:16 am

    @jon d,

    I agree, it has. I spent nine years at university during the ‘internet revolution’ and got to see the movement in attitude and the change in accessibility: a textbook that wasn’t available to me before is now a database online. Instead of a single copy hidden under the shelves (to prevent borrowing) everyone can now access it, which can only be an improvement.

    Unfortunately, I’ve had the task of trying to explain to students why simply copying and pasting a greater number of sources from the internet won’t get them higher marks alone. They really had difficulty with this idea.

    We didn’t tend to have the problem with razored (or scapeled) pages as the subjects this affected were ones that you’d immediately want to alert people about anyway.

  68. kim said,

    August 23, 2010 at 8:57 am

    Jenni Russell did a good article on this six years ago, in which she also mentioned the Engineering Council report: www.guardian.co.uk/politics/2004/aug/20/schools.alevels2004

    I can’t help thinking there’s an element of Occam’s razor in this: 20 or so years ago, about 5-10% of students got an A grade; now 25% do. Which is more likely – that students have become vastly more intelligent in that time, or exams have got easier?

    (Incidentally, the exams themselves don’t have to be easier – it could just be that the marking is more generous.)

  69. Marky W said,

    August 23, 2010 at 10:35 am

    To me, the most compelling argument for systematic grade inflation is the consistency of the rise.

    The average grade level has risen *every* year for the past 28 years. While it’s possible (even likely) that teaching standards have trended upwards, the consistency of the rise just….smells bad. Annual road accident fatalities, for example, have trended downwards over the last 20 years, but progress has been sporadic; some years better, some years worse. The latter case feels a much more reasonable way to progress (and is much less subject to government interference).

    Any statisticians out there care to comment?

  70. WillAllen said,

    August 23, 2010 at 11:30 am

    There’s a fairly simple explanation for grade inflation that doesn’t require exams to get easier or children to get smarter.

    When you create a metric (e.g. grades) to measure something (e.g. student ability), it only works properly if the people involved don’t know in advance how they are being measured. Once people begin to wise up to the assessment criteria, they start to game the system. Witness the crazy hoops university departments jump through in the leadup to the RAE, or the obsession with impact factors above everything else for publishing in journals.

    Since schools and teachers are judged on the grades their students get, it makes sense for them to teach their students to pass the exams. A component of this is certainly teaching them the subject, but another large part is teaching them how to answer exam questions and which bits of the curriculum are likely to come up. Over time, knowledge of how to game the system improves and becomes more widespread, as the successful teachers (those whose students get good results) pass on their skills to other teachers, and perhaps are even selected for by their ability to stay working as teachers.

    Since we already know that people adapt their behaviour when they know they are being measured, Occam’s razor suggests that this should be the first thing to look at when results get progressively better.

  71. skyesteve said,

    August 23, 2010 at 12:04 pm

    As with others I had the benefit of a Scottish education – one examination authority and every candidate sitting an identical paper. Although the percentage of passes at Scottish Higher and Scottish Advanced Higher is rising I’m not sure the number of A grades is and I still needed straight As in the early 1980s to get into medical school in Edinburgh.
    I think As should be the exception not the rule and exams should be set in order to try and ensure that only the top few percent of candidates get an A but with enough leeway to compensate for the fact that there will inevitably be a year-on-year variation (up or down) as to how “clever” the candidates are. In other words if you make the grade you should get the A no matter how many others make the grade that year but the bar should be set high.
    Evan Harris’ contribution at No. 4 is excellent and, rightly, points out all the bad things about making exams easier. But ultimately there are still the same number of (or fewer) jobs or University places out there – easier results just mean higher entry requirements.
    But here’s a nice anecdote. I was recently talking to a student at one of Scotland’s prestigious art colleges who told me that the lowest pass mark in the college is an A! better pieces of work get awarded a star or, if they are really, really good, two stars! So it’s not just A levles that have been seduced by the stars.

  72. kim said,

    August 23, 2010 at 12:13 pm

    @clt42: “The idea, according to a senior examiner from Edexel, was to make the exams more standardised, to eliminate subjective responses by the markers, and so make them more reliable. The effect has actually been to reduce confidence in the exams, partly because so many essay questions have effectively been reduced to learning formulaic answers.”

    I’m sure that’s true. Viv Groskop did a piece in the graun last week about taking Eng Lit A-level 20 years after taking her first batch of A-levels. The box-ticking element was important – she lost a mark in a coursework essay on Seamus Heaney for not mentioning that he was Irish. That strikes me as barking.

    I think both Evan Harris and Skysteve make good points. When I did A-levels, it didn’t really matter to schools whether students got good results or not, apart from basking in the glow of having a good reputation. Now, with league tables, getting good A-level results is hugely important if you want to attract students (and money) to your schools. So schools do focus much more on getting students through the exam rather than having a good understanding of the subject and I’m pretty sure, as Harris says, that they shop around for the “easiest” exam board. Because exam boards are in competition with each other, it suits each board to set examinations that are easier to pass, with inevitable consequences.

  73. donovan950 said,

    August 23, 2010 at 1:53 pm

    An interesting article from the champion against misrepresentation.

    He does miss out one important aspect and that is norm referencing compared to criteria marking. That is, grades for exams up to the late 80s were awarded as a fixed % of cohort; top 15% get an A regardless of quality, bottom 10% fail, again, regardless of script content. The argument being that cohorts do not change, standards do not improve and therefore this was a fair representation of abilities of a year group. If you had a particularly easy paper then it is possible a small number of marks would separate a large breadth of grades

    Then in the late 80s criteria marking was introduced; if you completed a script to grade B level you got a ‘B’ etc. Suppose 60-80% of a paper was grade ‘B’ standard then a mark of 68+ (scores are made up) would be rewarded with at least a ‘B’. Now revision from lots of students pays off; if 50% of the cohort achieve 68+ they get a B+; remember previously only 30% could possibly get a ‘B’

    So, students who practise past papers and revise can achieve the standard required to get a particular grade. It is now about achieving a particular standard and not a position in an established meritocracy.

  74. morph said,

    August 23, 2010 at 4:02 pm

    Readers might like to have a look at the superb lecture ‘Beyond the Flynn Effect’ (maybe someone would be kind enough to help out with a web link).
    From this we can see that the typical conditions for a Flynn Effect are a relatively narrow set of skills which have been previously underdeveloped, combined with a feedback loop of increased valuing of these skills causing increased competition causing increased skill levels etc.
    Thus it can be seen that the hypothesis that children are generally more intelligent than previously is highly unlikely, but the hypothesis that they are much better at a particular narrow range of skills is quite plausible.

    Any exam tests the ability to answer the sort of questions that make up that exam, and any set of questions will cover a certain range of skills to a certain level of difficulty.
    The anecdotal evidence suggests that high grades can be achieved by having good exam technique and repetitive practising of standard essays. When added to the high level of pressure on teachers to produce good exam results, this would fit in with the requirements for a Flynn Effect.
    Thus it would seem likely that children have become very good at a narrow range of skills required to produce good exam grades. The essential questions are therefore ‘What has been sacrificed elsewhere to achieve this?’ and ‘Is the overall skill set achieved under the old system more or less desirable than that achieved under the current system?’ Part of the answers to these questions can be achieved by research but part depends on answering the philosophical question of ‘What should be the purpose of education?’

  75. gruoch said,

    August 23, 2010 at 4:55 pm

    morph:

    It is barking if true. However, without knowing anything about this course work, the fact that Heaney is Irish may well have been contextually relevant and missing that out would cost marks.
    A level English Literature is not a box ticking exercise and A and A* grades are relatively rare compared to say, mathematics. Further Maths at A Level had 29.9% A* passes, whereas all English disciplines only had a 7.4% A* pass rate (the figures are from the BBC web site where English is treated as 1 subject).
    Students know what they are being assessed on and are taught to address the assessment objectives. This is only sensible as they can’t get marks for writing about something which is not being assessed.
    Twm:
    Past papers and mark schemes have not been available for A2 as the first exam in the new syllabus was in January and sat by very few candidates. Interestingly the A pass rate last year was 23% and combined A and A* this year was 23.1%. Last year’s candidates had a wealth of past papers/mark schemes to refer to, so it seems to make very little difference on this evidence, though meeting assessment objectives is slightly different in that they now all carry equal weight.

  76. tjdavies said,

    August 23, 2010 at 6:28 pm

    It is clearly a hard problem to normalise exam scores across many years, but it is surely not a hard problem to normalise from one year to the next. Simply pay a representative sample of students to do next year’s paper as well as this year’s for you, and make them sign NDAs so that it doesn’t leak on to the ‘net. This would allow scores to go up slowly if students were genuinely getting better at the subjects, since on average your guinea pigs this year would do worse than your actual students the next year.

  77. k.r.johnson said,

    August 23, 2010 at 7:50 pm

    I became convinced that exams were getting easier when (a) the Sunday Times published evidence that old O level questions were being recycled as A level questions and (b) when I looked at a CSE maths paper and found questions like “measure the length of this line” and “write the number one thousand and seventy three in figures.” – both things which I would’ve been able to do before reaching the age of ten.

  78. bullwinkle said,

    August 23, 2010 at 9:07 pm

    It is anecdotal but I wanted to share my experience. I did a modular Chemistry course for A level for which I received an A grade (1993). I gained a place in the top science university in the country to study chemistry. My first lecture was on electron orbitals which was treated as assumed knowledge. I was flummoxed and approached my lecturer to tell him I had never been taught anything on electron orbitals, he was horrified, asked me what grade I’d achieved in my A levels and then tutted and sent me to the library.
    As I had spent my years of schooling being spoonfed information and taught how to answer exams. I failed my first year at this ‘old school’ establishment because I hadn’t been provided with the same tools to learn that I had been trained with, and wasn’t able to adapt in one short year.
    I then moved on to a university with a slightly more ‘modern’ approach (though still a prestigious institution) which was more adapted to my schooling and went on to receive a first class degree.

    Finally I went on to get a PhD in Theoretical Chemistry from the top university in the country.

    Does this mean that the university from which I got my degree was ‘easier’ or was the teaching style just better suited to the way I learn? My A level result would have been considered worthless by that first lecturer but then my performance at the second university, shown to be a reflection of my ability.

    My point is that the exam results do not demonstrate level of knowledge but rather the ability to gain it, using the right methods. It comes down to defining what is ‘intelligence’ and what is ‘easy’, as a certain skill set and motivation is still required to achieve high grades even if you know the mark scheme, but it is perhaps a skill set that more possess.

    I think the question is what do we want these exams results to tell us? What is their purpose? To provide kids with knowledge, or the tools to gain it? Or is it to provide them with the skills they need in the workplace? At the moment it seems to me that A levels do not tell us how good someone is at lateral thinking or problem solving, this is from my experience of working with younger generations many of whom have excellent academic results but still need spoon-feeding information and do not have the confidence or ability to think for themselves. This skill is not essential to some employers (may even be a disadvantage!) but it is to others, and assessment of this from potential candidates may have to be obtained through other methods such as aptitude tests which many firms use.

  79. Lifewish said,

    August 23, 2010 at 9:23 pm

    The problem seems to be that a bad A level is ranked the same (by employers and unis) as a good A level. Why not just insist that each board offers their own A level (“the Edexcel A level”, “the OCR A level”) that is ranked independently of its peers?

    That way, the market dynamics would equilibriate. Exam boards can lower standards to attract schools… but then they’d get slated by the universities, kids with their A levels wouldn’t get accepted so easily, and schools would go find a harder exam board.

    This approach seems awkward, but people are naturally good at assessing reputation in this way*. Why do Oxbridge grads go to Oxbridge when they could probably get a better grade from another uni? Because everyone knows that a 2ii from Cambridge is comparable to a First from a polytechnic. Education doesn’t have to be fungible; arguably, it shouldn’t be.

    In this world, pulled both ways, how could exam boards improve their standing? Firstly they could attempt to bribe the universities. But that would need to be widespread and hence easily detected. Secondly they could try to match their exams to the needs of unis and employers. No harm there – that’s what they should be doing anyway.

    Thirdly, however, they could work with the schools to improve education, to make the kids taking their exams genuinely more ready for their next steps. How awesome would it be if that was where boards’ incentive lay?

    * Ref. “The Origin of Virtue” by Matt Ridley

  80. dumb_physics said,

    August 23, 2010 at 10:38 pm

    Couldn’t resist leaving my anecdotes.

    I did my GCSE’s in 1997.

    In German I got a D, just by turning up. I certainly remember that I didn’t do any work, I gave up going to that class, after the first term of GCSE.

    However my real goal, and something I put more effort into than German, was Design and Technology. I wanted to get an F.

    During my GCSE graphics exam I picked up all the coloured pencils in my fist and scribbled, essentially, my paper must have looked like the work of a three year old.

    The general exam, I answered two questions .
    1) What is Mahogany? I answered that it was a wood that only slept with one other type of wood .
    2) What would you want to know about a set of shelves in a catalogue ?(or something similar). I wanted to know how much cheese could be stored

    My result for DT ? a D
    If that qualifies for a D I can’t imagine what one would have to do to get an F?
    Maybe we should give people a GCSE for just turning up, and extra marks for not eating the pencils!

  81. Marcus Hill said,

    August 24, 2010 at 12:00 pm

    Another possible reason for rising grades is the ratcheting effect of the constant (usually politically driven) changes to the specifications and styles of exams. After a new set of changes come into effect, teachers take some time to get used to them, and so they become better at preparing students for those particular exams as the next few years go by. Of course, there’s only so much improvement to be had from this effect. Fortunately, by the time teachers have managed to squeeze as much grade inflation as possible from improved knowledge of the foibles of the specification and exam, it’s time for the next set of wholesale changes. Naturally, when this new set of exams runs for the first time, they need to fine tune those grade boundaries. The easiest way to do so is by simply setting them so roughly the same proportion of candidates get each grade as the previous year. As if by magic, you’ve reset the system so the teachers can continue to improve grades by teaching to the test better.

  82. cambo said,

    August 24, 2010 at 3:59 pm

    Another interesting article from Dr. Goldacre. 2 things:

    1. Exams have definitely changed since I did my GCSEs 12 years ago. OK, I don’t have any proof and I haven’t done a proper scientific study but I did Maths and Physics GCSEs and I remember them. Now I teach them and you can most definitely learn less and do better. I don’t know whether this means exams are easier, maybe they’re just different. Personally (non scientifically) I think the fact that the iGCSE is an option and Universities take it seriously is a demonstration of the fact that the “normal” GCSEs no longer distinguish well enough between candidates.

    2. As I have taught more over the last few years, I have become amazed by the difference in standards between exam boards. I won’t name any names, but it’s perfectly obvious that the standards vary considerably depending whose board you’re learning. That’s just not right, surely?

  83. HuwOS said,

    August 24, 2010 at 5:26 pm

    I don’t understand why people think the purpose of exams should be to distinguish between the bottom, middle and top of students taking them.
    It should be determining whether they know the subject they’ve studied to the level it was being taught.
    If they are doing that and everyone is achieving the same grade then either the teachers and pupils are brilliant, in which case, there is no need to distinguish between the pupils who will all have grasped the subject completely at the required level or the course isn’t much of a challenge.

    But the fact that a reasonably challenging course, taught well will probably allow 3rd parties to distinguish between the best, worst and midstream students is a secondary effect, it should never be the point of the exams.

  84. jdharding said,

    August 24, 2010 at 9:31 pm

    I don’t think it is just the exams getting easier – its the syllabus.

    A few years ago, when I was doing my Masters, I coached (for one week) my brother in law to be in Maths as his Maths GCSE exam was approaching, and his parents wanted him out of the house for a week around Easter.

    I had taken my GCSE in 1997. This was 2004. A sizeable but not a huge difference. Beforehand I managed to download the past papers from an exam board (OCR I think) and the practice paper for that years exam for the intermediate. He just about passed, but when marking it I thought it was considerably easier than my memory of my GCSE paper from seven years previous.

    So I managed to obtain it – and the intermediate paper from my year. And got him to take it. And he failed. Utterly.

    I checked the syllabus and my years syllabus for the advanced paper was considerably harder than for the 2004 students.

    So I know…sample size of 1. Subjective interpretation not objective measurable degrees of “hardness”…..but I did leave concerned that my GCSE Mathematics grade is “worth” the same as GCSEs awarded that year. And (irrationally, maybe) feel that unless they can repeat that grade in the paper I sat, it should be worth less. Considerably so.

  85. donovan950 said,

    August 25, 2010 at 8:05 am

    @dumb_physics you don’t mention your coursework which amounts to 65% of the course. I hope you’re not misrepresenting.

    Quite a few comments how it was much more difficult “in their day”. Reminds me a bit of Plato’s apparent quote:

    “The children now love luxury; they have bad manners, contempt for authority; they show disrespect for elders and love chatter in place of exercise. Children are now tyrants, not the servants of their households. They no longer rise when elders enter the room. They contradict their parents, chatter before company, gobble up dainties
    at the table, cross their legs, and tyrannize their teachers.”

    I do wonder how much of this exam-bashing is a means to scold teenagers; how many of us think it was so much more difficult for us in our day.

  86. xiaoyan said,

    August 25, 2010 at 8:11 am

    fasfsdg

  87. mikewhit said,

    August 25, 2010 at 5:41 pm

    I thought the reason for different exam boards was the different universities; at our school we had the JMB exams in some subjects (“Joint Matriculation Board”, look up “matriculation” !) and other exams were Oxford & Cambridge.

    Don’t know if there were any other boards.

  88. skyesteve said,

    August 25, 2010 at 7:29 pm

    But why is it so hard to have a single English exam board so that everyone in any given subject sit the same exam? Then there is no dubiety about the results. Of course, the exam may be too easy (everyone gets an A) or too difficult (no-one gets an A) but otherwise the only differences will be down to indidual schools and individual pupils. Clearly results may be poorer in areas of higher social deprivation for a whole host of reasons that don’t need outlined to readers of this forum. But if similar schools serving similar areas have vastly differing results that could help identify more readily poorly performing schools could it not?

  89. David Millar said,

    August 25, 2010 at 10:43 pm

    Coe, R. 2010. Understanding comparability of examination standards. Research Papers in Education 25, no. 3: 271-284.

    Newton, P. 2010. Contrasting conceptions of comparability. Research Papers in Education 25, no. 3: 285-292.

    Also for those of you who want something statsy try the excellent

    Newton et al (eds) 2007. Techniques for monitoring the the comparability of examination standards. London: Qualifications and Curriculum Authority.

    Of course, none of it matters a damn compared to face validity. What people believe is vastly more influential than what is true.

  90. Marcus Hill said,

    August 26, 2010 at 9:05 am

    True, David, and what people believe is guided by the popular press, which has a tendency to be reactionary and view the education system of four decades ago as some sort of utopian ideal.

    Whilst I wouldn’t want to make the bold claim that exams aren’t getting easier (and have, in fact, argued anecdotally for the opposite proposition), I would suggest to those comparing current exams to those “in my day”, especially when seeing current students faring worse in old papers than current ones, that a sixteen or eighteen year old attempting an unfamiliar style of test when he or she has been coached for one particularly narrow method of examination is at a disadvantage over an adult who has years of experience in applying knowledge in new and unfamiliar situations taking the new test. Whilst there may indeed be a difference in levels of difficulty (however you choose to measure that), the perception of the gap will be distorted by this asymmetry in the facility of applying knowledge more flexibly.

  91. irishaxeman said,

    August 26, 2010 at 11:07 am

    Writing as someone who has examined since 1986 and latterly as a Chief Examiner, my experience has been that exams to some extent have become easier. This is principally down to
    1 fairness rules dating from 2000 that ensure predictable examination questions (i.e. must use words from the specification) which makes teaching to the examination MUCH more likely
    2 date-related league table pressure that moves teachers towards generating model answers that students memorise (we call ‘em clone centres)
    3 government demands for normative % for grades, so we award a certain range of A grades regardless. This is supposed to smooth out difficulty differentials between papers but in practice disguises declines in performance

    In my – an many other colleagues discussing this at INSET, conference etc – experience the has been a relatively recent dive in student capability due to two major reasons
    1 the whole system up to and including GCSE has been totally focused on passing tests instead of education so students are becoming less skilled especially at thinking
    2 the rise of intelligence and communication aids which substitute for thinking or act as profound distractions (search engines, phones, XBox etc)

    It’s a complex phenomenon, but I share Baroness Greenfield’s worry about changes in brain use/development. Many of my students are book-averse!
    2

  92. cdubb said,

    August 26, 2010 at 1:42 pm

    This is, as you correctly point out, a very tricky issue to research and come to a meaningful conclusion. However, I would argue that we’re asking the wrong questions. Are exams still serving their purpose? This is the question we need to answer. Is our education system serving it’s purpose? The ‘exams are easier’ debate is interesting, but not going to change the world.

  93. Bishop Gillian Wakefield said,

    August 26, 2010 at 3:01 pm

    93 replies. Is this some kind of record?

  94. Marcus Hill said,

    August 26, 2010 at 3:18 pm

    Nope.

    www.badscience.net/2010/02/obvious-quacks-the-tip-of-a-scary-medical-iceberg/

  95. DanielDWilliam said,

    August 26, 2010 at 4:00 pm

    Is this essentially a question of inflation? Exam results play the part of money and entry to University courses the role of goods and services and student effort plays the part of input labour.

    Entry tarrifs are the price students pay to get onto a particular course. Assuming everything remains the same the same amount and quality of student labour should buy you entry to the same course. The question as with money is not how much do you have, but how much more or less than everyone else do you have.

    Tarrifs will be adjusted specifically for the relative popularity of the course (itself being a proxy for the quality of the course, the costs of attending and the job prospets of graduates). This is a quality of goods question.

    Tarrifs will also be adjusted generally if there is an imbalance between student numbers and places. This is a general supply and demand question.

    Tarrifs will be adjusted too if exams became easier (i.e. the currency was debased). However, tarrifs would also be adjusted in the same way if the quality of the input of student labour improved, say through improved teaching (or teaching more focused on exam results). This would result in too many genuine good grades chasing too few places. This is the inflation question.

    Over time the response of Universities to improved teaching should be to increase the number of places on offer. As inputs (A level students) to the courses improve the outputs (graduates) should also improve. This should in turn increase the amount of wealth created by the tertiaty education sector leading to more resources being allocated to tertiary education and an expansion in the number of places on offer. (This appears to be what has been happening).

    One way of measuring whether exam result inflation was occuring would be to compare over time the tarrifs for particular courses. You would have to adjust for any improvement in those courses popularity which would make them more desirable. This could be done by looking only at the tarrifs for, say, the top 5 rated Universities for each course and the balance of students to places, by adjusting for the % of applicants who were denied a place at all. Assuming that any effect caused by improved teaching was cancelled out by expanding places having identified that tarrif inflation was taking part you could assume that it was due to debasement of exams. This would be a brave assumption.

    A way to test this assumption would be to offer a small but signficant pool of A level students a standardised exam. Questions would be drawn from a pool of carefully moderated questions that only changes slowly over time (the methodology used by the Best Companies questionaires might be appropriate). This would provide a Gold Standard or the base year of an RPI basket of goods. Over time you could correlate the performance of students in the Gold Standard exam against the trend of tarrifs. If both go up in line with each other, excellent students or teachers are getting better. If tarrifs go up but performance against the Gold Standard remain static, exams are getting easier.

  96. NorthernBoy said,

    August 26, 2010 at 10:46 pm

    One point that I think may not have been mentioned is that even if people really are better at sitting a-levels now, or are better taught, it’s not “a-level ability”, or knowledge that most employers or universities actually want, but rather they are seeking a measure of a person’s ability and work ethic, or some other indicator of how useful they will be in the job. The actual exam result is just a proxy for this other, harder to divine set of qualities.

    This means that even if people are better taught now than ten years ago, I’d still prefer normalized grades, as I haven’t seen any evidence that the skills that matter to me as a recruiter have risen as dramatically as exam performance has.

    On the subject of what research has been done, not all research ness to be formal and published to shed some light on the question. A few years back I was comparing 80s Maths levels with 00s GCSEs, and the more recent papers covered what appeared to be a far narrower range of subjects, with far more hand-holding en route to the solution.

  97. NorthernBoy said,

    August 26, 2010 at 10:54 pm

    HuwOS, I think that your view, that exams are not primarily to allow a third party to judge a student’s ability is one that not many will share. The reason that we educate children is generally so that they can make their way in the adult world, and much of the reason behind giving them exams is so that employers can find out which ones are best suited to which job.

    The idea that exams are just so that pupils and teachers can measure how well a subject is being learned obviously raises the question of why we should care what the results are, if they are not going to be used as a point of reference by anyone else.

  98. andycal said,

    August 27, 2010 at 9:17 am

    Here’s a thing. Our comprehensive school was one of the first to introduce the GCSE. We’d started by doing a year of O-level style work, then we suddenly switched in the second year. You can imagine it was quite a jolt, we were young, hormone filled and here we were faced with a complete change to our schooling. The school was also crap and failing, it was pretty clear we were being used to test something new.

    Anyhoo, I quite fancied myself as a scientist so I practised my physics and chemistry, especially as the o-level test papers were so bloody hard.

    So we went into our GCSE ‘higher’ physics exam and I kid you not, here are some of the questions:

    “Estimate the amount of water in the flask”.

    OK, estimation is a useful skill, but it’s not really challenging when there is a level marker on the flask.

    “How far is the bottom of the suspended weight from the floor?”

    Here we had a weight on a piece of string suspended from a clamp attached to the desk. To make this task particularly tricky they provided a metre rule and we had to get on the floor which was pretty dirty.

    Actual questions. I only got a ‘B’ too, so I was particularly miffed.

    Oh, chemistry had a question where there was a picture of a big container full of water with holes down the side. We had to draw the flow of water out of the holes.

  99. Neil said,

    August 27, 2010 at 5:41 pm

    Would an international comparison perhaps shed more light? I imagine there are more static curricula around the world than in the UK. Perhaps testing against an international baccalaureate or similar would help.

  100. Crantock said,

    August 28, 2010 at 9:03 pm

    When I was at school and university the teachers explained the importance of “outside interests” etc and failed to explain what they meant. 25 years later as I look at new joiners I find that whether there is some “life”, “personality”, “spark” is more important than academic ability which I can assume meets a basic threshold.

    Someone who is a quietly bright will progress less far than someone who is brilliantly average.

    So once someone in HR has done the filtering do exams matter to me?

    Yes because I am about to lose one of my better junior colleagues because they can not pass their professional exams because they are too bright and “go off on one” rather than spread their effort.

    And the point is……exams are just one factor, they should not be the be all and end all.

  101. jamesc said,

    August 29, 2010 at 11:28 am

    The London Mathematical Society produced this in 1995 and is very relevant to the current discussion.

    www.lms.ac.uk/policy/tackling/report.html

  102. eddwilson said,

    August 29, 2010 at 7:58 pm

    I could believe dumbing-down skulduggery if the results soared for year n compared with n-1, but 20 years of incremental advance? That would require an amazing amount of careful planning and implementation which could not stay hidden.

  103. penglish said,

    August 30, 2010 at 2:23 am

    There seem to be two purposes for exam results. A secondary purpose is to monitor the quality of teaching, learning, and testing. This latter one is harder, as we have observed grade inflation that is probably not really because people are becoming genuinely cleverer.

    The main purpose of exam results, however, is to distinguish between the achievements of individual students, mainly within the same year. I can see no reason why the grades given could not be given as centiles, with the best 1% of students being in the 99th centile, and so on. Or even, why the actual marks could not be published. If the student’s actual mark, and the average mark for the year were also published, this would go some way to comparing students in different years.

    In reality, if we make the (possibly not exactly true, but likely to be very close to the truth) assumption that people on average are equally intelligent year on year, then knowing which centile a student was in would be a pretty good discriminator between results.

    This all presumes, of course, that the exams accurately measure something important, and which they are setting out to measure, of course – another matter again.

  104. irishaxeman said,

    August 30, 2010 at 8:11 pm

    Probably the most useful course I’ve ever taught on, moving Post 16 students on to uni, was BTEC National Health Science. It was largely project-based. Skills were paramount.

    A-levels currently are possibly fit for purpose for a few subjects (sciences most usefully) but far better tests and a far better education system eludes the blinkers of the political set. The Bacca is better but not perfect. Frankly I cannot for the life of me see why a foreign language is not mandatory from primary school onwards – most other countries manage at least one if not more, for all levels of accomplishment.

    All citizens should be literate, numerate, scientifically literate, have a foreign language, geographical nous and a grasp of history beyond the Oxbridge Greek lovers or the post-modernists that have dominated C20 specifications. I teach a short and very popular general studies topic which traces the development of what is now Britain from first inhabitants 1/2mya. Most students ask why this is not taught in history. The curiosity is there, the educational system isn’t.

  105. rhymos said,

    August 30, 2010 at 8:43 pm

    The main reason why research is so limited in this field relates to the way teachers are taught to research on higher level qualifications at national and international scale. The scientific model has been replaced by action research. This methodology is a ‘flexible spiral process which allows action (change, improvement) and research (understanding, knowledge) to be achieved at the same time’ (Dick 2002 p. 2). This is a methodology takes out many of those pesky things like pre-protocols, blinding, control groups and statistical analysis. Check out the National College of School Leadership website (www.ncsl.org.uk) to look at the standards of research presently being promoted. Alternatively a good way into the educational establishments views on research methodology try www.ngfl-cymru.org.uk/eng/pedagogy-conf-o8.
    DICK, B. Action Research: Action and Research. www.sci.edu.au/schools.html2002

  106. rhymos said,

    August 30, 2010 at 9:09 pm

    Any UK based research has to take into account the great divide between private and public sectors. Any meaningful study has to separate pupils into these discrete groups and then examine how their achievements have changed. I suspect grade inflation amongst pupils from private schools have risen at a much slower rate than those from public schools. Our kids are catching up. Societies response – move the goalposts The new A* is a marketing tool for the private sector to differentiate their product.

  107. NorthernBoy said,

    August 31, 2010 at 12:20 pm

    rhymos, what is this great divide between private and public sectors which you mention, and what justification do you have for proposing this as the important divide? Would it not make more sense to consider, for example, selective against nonselective schools, or those with streaming against those which put all abilities into the same class (not to be confused with comprehensive schooling, where all abilities are in the same school, but where there can still be different classes for different abilities)?

  108. rhymos said,

    August 31, 2010 at 4:49 pm

    NorthernBoy – 7% of UK pupils go to private schools yet these pupils dominate the top universities and professions. Remember if the new cabinet reflected the UK’s population as a whole then 4 million of us would have gone to Eton. The first set of A* results are dominated by pupils from private schools. The divide is central to understanding what has happened to UK schools results. All your other suggestions for investigation are sensible and should be considered in explaining differences within the public sector. You never get far in research when you compare chalk to cheese and pretend they are the same.
    I am an experienced teacher and my view is the main reason the data has changed is state teachers are much more effective at getting pupils to pass exams (refer to comment 91). I am not suggesting for a moment that state teachers are better educators than past teachers only better at “playing the game”.

  109. irishaxeman said,

    September 1, 2010 at 5:48 pm

    The NCSL is a nest of vipers dedicated to working teachers until they crack. The only interest is in preserving their salaries by hitting the statistical targets. Real research is of no interest – they are ideologues not scientists.
    Some heads stand out with their dedication to real education. The number gets smaller every year.
    Bitter and twisted? Biased by 30+ years experience?
    Yup.

  110. grumpy bloke said,

    September 1, 2010 at 9:25 pm

    In John Pilger’s excellent book,Heroes, there is a telling(standby for really bad paraphrasing) statement where a senior Minister or Civil Servant is quoted as saying that it is bad enough having 4 million people unemployed and even worse if 4 million well educated people were unemployed. This was a pre-cursor to the GCSE system.
    After last years GCSE results I followed a few threads on the exams getting easier debate. One of note was that research by Norwich? University indicated that todays A* grade was equivalent to the old GCE grade 6 or F.
    As for private schools, results mean income. My ex wife taught English in a state school and the GCSE was 100% coursework with controlled submissions by the pupils to enable the examining board to check the standard of grading by the teacher. On a day exchange with a local private school these controlled submissions were marked and corrected by the teachers and handed back to the pupils in order to re-submit them.
    I agree with rhymos (108) that teachers are more effective at getting pupils to pass exams and the curriculum material is tailored to the exams e.g. when looking at a French GCSE paper in my step-daughter’s school I recognised many of the questions from the coursework completed that year. In other words pupils are being coached to pass exams, whether this is the right approach or not I don’t know.
    It would be interesting to see how our education system compares to our European partners.

  111. Ted Ainsworth said,

    September 3, 2010 at 10:42 pm

    Here’s why:

    1. Schools compete to get the best league table results.
    2. Schools are able to select which exam board syllabus they teach
    3. Some exam boards set easier exams than others
    4. The exam boards therefore compete for business by setting the easiest exams/courses they can get away with, including methods such as modular courses and assessed project rather than 100% exam.

    Government has turned a blind eye to this because the improving grades make good, but now unconvincing headlines. The remedy is a single, national exam board, that is independent of government influence. The conservatives included this in their manifesto, so we’ll see…

    When I did A levels back in 1995, we did past papers from the previous 5 years as preparation. The older ones were much harder than the more recent, and there was significant content that had been cut from the syllabus just in those 5 years.

  112. Bonsai said,

    September 4, 2010 at 12:20 pm

    An older female academic, successful professor, hod, permanent employee but post-menopausal and pre-retirement – informed a group of younger female researchers that ‘these days a Bachelor of Honours is the equivalent of what A’levels used to be (back in the 1600s or something?!).

  113. Bonsai said,

    September 4, 2010 at 12:20 pm

    oops missed the apostrophe – do you have an edit function?

  114. Bonsai said,

    September 4, 2010 at 12:21 pm

    err I mean quotation marks. Maybe she had a point?

  115. ferguskane said,

    September 5, 2010 at 11:23 am

    The ‘Flynn Effect’ and ‘the idea of kids getting cleverer is not ludicrous’.

    IQ tests are used to obtain an estimate of intellectual abilities. As morph says, what they actually do is test one’s abilities on a selected range of specific tasks. This makes IQ tests pretty good at finding intellectual deficits that may result from trauma or developmental difficulties – although they have many well known flaws even here (problems for instance with ecological validity). What they don’t really do well is measure any kind of ‘innate intelligence’. That’s a tad harder!

    One’s scores on such a test are clearly and demonstrably affected by cultural/educational factors, as well as by one’s ‘innate’ intelligence (although I guess ‘innate’ is rather hard to define here). Education and culture clearly change over time and will affect average IQ scores.

    So: ‘the idea of kids getting cleverer is not ludicrous’. No it’s not. Especially if you consider ‘clever’ to mean ‘good at IQ tests’. But if you mean clever to mean something else, some evidence would be nice.

  116. Chenneth said,

    September 5, 2010 at 11:42 pm

    Comment 111 is pretty spot on. I work in futther education and from my unscientific expereince there is a dramatic decline in standards.
    1. Where as a student would previously need 5 GCSE grade C and above, including English and Maths to go onto A levels the college I work at at(biggest one in the country) this has been reduce to 4 without English or maths, all a student needs now is to pass a level 2 literacy and numeracy exam (www.keyskills4u.com/tests/) have a go an you tell me if you think this is the equal to a GCSE, becauase it’s considered equally when taking on students. So as an example I was forced to take on a student who has an E in English and a D in maths but managed to pass the basic skills exams… is it realistic to expect this student to pass A levels?
    2. This has a futher effect of lowering standards because lecturers are expected to get certain pass rates or face the sack therefore staff will do ANYTHING to get these student through their exams… hence why many courses at the college have a 100% pass rate.
    3. The college is paid for the number of students it takes and the number who pass… end product eveyone pass and the qualification is worthless as it is too easy and does not seperate the good from the bad.
    4. The government is able to porduce glowing reports of improvements in education and continues to roll out even lower standards and the cycle of decline begins again.
    5. All the lecturers have taken level 2 numeracy tests… why? it increases the number of passes and so the amount of money that can be claimed from the government.
    6. The college is a private business that sells qualifications… the more it sells the more money it makes… anyone see the confilct of interest?
    I am far more worried about the privatisation of education and dubious business models being used within the sector.

  117. grumpy bloke said,

    September 6, 2010 at 9:37 pm

    Re 116, your first sentence alone is evidence of a decline in standards. I hope your subject is not english.

  118. davevincent said,

    September 7, 2010 at 7:41 am

    This article and the ridiculous discussion reminds me of an amusing paper: ‘Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials’ (see www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/).

    Of course exams are getting easier, you fools! Doesn’t mean kids are any cleverer or more stupid than they used to be, though.

  119. NorthernBoy said,

    September 9, 2010 at 12:26 pm

    Chenneth, I have to agree with grumpy bloke, your writing is absolutely execrable, and it’s clearly not only typographic errors. If you are seeing standards even lower than those which you demonstrate, then we are indeed in a bad state. I know that it’s impolite to mention someone’s poor writing skills, but it seems relevant when you are questioning the literacy of others. This, I suppose, though, is a separate issue (although a related one), to the question of grade inflation.

    As we see more and more people gaining the higher grades, recruiters are going to need to start placing more emphasis on higher qualifications, and hopefully the most able students are still going to rise to the top eventually, and show their ability in other ways. Gade inflation is an issue, but as long as discrimination occurs eventually, it’s not a fatal problem for people seeking to rank candidates.

    I never crunched the statistics when I was a recruiter, to compare across years, but as we started looking globally, one of the metrics that turned out to be a good guide was quality of university degree. The guy with the engineering degree from Imperial may have similar GCSEs to the one who studied at a poorer institution, but it was very rare indeed to find a greater level of ability in the latter.

  120. sockpuppet said,

    September 10, 2010 at 12:09 pm

    1-Most of the moaners about educational standards were educated many years ago.
    2-Science is better understood now. If you take the view that science is only a hundred or so years old(as an average)then thirty years of progress may create a significant change in the field leading to better understanding and improved teaching. Particularly if in your purview of the last thirty or forty years you consider the contribution of IT. So it may also be that the subjects just aren’t as hard as they used to be but that doesn’t excuse them from being easier. If that is the case perhaps we should expect more from our students rather than less. To make up the shortfall if you will.
    In the meantime rpbt1 will have to deploy his own maths and verbal reasoning tests if he is not satisfied with the current lot from the exam board.

  121. Su_e said,

    September 12, 2010 at 8:32 pm

    Exams have got much much easier – I still have some of my old O-level papers, and picking one that is not subjectively marked (maths); I can breeze through a modern GCSE paper, but I would have to do some revision to do well on my late 1970s paper.
    Yes, I know I fall into the anecdotal camp, but we could test this out by scanning O-level papers and giving them to people of their vintage along with today’s GCSE papers. This gets rid of the problem of a trial of teenagers not being fair as they have only studied for the modern exams

  122. Ely said,

    October 2, 2010 at 2:54 pm

    Well, I both dumbing down and students becoming better at it have to do with it.

    Most schools stick to the same exam boards because the more years the teacher has spent teaching the specification the better he can actually “teach the exam”. I’ve seen it. My teachers tell me that so openly (because I’m not native to the UK), is frightening. Nowadays, books have harder questions than the exam (specially in science subjects, which I’m taking).

    Also, students are actually quite smart. If the only thing that you’ve to do to learn is “box ticking” and experience doing exams, they’re going to tick the boxes AND look for someone who already went through it and pester that person for how to give the perfect exam answer. It becomes even easier if they insist on dumbing down the papers.

    It would be easier if there was a draft of a National curriculum and the school are free to teach more (not less) if so they wish and schools were in control of their own exams, made by each department, with a government watch-dog body to maintain the exams’ standards and to ensure the kids are learning what they’re supposed to be learning, if not more.

  123. Phil said,

    October 7, 2010 at 5:18 pm

    Unfortunately I know from my own personal experiance that exams are indeed getting easier. I took my GCSE’s in 1989 (the first year that they were introduced) and then subsequently A-Levels in 1991. For A-level maths exam practice we were give O-Level papers form the 1960’s/70s that contained, amoungst other things Calculus which was’nt even on the GCSE sylabus. Also in my first year of Uni (Science degree) they had to lay on extra maths courses to get the students up to an acceptable level.

You must be logged in to post a comment.