UK Government Was Too Slow to Recognise Drawbacks of Grading Algorithm, Scholar Believes

© REUTERS / SIMON DAWSONPeople take part in a protest over the governments handling of exam results, outside the Department for Education, amid the spread of the coronavirus disease (COVID-19), in London, Britain, August 22, 2020.
People take part in a protest over the governments handling of exam results, outside the Department for Education, amid the spread of the coronavirus disease (COVID-19), in London, Britain, August 22, 2020. - Sputnik International
The UK has moved to cancel the exams for A-level and GCSE students due to the COVID pandemic and replace them with an algorithm that calculated an approximate score based on teachers' grades and the average school performance. However, the government had to ditch the algorithm, which resulted in nearly half of the predicted grades being downgraded.

Dr Matt Cole, political historian of modern Britain at the University of Birmingham, believes that the government was too slow to recognise that the set up system was a fiasco.

Sputnik: At least 79 percent of respondents believe that the UK government "badly" handled advanced (A)-level and General Certificate of Secondary Education (GSCE) results this year, a Daily Mail poll has revealed. The British PM has not yet publicly apologised to students and parents over the exams fiasco. What are the right steps to take to resolve this crisis that should be expected from the UK government? What are your expectations?

Matt Cole: I'm not surprised that 79 percent of people think that it was badly handled because I think the government is more or less admitted that it was badly handled. The education secretary has apologised and has reversed his policy. The problem is - it's not quite clear what he apologised for. And the reversal of the policy was too late. It caused significant problems in the transfer of students from school or college to university. It took so long. So, it's not merely that a bad system of replacing the A-level exams was set up, but the government was slow to recognise it. It was recognised immediately in Scotland on August the 4th for example. And everyone could see why it was widely anticipated that the same problem would crop up in England. Should the prime minister intervene? The prime minister has been notable by his absence, it might be said. Pictures have been published in newspapers of him on a camping holiday in Scotland. He did speak about the exams early on in the controversy and said he thought they were robust and that the original version of the exam results would serve employers and universities well. That, of course, made it more difficult for the education secretary to reverse his policy.

A-level student holds a placard during a protest about the exam results at the constituency offices of Education Secretary Gavin Williamson, amid the spread of the coronavirus disease (COVID-19), in South Staffordshire, Britain, August 17, 2020. REUTERS/Jason Cairnduff - Sputnik International
UK Exams Fiasco: Most Voters Want Education Secretary Williamson to Resign, Poll Reveals

So, yes, the prime minister has not commented on this process as the policy has been changed. Doubtless he has been consulted by the education secretary. But from the prime minister's point of view to intervene would be to acknowledge that it was a national crisis and he surely does not want to do that. But it also allows him to avoid any blame for the crisis. So, there is a legitimate desire not to escalate the crisis by intervening and thereby undermining the education secretary of course, if he stepped in it will be fighting the education secretary’s battle for him. But it also means that the education secretary, who is a vulnerable minister at the moment, has been subject to a lot of criticism from within the Conservative Party and outside. His weakness or the poor opinion of him, does not reflect back on the prime minister too much.

Sputnik: The poll was released as the chair of the Office of Qualifications and Examinations Regulation (Ofqual) Roger Taylor reportedly threatened to quit this week unless Williamson publicly supported the exams regulator. What would be the consequences of this, in your view?

Matt Cole: What this is revealing is what went on behind the scenes in the week after the results came out. What apparently happened, and this is what we know is made up of leaks and rumours. So, we have to be a bit careful. But it appears that Ofqual quickly became divided about whether to stand by the results that they had originally produced. And an increasing number of the Ofqual board were telling the education secretary, we have to accept the teachers’ assessments. What Roger Taylor is suggesting is that at some point he said to the education secretary, look, we have to change policy now and that it was the education secretary who was reluctant to do it. Now, since the results have been changed, the education secretary has hinted that Ofqual had been the people to resist the change. So, there is now a blame game going on between the administrators – Ofqual – and the education secretary, who in British politics is ultimately responsible for the policy of the department and the policy of the department was to run this algorithm.

The education secretary had every opportunity to monitor its operation and it produced outcomes which could have been anticipated and about which he was warned. And which he then took several days after they’d come out to correct. So, the British government is skeptical about the civil service and about government agencies and employees and feels that they are often not responsive, not dynamic, and not effective in their work. And it was tempting for a government minister to point to or to gesture towards Ofqual on this occasion and say they we commissioned them to do this. They did it badly and we had to turn round and correct it. Ofqual are now indicating that they were the people who realised earlier than the education secretary what went wrong. The education secretary has done his best to give his support to Ofqual. He has rolled back somewhat in his tone in talking about Ofqual. But the question of responsibility hangs in the air. This matter has not gone away for the education secretary because there remain students who have been unable to take up very prestigious university places or university places, they were very keen to take up urgently, whose places have now been filled as a result of this process - by other people. And he is facing the reopening of English schools in the first week of September. He promised to reopen all schools and the prime minister backed him in this, or primary school - back in June, and he failed to do it. If he fails to reopen schools at the beginning of the coming term, he will be in great difficulty. So, the education secretary is keen to present this as a technical problem, to which he responded as quickly as he could. Ofqual are saying this is a problem that we pointed out to you somewhat earlier. And were urging you to correct. And you, for political reasons didn't want to.

People take part in a protest in Westminster in London over the government's handling of A-level results, Friday, Aug. 14, 2020. Thousands of school-leaving children in Britain have been left distraught after finding out that they were handed lower-than-expected grades. The government is under growing pressure to address the question of how to fairly award students grades in the absence of actual exams. - Sputnik International
Britain’s Exams Chaos: The Fiendish Algorithm Behind Shock A-Level Results

I mean, there are two political problems for the government. One is that they have a let’s say, somewhat distant relationship with the teaching profession and did not wish to be seen to bend to their judgement. And they also did not wish to be seen to take their cue from Nicola Sturgeon in Scotland, who had it within 24 hours, turned round and used teacher assessed grades. So, they didn't wish to be seen to be weak and to give in to criticism. But on the other hand, the evidence was building up that it would be necessary to do that. But Gavin Williamson remains on the gangplank, if you like. And how the school reopening goes will determine what happens to him. 

Sputnik: Some 280,000 students saw their grades fall by one grade, with many losing their places in their chosen courses after not obtaining the required grades. How could this situation affect the attitude of young people towards the government? How will it affect the government's image?

Matt Cole: Those that have still been unable to go on the courses they wanted, who are relatively small in number, I mean, these are people who wanted to go to courses that have placements like medicine, for example, or certain engineering courses. And those people may have been told you can't go to university this year on the place that you were promised. Now they will be very upset and they may even go to court about it, because if they're promised a place by university, on condition they get the grades and they now have those grades they will be enormously resentful, not only to the university, of course, but to the government that created this situation. The student that ended up at the university or in the job that they wanted to go to will still feel the distress of the week they’ve just had first. They will still be resentful. And even those that weren't personally affected by it will know lots of other students that were affected by it. And there is a perception not just amongst the students, but, of course, their parents and grandparents, people working in universities, employers trying to recruit students. There's a perception of incompetence. And this is a problem that the government has come up against in other areas. The real significance of a political event, of a crisis, is whether it reinforces or undermines the previous existing impression of a government. And the weakness of the government in opinion polls has been that it does not seem to be able to achieve or sometimes even to describe its objectives. That's been seen in COVID.

Cambridge, UK - Sputnik International
A-level Results Row Spirals As Students Hold Exams Regulator to Account Over New Appeals Policy

It's, of course, still being seen to some extent in Brexit where no agreement has been arrived at. And so, there is an effect which goes beyond age, that is of all generations. But of course, young people are, generally speaking, not well-disposed towards the Conservative Party. That's just a historic voting pattern. The older people get the more conservative they are. That might be the loss of one's mind or the growth of wisdom… That’s up to you to decide. But probably the most significant thing about this is that it will strengthen, if it persists - bear in mind there’s three and a half years to go to the next election. If it persists, it will encourage those young people to vote, apart from not voting Conservative -- the main pattern of young people's voting pattern is that they don’t vote at all -- fewer than half of them turn out to vote -- 18 to 24 year olds. It might cause them to vote more. And I suppose the thing you think about here is what the Liberal Democrats did with tuition fees. They gained the reputation of having broken a promise to young people, having imposed costs on them rather than the distress that is in this case. And they were severely punished at the ballot box for it. They lost. Now, that, again, was part of a larger pattern in which they were seen not to have had enough impact on the coalition government. But here, young people will see in very sharp relief, an existing perceived weakness of the government for being unable to carry out its own aims. And I think the absence, the invisibility of the prime minister does not help with this. If the prime minister does not seem to be doing anything, then he becomes perceived as responsible for the failure of the government.

So, yes, this will have a very specific impact on certain young people. It will have a more general impact on others, but it feeds into a wider impression of the government as being unable to achieve the aims for which it was elected or which it had promised to achieve, in the case of the A-level results. There is mounting evidence that this problem had been pointed out to the government. The problem of the A-level results being pointed out to them during the summer. They knew that a particular group of schools had overpredicted students, which is why they imposed this particularly severe and clumsy algorithm on the process. And that they could have acted sooner. So, in a sense, it's not the government's intentions or its policies or what it promised to do that’s wrong in these people's minds. It's the fact they weren’t able to do it and they could have been able to do it. Have they applied themselves more effectively. 

Sputnik: Could this failure indicate a crisis in the education system in Britain as a whole? What is the main mistake in your opinion that led to such a situation?

Matt Cole: Everyone agreed that the starting point of the process -- I'll come to the crisis bit in a minute -- but everyone agreed that the starting point of this should be teachers’ assessments. And they were made back in June -- in some detail, according to instructions given by Ofqual which said -- don't go wildly out of the of the range of results you got last year. We expect you as an institution, as a school or a college to stick roughly within the range of results you got last year. And all that was done and sent in to Ofqual. Ofqual then said look, we will put this through a process of moderation to check that the results are plausible. If they are wildly different from previous years, we will impose an algorithm onto the results, which will bring them back into line with last year's results. The problem is that they applied an algorithm which treated some, not intentionally, but they ended up treating some sorts of students and some sorts of institutions different from others.

So, for example, schools had very small numbers of students doing particular subjects, weren't moderated, they didn't look at them. I think the numbers… if there were fewer than five students doing the subject, they didn't bother looking at them. They accepted the teacher's grade. And that was one of the first mistakes. They used individual students’ previous performance as a way of judging them, which, of course, meant the students who had been at weaker schools, usually working class students. And, of course, the smaller classes would be more often in the private sector where middle class students were. It meant that students going to schools or more often colleges with very large entries were moderated very hard, that their results were examined very closely and were downgraded more often than those who were in small classes, in small schools. And that tended to affect working class students. If you use their past results, they were going to disadvantageously affect poorer students. And of course, if you use the previous results of the school, which was the other basis of the process, then schools that were improving from a weak position or individual student in a school which generally got poor results, were going to be punished. So it wasn't that the moderation process was the problem. I think most people seeing those criteria who knew about the education system would have spotted quite quickly that it was likely to produce distorted results. So, there were whole colleges, for example I know one in the West Midlands district, that got their worst results ever. So certain schools maintained good results. Some even went up.

But larger exam centres, which even if they had good results in the past found them dramatically reduced. They found they had no top grades in certain departments where they’d had lots in the past. So, the algorithm was a blunt instrument. And what Ofqual could've done I gather and we understand from the previous head of the government’s schools inspector (Ofsted) Michael Wilshaw there are about 5,000 schools in Britain, but apparently 450 of them had over graded their students, had given unrealistic grades. Now, it would have been possible, surely, in five months or the four months after they'd had the centre assessed grades to get in touch with those institutions and say, can you justify these grades? They don't seem to be in line with last year's. Instead, they punish them all. Rather like the school teacher who puts children in detention randomly because somebody is whistling in the class. They may not get the person who's whistling, but they will punish lots of people who weren't. And, of course, whilst I'm joking, this meant that they the destinies of hundreds of thousands of young people were very dramatically and disadvantageously affected. I think what the other impression that came across from this – certainly to people who are studying or who work in education or who have children it was startling that the government did not realise this was likely to be the effect. And they did not seem to understand quickly the impact it had on state schools and large colleges. And so, this became to some extent, it became a plea on behalf of state education to say, look, you obviously weren't thinking what we this would do. 

Sputnik: Why, in your opinion, did not you resort to an alternative method of conducting the exam, for example, online?

Matt Cole: Bear in mind the numbers you are talking about. You're talking about half a million students.

Sputnik: So, it’s not technically possible?

Matt Cole: Yeah, the idea of having exams with integrity where you could check that students weren't working with assistance from other people would be very, very difficult. Again, particularly in those larger institutions, there are institutions where you have, you know, 500 people studying math, for example, being entered as exam entrants. Or 500 people studying English or psychology is often a very large entry. And even in minority subjects like, sociology or music, you will have dozens, if not hundreds of students having an exam that had any credibility in the circumstances of the COVID crisis would be very, very hard. Of course, there are some courses and some institutions where the universities already had some data. So, if you apply for a law, for example, you take the Law National Attitude test, if you apply for medicine, you take a test. If you apply for history at certain universities, you take their history aptitude test. So, universities did have some evidence to work with and that helped them to say, look, we can see this is not accurate. And Worcester College, Oxford, for example, decided even before the results had formally been announced, you might want to check this, they certainly announced very quickly that they were not going to use teachers’ predicted grades. They weren't going to look at the results because they could see they said one in 10 of their private school applicants had been downgraded. Three quarters of their state school applicants had been downgraded. And they said, look, we've interviewed these candidates across Oxford -- Cambridge, interview everybody. We've interviewed them, we've tested them. We know that they're good candidates.

© REUTERS / HENRY NICHOLLSPeople take part in a protest over the governments handling of exam results, outside Downing Street, amid the spread of coronavirus disease (COVID-19), in London, Britain, August 21, 2020.
UK Government Was Too Slow to Recognise Drawbacks of Grading Algorithm, Scholar Believes - Sputnik International
People take part in a protest over the governments handling of exam results, outside Downing Street, amid the spread of coronavirus disease (COVID-19), in London, Britain, August 21, 2020.

So, we don't care what the A-level results say. Now that takes us to the crisis question. It does raise a question about how A-levels are run and how they will be restored. There is both, you might call it a crisis, but it is also an opportunity to re-examine a level A-levels have been ever been around since the 1950s; they are 70-years-old. They were invented at a time when perhaps 15 percent of students took them. Now it's about 70 percent or 80 percent. They were taking those exams as a basis for applying to perhaps 20 universities. There are now 120 universities. And of course, it was in a world which is technologically primitive for today's working world. So, there have been various attempts to reform A-levels. I mean, they've been changed in each of the last four decades in the way they assess and the way they’re graded, and so on. This might be an opportunity for the exam boards and they effectively had their role suspended, they were sort of superseded by Ofqual. The exam boards that actually designed the exams when they're taken may see this as an opportunity to say, OK, we're going to gently revisit the format of A-levels. Maybe they might they reintroduce some coursework. Michael Gove when he was education secretary removed coursework from most of the A-levels and GCSE’s. Now, had that data been available it would have been a very useful measure of those students. It would have been a real form of assessment.

So, it may prompt another revisiting of the format of A-levels. It may prompt a determination to make sure that they are not as generous in their grading next year as they were this year. Although, of course, next year students will say, well, we're competing with some of last year’s students to get into university, so it shouldn't be any harder for us than it was for them. So, yes, it is both a crisis and an opportunity. I can't say I have enormous confidence in the political system’s ability to improve qualifications at short notice. And clearly, they have a number of other challenges at the moment, including COVID and Brexit. So, I'm not optimistic about an immediate improvement in the that way A-levels work. I do hope there'll be a stabilisation in them next year. And I do hope that lessons will be learned about the process of standardisation. I mean, it has to be said, even when there are real exams the results are standardised to make sure that different markers and I’ve marked A-level papers and you have to go through a process of standardisation with colleagues afterwards. And sometimes that involves applying statistical models. That's what people are calling an algorithm. So, that much is not entirely unusual, but of course in a normal year you have an exam paper to return to and in the end, the chief examiner can look at an exam paper and say, actually I think this is at this level and it's worth this grade, we don't have that this time.

But what I think this has revealed to some politicians and to some members of the public who aren't normally involved in the education system, it revealed quite how complex it is, how diverse the British education system is. And also, I think a number of people, quite rightly, have been impressed by the articulacy of students who’ve been talking about this. They've been impressed by the amount of work that teachers have put in to preparing the students and to trying to make reasonable assessments. And they've been impressed that some people had already spotted this problem before the government. I think that's the question mark that will always hang over people's heads. Now we move on to the challenge of reopening schools. And this again, this could go either way, but it might make the public more curious about the opinions of people who work in education, about whether it's safe to return to school. It might make the government more determined to have their way on this occasion. 

To participate in the discussion
log in or register
Заголовок открываемого материала