On-Line Marking

The demise and downfall of ETS this summer has sadly led to the abandonment of KS3 tests, much to the delight, I've no doubt, of thousands of 14-year-olds, not to mention their schools and teachers.
Personally, I've never really understood what the schools and teachers had against KS3. From the pupils' point of view, they gave an indication of where they were in comparison to where they wanted to be in two years' time at GCSE. The scores achieved at KS3 were of no significance in themselves and were not recorded on UCAS forms, nor indeed did any future employer require, or want to see these results. Thus pupils received only a boost to their confidence or, possibly, a warning shot across the bows.
Schools, and all those who had access to the results, knew where the school was in relation to other schools. Many Heads, although they would not admit it, were not unduly worried if the KS3 scores were lower than might be expected. They could firstly blame the Primary schools, and, more importantly, in two years time they could claim credit for the "Value Added" when there was a vast improvement shown at GCSE. Whereas good KS3 scores meant less "Value Added".

However, I suppose I should stand by the title of my blog, even though I'm faced with a disappointing drop in income next summer. With that in mind, I have recently gone back to my ex-employer, Edexcel, and applied to do some on-line marking in February.



A few years ago I swore I would never again subject myself to the horrors of on-line marking. Those of my readers (are there any?) who have ever marked on-line will know what I mean.
For four years or so I tried to get into online marking but I'm afraid I found the whole process soulless, boring and irritating.
I missed the empathy one can achieve with a candidate when marking a complete script, some excellent and scoring very high marks, some with little aptitude for the subject but trying really hard to achieve something, and others whose despair overflowed into sad and desperate pleas to the marker.
Then there was the irritation of having to click on three blank pages and insert ten zeros in the appropriate boxes, when a candidate has omitted a complete question: or inserting ten ones when the candidate has scored full marks. When on a real paper a ringed nought or ten is sufficient.
And the deceit of the marker who, on coming across an "awkward" response requiring some thought in applying the mark scheme, simply logged off and marked a different question for a while, knowing that on return some one else would have taken the "awkward" question.
Then there was no longer a need to hold a standardisation meeting, with the chance to meet and interact with other markers, discuss the nuances of the mark scheme and meet one's team leader face to face.
I appreciate that the system is more efficient and less costly, but I for one was quite happy to add up marks and enter the total on the script and the mark sheets, and my checker enjoyed her role even though the board had not acknowledged the work of checkers for some years.
Thus, when I was offered some marking by another board involving "real" candidates submitting "real" scripts from "real" centres, I finally deserted Edexcel and have marked ALevels for Cambridge, winter and summer, ever since.
In fact, I am off to an Examiners' Meeting tomorrow and will spend the next six weeks or so considering the, often excellent, offerings of young mathematicians from Singapore.

Just heard I've been offered some marking on-line for February. Something to look forward to .... not!

Single in Sicily

Just got back from a week's package holiday in Sicily (naturally paid for with the fees received for my part in the downfall of ETS). This was the first holiday I have ever taken on my own and I am grateful to family and friends who encouraged me on this venture.



Having been on a number of these holidays with Voyages Jules Verne, I know that the great majority of holidaymakers are middle-aged or elderly couples, with only a few singles. These tend to be widows, widowers and divorcees, with a few younger people of either sex, often schoolteachers visiting places of historical interest. Mealtimes can be a problem, with most tables being for four, a pair of singles often get lumped together, sometimes for the whole week. Accordingly, I chose a hotel which only offered B&B and decided to fend for myself for lunch and supper at suitable bars and/or restaurants.



My daughter, a seasoned single traveller, offered some good advice. Always carry a newspaper or book and avoid eye contact if you wish to discourage the unwelcome attentions of strange men. I have to say that this ploy proved to be successful on every occasion.



There were four excursions on offer and I signed up for all of them. First we went to Syracuse, a sleepy port with a fine Greek Theatre nearby, at which, we were told, the first performance of Aeschylus' "Persians" was staged. Syracuse was also the town where Archimedes lived, and met his death at the hands of a Roman soldier. Apparently Archimedes was doing a maths problem in the sand on the beach (as one does) and failed to defer to the passing soldier quickly enough. I was sorry that there did not seem to be any statue or memorial to the great man.

The next day to the top of Etna .... or The Etna, as the locals call it. The coach took us to 1900m. and the remaining 1400m. were climbed by cable car and land rover, for the exhorbitant extra cost of 50 euros per person. There we saw a couple of smouldering craters, clearly on the point of erupting, and acres and acres of black ash.
There was only one place for lunch and the whole party went in so, apologising profusely, I plonked myself down at a table with a couple from my hotel. They were very nice, him particularly when we realised that the litre of wine provided with lunch was meant for four, so we did well. Conversation was easy and they weren't too nosey. They reminded me very much of Howard and Hilda, characters from the excellent "Ever Decreasing Circles" with Richard Briers. Howard and Hilda were the couple next door who always wore matching jumpers and were endlessly enthusiastic and positive about life. At one point they were discussing their intended bus journey to Messina the next day, and worrying about the hills and sharp corners. She (Hilda) then said that it might be nice to go by train, it would be flat and fewer bends. "Howard" was clearly taken by this. "That is a very interesting suggestion, Hilda, and you may well be right. It certainly gives us another option to consider."

Next day to Agrigento to the Valley of the Temples where the impressive Greek temples, one in particular extraordinarily well-preserved, stood on the hillside overlooking the plain down to the sea. During this tour I became aware of another "single". She was a keen tourist and bustled about questioning the guide and chatting to others in the group. She looked a bit like Judi Dench but without the twinkle and was, I decided, a retired headmistress. Lunch was at a restaurant at the beach, and I thought about it, and then chickened out. Sandals off, I walked along the sand at the edge of the water for half an hour, had a quick beer in a bar and then walked back just in time to catch the coach.

The final excursion was a boat trip to the volcanic islamds of Lipari and Vulcano. "Judi Dench" was there again and lunch was in the only restaurant on the extremely smelly island of Vulcano. This time I plucked up courage, followed her in and suggested that, as we were the only singles, we might as well share a table rather than sit separately. She agreed and, for the first time in nearly fifty years, I found myself having a meal alone with a lady other than my dear wife.

It wasn't too bad! I didn't talk too much, we told each other about our families, she was a divorcee of 64 with a son of 26 who suffered from MS, she told me of the years she had spent in Washington when her ex- had been awarded a fellowship and said I really should go there. We shared a bottle of wine and the bill and went our separate ways, perhaps deliberately avoiding further contact for the rest of the day.
She came and sat next to me at the airport a couple of days later and we chatted about the holiday, and we waved vigorously across the baggage reclaim at Heathrow. It probably meant little to her but, for me it was something of a milestone.

ETS Fiasco

Well, I have to admit that the mess that ETS have got themselves into is far greater than I had predicted or expected. You only have to read the non-sensational parts of the national press to be aware of their (mainly self-inflicted) problems.
Quite apart from the all the troubles they had recruiting markers, arranging meetings and setting up the computer to receive marks, there are three big mistakes which (from my personal experience as a KS3 Maths marker) could have been avoided, and have contributed significantly to their present embarrassment.

First, they should have had all schools send the scripts direct to the markers instead of to some distribution centre. Markers immediately lost at least a week's, and in many cases a lot longer, marking time meaning that many markers were unable to complete their allocation before the deadline. I received many emails from ETS asking if I could mark extra papers. Normally I would have been able to but, having "lost" a week at the start of marking, I was then starting my Cambridge Overseas A-Level papers.

Second is the question of accuracy of the marking. As an experienced marker, I hope my errors are very few and far between. However not a single script that I marked was seen or checked by any other person and, thus, I received no feedback or comment on my performance. The only check on me was the periodic "benchmarking" procedure whereby I marked four or five sets of papers online. Presumably the number of errors I made was fewer than some minimum requirement and I was told to continue marking. But how many errors did I make, and what were they, and did I keep making the same errors throughout my marking? No one will know until the scripts are returned to the schools and eagle-eyed teachers and pupils scrutinise my work and appeal if a higher level has been missed because of an error by me. If I had known the marks required for each level I could have looked again at borderline cases, found any errors and, where appropriate, adjusted the award.

Third is the entering of the marks online, an excruciatingly boring and mind-numbing process, which took far longer than totalling the marks of three papers and adding the three together. Clearly the computer programme has been unable to cope with the volume of data and award the levels, a task previously undertaken by the individual markers and delivered to the schools on time.

So, what changes will there be next year? My guess is very few. Ed Balls will conduct his inquiry and he and ETS will announce that they have "listened and learned" and they will continue to "get on with the job."
But how many disillusioned markers will pack it in? Quite a few I expect, but the rest of us will carry on because we quite enjoy the marking and the money's useful.

KS3 Maths Completed

All the scripts were successfully marked a week or so ago, packed up in parcels, and returned to ETS after all the marks had been (tediously) recorded at the appropriate website. It took ages to record the marks because they didn't want just totals, they wanted every single mark for every single question. So 80 or so marks each for 450 candidates meant a long time sitting in front of the screen.


ETS had apologised for this and pointed out that markers were no longer required to fill in the marksheets and award the grades. We used to record the marks of each of the three papers and then the total mark. Towards the end of the marking period markers would be told the grade boundaries for each of the levels, and we would then translate the total mark into the corresponding grade and complete the marksheets ready for returning to the school with the scripts, while copies were sent to the Exam Board.


Obviously accuracy was all-important and great care had to be taken to avoid errors. Tricky when the marks needed for a grade 6, for instance, would differ depending on which Tier the candidate had taken, 4-6, 5-7 or 6-8. Any errors would be quickly spotted by the school and/or the candidate and the scripts returned to the board for re-consideration.


For this reason markers were required to carry out what was known as a "borderlining exercise" for any candidate who had just missed a higher grade by 1 or 2 marks. Usually the marking remained unaltered but occasionally an error was found, or a particularly harsh judgement was rescinded, and the candidate could be bumped up to the higher grade. Schools would seldom comment on errors in the candidate's favour, or would bother if the level awarded was unaltered, but would scrutinise very closely the scripts of candidates just below a grade boundary. Thus, borderlining revealed any error before the school found it and saved everyone the hassle of appeals.



This year the grades will have been awarded automatically, and sent to the schools with no borderlining having taken place. Nowadays all schools seem to be very conscious of their level of performance in public exams and their positions in the (so-called) league tables, and I would guess that, when the schools have looked carefully at the results this week, ETS will be flooded with appeals on behalf of candidates just below the grade levels. I hope they're ready for it.

I am now getting stuck in to my next load of marking. (Too many exams!) This is a Cambridge International A-Level paper. About 500 overseas candidates tackling the sort of paper we used set for A-Level Mathematics about thirty years ago in this country. Very few of our today's Maths A-Level candidates would fancy either the style or the content of this paper.

Team Leaders

In a previous post I suggested that team leaders were important people whom one should keep on the right side of. (Sorry ...... on the right side of whom one should keep ...)
Well it seems that I was wrong. Not that I have anything against my current team leader, I'm sure he's an excellent fellow. It's just that, apart from one email asking if I was ok, I haven't heard from him.

In previous years (I almost said "in the old days") after the standardisation meeting one sent 10 sets of scripts off to the team leader for checking. After a cheery phone call telling you how well you'd done and how few errors you had made, the scripts were returned with corrections, explanations and hints, and you were cleared to continue marking.
This year a standardisation exercise had to be completed online and a banner came up onscreen saying "you have successfully completed the standardisation and you may continue marking".
No "well done", no info. about any errors, careless, or of interpretation of the mark scheme. Just "carry on".

The next sample, in the old days, (whoops) had to be sent to the team leader when you were about a third of the way through the marking. Again, the feedback from the team leader could be very useful. Perhaps you were being particularly harsh, or lenient, over a certain question. Perhaps you were making an error of interpretation and had to go through all the scripts already marked and make alterations. In any event the personal contact was valuable.

It doesn't happen this year under ETS (Europe). (Actually I think they are also supplying my gas and electricity at ever more exhorbitant prices)
This year we have a procedure called "benchmarking", to be undergone online every time you have successfully submitted the complete sets of marks for 80 candidates. This causes a problem straight away because one doesn't mark complete sets. Each candidate sits 3 papers and it is much more convenient to mark batches of one paper before moving on to another. I have, in fact, marked all my quota of 450 or so Mental Tests already.
Incidentally, it takes longer to enter the marks for a Mental Test online, than it does to actually mark the paper. ETS have recognised this and have doubled our admin. payments, which is something out of the wreck.
Anyway, once you have finished the benchmarking exercise, up comes the banner again. "You have successfully completed the benchmarking, you may continue marking".
Again, no indication of the number or nature of any errors so you are none the wiser.

Still, back to the ever-diminishing piles of scripts. It would appear that, according to the emails currently flying around, there are still some markers who have yet to receive their scripts!
They have my sympathy .......... see you next year?

Markers' Meetings

Still no sign of any scripts but the good news is that the ETS Parcel Tracking system tells me that 6 parcels are "currently in transit" and the remaining 8 are "to be collected".


The markers' meeting last week was attended by all the usual suspects. Retired teachers supplementing their meagre pensions, newly qualified teachers (some of whom looked about 14) supplementing their meagre salaries, pregnant women on maternity leave and, as ever, more females than males.

There are always those (generally new) markers who regard this meeting as a chance for them to provide their own input to the mark scheme.
"I think we should allow this ...."
"What if they've written that ..."

They don't seem to realise that no Chief Examiner is ever going to change the mark scheme at their suggestion and the sole purpose of the meeting is to ensure, as far as possible, that all markers mark consistently.

Markers are divided into teams of about ten, each under a team leader who will monitor your progress throughout the marking period. It's best to keep on the right side of your team leader for it is he or she who will grade your overall performance and possibly recommend you for team leader next year.

One year I had a team leader who was very keen on team spirit and identity, and she insisted that we should each introduce ourselves so we could bond more easily.
"I'm Sally, I have Annabel who is nearly 4 and has just started nursery, Josh who's just learned to walk and is into everything, and I'm expecting another in 3 months. This is my first time marking cos I want to keep my hand in ready for when I go back."
"I'm pascal, my eldest is 43 and works very hard, as does my middle one who is 38. My youngest is 34 and she's a bit of a wanderer. I've been marking every year since 1973 and I do it for the money as it helps pay for my holidays."

This year's meeting was held in a very posh hotel, the food was excellent, the work space was inadequate, the chief examiner was efficient and clear and the admin was organised by two charming American girls in neat blouses and skirts. They were presumably employees of ETS, had very strong accents, spoke very quickly and, thus, were virtually incomprehensible.

Still, we got away by 5.00pm and I was in the garden enjoying a beer at 6.30!

KS3 Mathematics

Last week's tests have been completed by thousands of children. On Saturday I attended the Markers' Meeting where we went through the mark scheme in detail and completed some training scripts. Most of Sunday was spent completing the standardisation excercise online, (apart from a couple of hours off watching Manchester United clinch the Premiership title .... well done Giggsy!) and it was pleasing to be told "you have successfully passed standardisation" and may commence marking.

So, the past three days have been spent getting stuck in, and making inroads into the mountain of scripts on the floor of the spare bedroom. I wish ......
The trouble is, there aren't any scripts. They haven't arrived yet. They probably haven't even been dispatched yet.

I have been marking KS3 Maths every year since it began, 1993 or 1994 I think. Over the years the Tests have been delivered by a number of agencies, Edexcel (twice) and AQA, and this year the organisers are someone called ETS Europe, a division of ETS Global BV.
In every other year the schools sent the scripts directly to the marker and, as soon as the marker had completed the standardisation exercise, marked the sample scripts and had them approved by the Team Leader, he/she could get stuck in to the marking.
This year, in their wisdom, ETS decided to have all the scripts sent to a central point (in Hartlepool I believe) from whence they would distribute them to the markers. Someone has been spending the last few days sorting through in excess of 100,000 scripts and (hopefully) started to send them out.

At lunchtime today I'm off to look after the grandchildren for a couple of days. I was hoping to take lots of marking to do while the children are at school but the chances of any scripts arriving before I leave are fairly slim.

Never mind, it says in the handbook "scripts should arrive in three working days and certainly before May 27th."