Although I'm sure that everyone here has experienced an examination from the receiving end, many of you have probably not had the experience of giving an examination.
Here, for your edification and enjoyment, are some of my experiences while preparing, giving, marking and returning examinations while lecturing in Computing Science
at the University of Alberta
in the early 1980s.
The two styles of exams that I'm familiar with are the ever popular multiple-guess
exam and the more traditional written answer
The pros and cons (from the instructor
's perspective) are roughly as follows:
giving the exam (i.e. the actual one-to-many torture session which is
traditionally held in a noisy room with poor lighting and no air conditioning):
- preparing the exam:
- a multiple choice exam is considerably more work to create
as each question must be carefully crafted to provide a set of
reasonable alternative answers and it's generally considered "bad form"
if there is either no correct answer or more than one
- putting aside the basic mechanics of creating a multiple choice
exam, designing a multiple choice exam which properly evaluates what
the student has learned is also a lot of work.
- in contrast, crafting good questions for a written answer style
exam is easier and considerably less error prone.
I'd even go so far as to say that some areas of knowledge can't really
be properly evaluated using a multiple choice exam (e.g. "describe an
algorithm to do blah" explores abilities and concepts which are really REALLY difficult to get
into with a multiple choice exam).
- fortunately, there is little difference between the two styles
here although if cheating is a substantial concern then it is generally easier
to create dozens of different versions of a multiple choice exam
(by just permuting the questions) than it is to create even a
handful of equivalent but different written answer style exams.
marking the exam:
returning the exam:
- multiple choice exams are much easier and faster to mark than
written answer exams (this can be a significant factor if there are 500
students who will be taking the exam)
- in contrast, a written answer style exam is harder and slower
to mark (see previous point about 500 students)
- if the question and answer alternatives are properly crafted, a
multiple choice exam's final score isn't open to debate (see previous
point about 500 students)
- the answers to even the most perfectly crafted written answer
exam are always open to debate (see previous point about 500
In my mind, the written answer
style exam wins hands-down.
I'll trade all of the advantages of the multiple choice
style exam for the greater ease (i.e. improved likelihood) of creating a good written answer
Maybe this is the result of some experiences with poorly crafted multiple guess
exams (both from the giving and the receiving side) but that's where I stand.
Has anyone ever not
experienced the fear
of sleeping through an exam that they are scheduled to take at 0800 the next morning?
O.K. now compare that fear with the nightmare
of sleeping through an exam that one is supposed to give at 0800 the next morning!
How many people are impacted by the broken alarm clock in each case?
How likely is it that it will be possible for the sleepy student to take a makeup exam at a later date?
How likely is it that it will be possible to get the 150 students together at a later date to provide them with the opportunity to take the exam that the sleepy instructor (i.e. YOU
Need I say more?
Speaking of excuses
My wife once had a student phone her to say that he was going to miss the exam that was being held that day.
When my wife asked why, he said that he been arrested and was in jail.
My wife considered this to be an acceptable reason to miss the exam and scheduled a makeup exam for later.
His encounter with the authorities had to do with some unpaid fines - i.e. it wasn't all that serious in the grand scheme of things but he was NOT going to be making it to the exam that day!
for the moment (were those last two sections really
We all know how important it is to be relaxed for an exam.
I once had a professor, Barry J. Mailloux, who handed out lollipops during a final exam.
Later, when my turn to inflict pain came up, I followed in Barry's footsteps.
I'd hand out lollipops, mints, licorice, whatever seemed appropriate.
I found that this seemed to help to relax the students and a lot of students thanked me for the treat later so I know that it was appreciated.
On one occasion, I was giving an exam to a class of about 150 students.
I'd booked the exam into the University's main gym and my students occupied every other row for about a half dozen rows.
About half way through the exam, I walked up and down each row and (very quietly) gave each of my students a candy.
About half way down the second or third row, I happened to look at the students in the rows between my students.
Trust me . . . if looks could kill then I'd have been dead already.
Apparently, their instructor had forgotten to bring candies for them!
Some students preferred final exams in the gym whereas others preferred exams in the regular classroom.
I'd generally use the gym for big classes (over about twenty students) and the classroom for the rest.
The Royal Order of the Chocolate Chip
In one course that I taught a number of times, I instituted a different tradition.
The course was an honours version of an introduction to computer architecture
course which generally had about a dozen or so students.
Immediately before the final exam time period
started and before I'd handed out the examination itself, I'd start the ceremony
(the students didn't know that there was a ceremony coming):
I'd then tap each student on the head or the shoulder with a ruler or something and that was it.
- I'd put on a strange hat (no, much stranger than that).
This would definitely get everyone's attention!
- I'd then call up each student by name and induct each of them into the Royal Order of the Chocolate Chip.
The investiture procedure consisted of giving each student:
- a chocolate chip cookie
- a computer chip (i.e. packaged integrated circuit)
- a scroll with their name on it to serve as proof that they
were now official members of the Royal Order of the Chocolate Chip
Once the laughter died down, I'd hand out the examination papers and get on with the business
It's been over twenty years since the last time I ran this ceremony yet I still run into old students from time to time who mention it.
Marking the exams
I'm sorry but I've been trying for years to forget the pain of this phase of the process . . .
What I will say is that, contrary to popular belief, the following weighted marking system was rarely used:
- stand at the top of a set of stairs with the student's papers
- throw the papers down the staircase
- assign marks based on how far each paper falls
- bonus marks can be awarded for style if a paper falls in a particularly graceful way
Drawing the line
Once the exams were marked, it was time to assign final grades.
No process within the university context has more mystery and symbolism associated with it than this one.
There are also few aspects or moments in a student's university career that are more important and they aren't even present
Quite literally, the future of each student could be determined by what grade they are assigned in your course.
The University's grading scheme when I was lecturing there was called the stanine system.
Each student would be assigned a grade between 1 and 9 (higher is better) for each course.
Each instructor would be expected to distribute their grades across a bell curve and the average grade was supposed to be about 6.
This may sound arbitrary, evil, unfair and just plain wrong but, in practice, the final marks in a large course would look like a bell curve and there was little if any fudging required to make things fit.
Here's what I'd do (I'm sure that the relevant statute of limitations has
expired by now):
- compute the final mark in each course (see the YAMP discussion in my Snobol writeup for a "story" related to this step)
- sort the students by decreasing order of final mark
- decide whether or not the class did well or not as a whole in the grand scheme of things (this would determine the average and distribution of the grades that I was about to hand out)
- look down the list of grades looking for gaps that could be used as boundaries between zones of roughly equivalent marks (a well placed gap could save a lot of time in the fallout stage - see below)
- assign the same grade to each student in each zone
In practice, it was never that simple.
For example, if there wasn't a gap near where I wanted to insert a boundary
then I'd insert the boundary where I felt it belonged (unlike one instructor that I knew, I wouldn't fudge a few marks to create a gap).
The important goal was to ensure that each student's grade accurately reflected how well they had learned the material presented in the course.
In comparison, the University's policy about how grades should fit curves and such was never all that important to me.
Quick definitions: a mark is what you get for an assignment, a paper, a project or an exam.
A grade is what you get for a course.
Dealing with the fallout
Once the grades were finalized, I'd post them along with the student's marks outside my office door.
I'd then wait for what happened next - students dropping by my office looking to see if there was any way that their grade could be increased.
My policy was very simple:
The line between each grade level was EXACTLY at the point where the student with the poorest mark at the grade level was at (e.g. if the student with the poorest mark who got an 8 got a mark of 85.6% then that was the line for an 8).
In order to raise their grade, the student had to find a large enough error in the marks during the term or on the final exam to increase their mark to be equal to or greater than the line for the grade that they wanted.
The result of this policy was that students whose mark was just below a line would have a strong incentive to try to get their grade raised by finding errors.
Students with marks much below the lines didn't bother unless they had reason to believe that a fairly substantial error had occurred.
I've heard of instructors who refused to even consider changing a grade once it was posted.
This never struck me as fair so I'd allow a reasonable amount of time for students to report errors to me.
The process was NOT much fun for me or the students but I felt that it was necessary in order to be fair.
Let's make a deal
Okay. Before we get started on this one, I need to make one thing PERFECTLY clear:
What I'm about to describe REALLY happened.
I'm sitting in my office one day about a week after the grades in my classes for the term had been posted.
Two students drop by and one of them starts talking.
Her friend, the other student, had failed one of my courses that term (she'd gotten a grade of 3 and needed a 4 to pass).
The person talking had passed the same course by a fairly wide margin (she'd gotten a 7).
She wanted to know if she could give one of her grade levels to her friend.
i.e. she was prepared to accept a grade of 6 in exchange for her friend getting a grade of 4.
I listened and then sat for a moment in the silence which followed.
I then asked her to explain her proposal to me again (I wanted to be sure that I'd just heard what I thought that I'd just heard).
She explained it again.
I sat for another moment or three of silence wondering exactly what to say.
I then said (in terms and in a manner which was, shall we say, clear and unmistakable) something along the lines of
I went on to explain that grades were a personal measure of how well someone had done.
They weren't something which could be traded or gifted to anyone.
They left my office.
I then just sat there for a while.
I made no attempt to find out who the two students were as it was pretty clear from their body language that they really didn't expect their proposal to be accepted.
It was a friend trying to be a friend to a friend in need.
P.S. I had a policy of failing students who were caught cheating.
This policy was clearly explained at the start of each course and I applied the policy on more than one occasion. The lowest grade that I ever assigned was a 1 and I only ever assigned one of them in the three and a half years that I lectured at the University. It was given to someone who had been dishonest in a very fundamental and corrosive way. I also tried (and failed for a really stupid reason) to get the student thrown out of the University. The grade of 1 did effectively end the student's chances of ever getting a degree in Computing Science so I suppose that I did achieve my goal in a manner of speaking. I'm not sure if I'm ready to say more about this event in public yet. That said, let me assure you that I still believe that being thrown out of University would have been a most appropriate punishment for the acts in question.
It's YOUR fault
Students fail courses sometimes.
It's a fact of university life that will never change.
I understand that it is pretty traumatic but I believed (and still believe) in handing out grades that each student deserves.
From time to time, a student would drop by my office to see if they could talk me into raising their grade.
My policy (see above) was clear - if you want your grade raised then find an error that is large enough to raise your final mark above the line in question.
There was absolutely no way that I was going to raise a grade for any other reason.
On a few occasions, when appealing to my sympathy hadn't worked, the student would then try working on my guilt.
I remember one student who tried to argue that I was personally responsible for them having to drop out of university.
It seemed that their grade point average for the year was too low and their department was kicking them out.
After listening patiently for a short while, I explained that the student's problem was that their grade point average was too low.
The grade that they had received in my course was one of ten grades that they'd received that year and suggesting that MY grade was the one that did them in was nonsense.
The student left.
I've no idea if I got the message across but I didn't raise the grade.
The only techniques that I would allow to work were ones which involved my sense of fairness.
I told more than one student that I always tried to be fair even if it meant that I couldn't be reasonable.
Studying for my exams
Let's end on a more pleasant note shall we?
I ran into one of my students one day in the halls somewhere.
It was the day before the final exam so I asked the student how things were.
The student replied that things were going pretty good.
He then suggested that I drop by the student lounge area later as they were going to be studying for my exam.
Since I was available to any student at any time that they could find me in the days leading up to an exam, this seemed like a reasonable request so I agreed (I made a point of being fairly easy to find at these times).
Later that day, I dropped by the lounge and found the student with a few of his friends sitting around a table with a box of pizza between them.
I sat down and quickly discovered that the topic under discussion had absolutely nothing to do with my course.
I (diplomatically) asked how their studying for my exam was coming along.
They said that it was going great and pointed at the pizza as evidence.
My puzzlement must have been obvious because one of the students then explained that they'd figured out a long time ago that there was simply no point in studying for one of my exams.
You either understood the concepts and did well or you didn't understand the concepts and were doomed.
I chuckled and then jumped into the discussion (and ate some of the pizza).
These students were studying for my exam in the best way that they knew how - by being relaxed prior to the exam.
I consider the statement that there was no point in studying for one of my exams to be one of the best compliments that I ever received as an instructor.
My exams were designed to determine if you understood what had been taught - i.e. lots of "why" questions and absolutely no "define XYZ" questions (good "why" questions make "define XYZ" questions unnecessary).
I wasn't interested in whether or not the students had good exam writing skills - I wanted to know if they had absorbed the material.
I really REALLY enjoyed my time lecturing at the University.
I met a number of (so far) life-long friends and wouldn't have missed it for the world.
Eventually, the time came to resume my career as a software developer so I moved on.
The rest of the experience was fantastic!
If you ever get the chance to lecture at the University or similarily advanced level then I strongly recommend that you accept.
It is an experience which is most definitely not to be missed!
Everything described in this writeup really happened.
There are no parables or other forms of "fiction".