I don’t know about you, by my least favorite job as a professor is marking exams. I’m always keen to give students as much credit as possible for their knowledge, but sometimes finding a correct answer can be like finding a needle in a haystack. Consequently, it’s wildly time consuming, not to mention discouraging and frustrating. Setting an effective exam can be a challenge, as well – it’s not too informative if the exam is so easy that everyone aces it. On the other hand, if everyone fails, you’ll be left wondering whether you’re a terrible teacher, they’re terrible students, or it’s just a terrible exam.
Over the past 23 years I’ve given my share of all types of exams – from WAY too easy to brutally difficult, and everything in between. I’ve learned to do some things well and managed to do some things terribly wrong. I’ve kept lots of notes on what worked and what didn’t, and I thought I’d come up with some pretty good methods. However, I recently attended a seminar on setting effective exams and I got some new and cool ideas from that, as well. So I guess that means I’ll be learning new things about this until I retire. Nevertheless, I think I might be able to save you some time and energy by telling you what I’ve learned so far.
First, let me tell you what my goals are in setting exams – since these are the driving factors in how I set them. If you have more, or different, goals then keep that in mind as you look at my suggestions later on. I generally have three goals in setting exams.
- Get an accurate assessment of how much each student actually knows.
- Make the grading entirely objective so that I can mark all students consistently.
- Minimize the time wasted while marking the exam.
I’m sure we’d all agree that item 1 is the most important in this list – however, I generally focus a lot more on items 2 and 3. The reason is that I discovered early on that if I got ‘cute’ at all in setting exam questions – no one would get any them. I’d just have a lot of blank sheets to mark. Although the seminar I went to recently was geared primarily towards designing effective multiple choice questions (which I cannot use in my exams) it still gave me some great ideas on how to improve my ‘problem-solving’ exams to ensure that they were also effective in assessing students accurately. Apparently the trick is to have a mix of questions types – straightforward, moderate, and difficult – which is obvious, I know, but the important thing is to then check and make sure there is consistency in terms of who does well on each type. For example, those who did the best overall on the exam should also be the ones who did the best on the difficult questions, and so on down the line. This is obviously going to be a trial and error process which requires some data gathering to document how students do on the individual questions, not just on the exam overall, something I’ll admit I’ve never thought to do! Nevertheless, if you can document this type of information for each category of question, you can begin to see what’s working and what isn’t, and use this info to design more effective exam questions.
As it turns out, goals 2 and 3 are actually the easy ones to achieve; it just means spending time up front in setting the exam. I can set an exam in an hour or less, but it’s going to take me 30 to 45 minutes to mark each one after. In a class of 80 to 100 students, that’s one to two weeks of marking afterwards! Also, chances are that I’ll not be consistent in grading those exams – no matter how much time I spend on it. In contrast, if I spend one day setting the exam, including developing a marking key to assign specific marks to each of the intermediate answers, then I can mark each student’s exam in about 5 to 10 minutes (which means only about 1 to 2 days of marking). At this point, the main thing eating up my time is shuffling through the booklets trying to find the actual answers. I learned early on not to use question sheets and blank exam booklets – the students tend to attack the questions in what seems (to me) to be an entirely random order. It’s absolutely insane how much time you can waste flipping back and forth through a booklet looking for answers when they’ve got them all mixed up. Some even split up their answers! I can control that a bit by making my own exam booklets, so that the students actually write right on the question sheets. However, although they’ve been taught from first year to document their work clearly and neatly, what I see on most exams presents a challenge comparable to deciphering hieroglyphics. Finding the actual answers amidst all of the extraneous meanderings can take forever!
Here’s where that recent seminar gave me a great idea. For my next exam I am going to include a summary sheet at the front with boxes for all answers. The students will be required to enter their final answers to all parts of the question in these boxes (both numbers and words). For the students who get the question right, I will not have to look any further than this summary sheet. For those who do not, I can investigate the details inside; however, I will warn them that the only way to get full marks for a question is to have the correct answers on the summary sheet. I expect that I should be able to cut my marking time in half by employing this “summary sheet” and it will help me to facilitate my documentation of how students do on the individual questions more readily, as well. (I’ll let you know if it really pans out that way. ;-))
I am also one of those professors who likes to give students ‘part-marks’ – especially if they make a little boo-boo at the start of a question that gets carried through. I always develop a spreadsheet solution for each question so that I can quickly enter any incorrect intermediate answer and see if they calculated everything else correctly. Again, time spent up front helps me to save time while marking, while still giving students the fairest assessment possible.
Well – that’s it – my suggestion for setting exams to achieve the three goals I’ve listed above. Hope you found a tidbit or two in there to help you out. If you have some suggestions or feedback to offer – why not use the comment feature below to share them?
Thanks for visiting my blog!
Update – Dec. 19, 2013 – Click here to see my post on how this solution box idea worked out.