Q&A: How to create better exams

What are the best strategies for designing effective online assessment? How the coronavirus pandemic accelerates the digital transformation, and how to respond to the change? If you missed our Q&A Coffee Break about how to create better exams, don't worry. In this article, established experts from Higher Education answer some of these burning questions from the audience.

The panellists:

Mary Richardson

UCL Institute of Education

Magnus Svendsen Nerheim

University of Bergen

Henriette Godal

Inspera

Q1: How can we mitigate concerns about high-stakes e-assessment validity? Are there any tools to ensure that students do not cheat?

Mary Richardson: There is a lot of research now regarding the validity of e-assessment and the same 'rules' apply for electronic as paper-based assessments. Is it appropriate, is it fit for purpose and does it give us the evidence we are looking for? Basically put, if you can answer yes to those three principals, then you can be reasonably sure of the validity. Of course, you can create a huge range of assessment types, some of which can be well standardised and others which are more challenging. It is about setting systems in place to check, recheck and reflect on practice. Assessment, like all aspects of education, doesn't stand still and we have to acknowledge that - we have to keep changing to keep up! There are lots of ways to discourage cheating, but I'll say again, it's best to start from a place of trust and confidence rather than assuming distrust. Levels of cheating increase the higher the stakes you place on a test, so think about the message you promote along with your assessment!

Q2: What strategies are out there for preventing 'sharing among students" during an open-book exam, when student may use any available and accessible resources?

Magnus Svendsen Nerheim: We recommend to design the assignments in such a way that students will not benefit from communicating with each other or using resources on the Internet, or in such a way that the students are challenged to use external resources or collaboration to solve the task. Ultimately, this boils down to assessing whether or not the students have achived the learning outcomes. If the only way to assess this is a 'locked down' written exam, then write several tasks assessing the learning outcome(s) and have the students pull X tasks from Y pool per learning outcome. If it can be done with any other form of assessment, do it. UiB resource page on assessment and Covid is available .

Q3: Is there research on typing speeds compared to handwriting speeds?

Mary Richardson: There is very little on this - generally, one of the issues is that we (humans) tend to estimate our answers as more accurate when they are typed - it's a neatness effect!

Q4: Can we consider that a well-done exam should contribute to a good performance in its execution and to obtain a good final result?

Mary Richardson: Overall, it's what we'd expect theoretically. Good design will discriminate well between candidates, will offer a range of opportunities for all test takers.

Q5: What are some of the best question types to use?

Magnus Svendsen Nerheim: Whatever makes sense for the specific course/field/topic/learning outcome. Use auto-graded (in essence pre-graded) wherever feasible.

Q6: What is your view on Programmatic Assessment, which now appears to be the future of testing if one does not want to use proctoring?

Mary Richardson: I'm a bit cautious about whether Programmatic Assessment is the future of assessment - to me, it looks very like the early cycles of assessment/feedback as devised by theorists such as Crooks, Kane and Cohen et al. I don't' think this mode of assessment is necessarily rejecting proctoring - I'd need to know more about the type of assessment to make a more considered judgement.

Q7: For someone just getting started with creating online exams, what practical tips would you give to make the most of Inspera Assessment?

Henriette Godal: I’d suggest you learn how to author one question type first. All question types are structured similarly from the author’s point of view, so understanding how to create one question type will get you a long way in creating all!