The first trial involved 70 masters students and took the form of an essay-based formative exam, with Bring Your Own Device (BYOD) mode of delivery. Students were given an option to choose whether they want to hand in their exam digitally or on paper, with 70% opting in for the digital exam.
Dr Donald Lancaster, Teaching Fellow in Marketing and Director of Studies for Executive MBA at Bath University’s School of Management, was the course leader of the exam:
“From an academic’s perspective, my marking was much quicker with digital exams, so I was satisfied. Some students didn’t want to do it online simply because they were nervous. Talking to them afterwards to get informal feedback, all of the students who did it online, reported great satisfaction.
All of the students who didn’t do it online, later said they wish they had done it. That is really interesting when you think about it,” Dr Lancaster reflects.
Following the success of that first trial, the collaboration between the University of Bath and Inspera continued as a one-year project in 2018/19. The overall aim of the project was to trial and evaluate the digital examination solution across different University departments, including the support requirements, user experience, and the capabilities of technology, in a variety of examination conditions.
To achieve the overall aim of the project, the objectives were set out as below:
The main purpose of the one-year trial was to investigate whether e-examinations would bring any of the below key benefits for students, academics and the administration.
Benefits for Students:
Benefits for Academics:
Benefits for Administration:
Both summative assessments and formative assessments were conducted within this pilot project. The question types used included multiple-choice, multiple response, drag-n-drop, missing words, click select, graphic gap match, graphic text entry, and text area.
Left: The university’s main campus, Claverton Down, in Bath, United Kingdom; right: one of the classrooms equipped for Bring Your Own Device (BYOD) exams in Inspera Assessment in May 2019.
First-year bachelor students attending the course Principles of Marketing had a choice of how they wanted to take the exam: 129 opted for a fully-digital exam, 60 opted for a hybrid solution where one part was handed in either for scanning (e.g. drawing as an attachment) or as written essay answers (50% of the exam was essay-based). The rest delivered fully-digitally. Only 2 students decided to submit fully on paper due to their typing speed concerns.
Scanning in Scantron solution performed well, and took about 15 minutes of administrator’s time for the entire exam, with only about 10% of the sketch sheets needing some manual adjustment (e.g. codes not shaded correctly). The exam was delivered in 6 locations, each with two invigilators and a support member of staff.
The exams in the pilot were conducted without incidents, and “overall, it was a very positive experience”, reports Lancaster. Rowan Cranwell, Solutions and Project Manager, agrees and adds: “The majority of students with special needs and learning disabilities said they much preferred digital exams to pen and paper”.
Even though there was no formal evaluation of the student experience, they believe that the targeted benefits were felt among all of the relevant stakeholders, and hence, learning objectives of the pilot were achieved.
Stakeholders at the University of Bath are satisfied with the outcomes of the pilot project, as it has confirmed the value of digital examinations, and met the key objectives in the form of numerous insights that emerged from the hands-on experience.
As Dr Donald Lancaster reflects: “Philosophically, everybody appreciates that this is the way the world is going and that there are many advantages of the system: in security, in authoring, in lower paper and production costs or costs to the environment, and with very few disadvantages.
We recognised the potential from the beginning, but actually going through the motions ourselves gives us a clearer idea of what is needed going forward.
Now we must take stock of what we’ve got, what we need to do, and how to best position ourselves if we want this to work on a larger scale.”
The collaboration with Inspera will continue in the future. “Eventually, digital exams will become the norm. I think it will happen; the only question is when”, as Lancaster concludes.
A pilot project is often a valuable way to explore and prove the benefits of digital assessment at low risk and cost.
Get in touch with one of our education consultants if you want to learn more. Contact us.