The University of Bath is among the pioneers in digital assessment practice in the UK Higher Education. The collaboration with Inspera began in December 2017 as a trial at the School of Management and was then followed by a one-year digital assessment pilot in 2019.

The first trial involved 70 masters students and took the form of an essay-based formative exam, with Bring Your Own Device (BYOD) mode of delivery. Students were given an option to choose whether they want to hand in their exam digitally or on paper, with 70% opting in for the digital exam.

Dr Donald Lancaster, Teaching Fellow in Marketing and Director of Studies for Executive MBA at Bath University’s School of Management, was the course leader of the exam:

“From an academic’s perspective, my marking was much quicker with digital exams, so I was satisfied. Some students didn’t want to do it online simply because they were nervous. Talking to them afterwards to get informal feedback, all of the students who did it online, reported great satisfaction.

All of the students who didn’t do it online, later said they wish they had done it. That is really interesting when you think about it,” Dr Lancaster reflects. 

Objectives and benefits of the pilot project

Following the success of that first trial, the collaboration between the University of Bath and Inspera continued as a one-year project in 2018/19. The overall aim of the project was to trial and evaluate the digital examination solution across different University departments, including the support requirements, user experience, and the capabilities of technology, in a variety of examination conditions.

To achieve the overall aim of the project, the objectives were set out as below:

  • Testing the system in terms of coping with larger student cohorts;
  • Testing different modes of delivery infrastructure (BYOD; on-site; home exams);
  • Testing the impact of various different question types on authoring, marking, and student experience;
  • Testing different modes of access to online resources (closed book, open book);
  • Testing online exams in formative and summative assessment scenarios;
  • Testing online exams across different faculties and in a variety of subjects, on both graduate and postgraduate levels.
  • Testing the possibilities of the hybrid digital solution with scanning papers (Inspera Scan).

The main purpose of the one-year trial was to investigate whether e-examinations would bring any of the below key benefits for students, academics and the administration.

Benefits for Students:

  • Aligning exam conditions with the everyday conditions students face in the classroom (e.g. submitting coursework in the VLE, taking notes) and later in the workplace;
  • More accessibility for students with learning support needs

Benefits for Academics:

  • Increasing marking efficiency;
  • Improving fairness of the assessment by removing the bias caused by the students’ illegible handwriting.

Benefits for Administration:

  • More time-efficient operations;
  • More cost-effective operations.

A successful exam day

Both summative assessments and formative assessments were conducted within this pilot project. The question types used included multiple-choice, multiple response, drag-n-drop, missing words, click select, graphic gap match, graphic text entry, and text area.

The University of Bath campusBYOD and digital assessment at The University of Bath

Left: The university’s main campus, Claverton Down, in Bath, United Kingdom; right: one of the classrooms equipped for Bring Your Own Device (BYOD) exams in Inspera Assessment in May 2019.

First-year bachelor students attending the course Principles of Marketing had a choice of how they wanted to take the exam: 129 opted for a fully-digital exam, 60 opted for a hybrid solution where one part was handed in either for scanning (e.g. drawing as an attachment) or as written essay answers (50% of the exam was essay-based). The rest delivered fully-digitally. Only 2 students decided to submit fully on paper due to their typing speed concerns.

Scanning in Scantron solution performed well, and took about 15 minutes of administrator’s time for the entire exam, with only about 10% of the sketch sheets needing some manual adjustment (e.g. codes not shaded correctly). The exam was delivered in 6 locations, each with two invigilators and a support member of staff.

Positive student experience

The exams in the pilot were conducted without incidents, and “overall, it was a very positive experience”, reports Lancaster. Rowan Cranwell, Solutions and Project Manager, agrees and adds: “The majority of students with special needs and learning disabilities said they much preferred digital exams to pen and paper”.

Even though there was no formal evaluation of the student experience, they believe that the targeted benefits were felt among all of the relevant stakeholders, and hence, learning objectives of the pilot were achieved.

Lessons learned from the digital assessment pilot

Stakeholders at the University of Bath are satisfied with the outcomes of the pilot project, as it has confirmed the value of digital examinations, and met the key objectives in the form of numerous insights that emerged from the hands-on experience.

As Dr Donald Lancaster reflects: “Philosophically, everybody appreciates that this is the way the world is going and that there are many advantages of the system: in security, in authoring, in lower paper and production costs or costs to the environment, and with very few disadvantages.

We recognised the potential from the beginning, but actually going through the motions ourselves gives us a clearer idea of what is needed going forward.

Now we must take stock of what we’ve got, what we need to do, and how to best position ourselves if we want this to work on a larger scale.”

The collaboration with Inspera will continue in the future. “Eventually, digital exams will become the norm. I think it will happen; the only question is when”, as Lancaster concludes.

Want to know more about pilot projects?

A pilot project is often a valuable way to explore and prove the benefits of digital assessment at low risk and cost.

Get in touch with one of our education consultants if you want to learn more. Contact us. 

 

Written by Jo Bowden

January 14, 2020
Digital Transformation