Higher education institutions play a crucial role in the advancement of technology, including cutting edge research currently being conducted, which will fuel the Fourth Industrial Revolution for decades to come. Closer-to-home universities have often been accused of being slow to adopt new digital technology to deliver their teaching to students. In my experience over the last few years, I have seen this hypothesis challenged many times, with institutions increasingly looking towards digital tools to improve the experience of students and deliver learning in more innovative, accessible, and engaging ways.

At Inspera, I help institutions get to the heart of why they are undergoing digital assessment transformation. In doing so, I have developed expertise in helping them understand what they need to succeed. A key observation during these conversations is this: the importance of change management is not to be underestimated.

While choosing a technology platform that adheres to your technical needs might appear to be your main focus, to reach your long-term digital transformation goals, you must also ask yourself: how are you going to adopt the technology, who is going to use it and support it, and what training do you need to provide to ensure it is used effectively? Answering these questions will lead you on your way to choosing the best platform for both students and educators.

So, to help set you on the right path, I have summarised four considerations for success when undergoing a digital transformation project to enhance your assessment capabilities.

1. Understand your students

Universities often have a natural inclination to consult heavily with the staff using systems but spend less time engaging with students. As the ultimate end users of any digital assessment platform, it’s important to understand how students want to be assessed, what kind of technologies and systems they use as part of their teaching & learning, and what accessibility requirements they might have.

An institution’s overall goal in a digital assessment transformation is to increase efficiency for educators and improve student learning outcomes. So, take the time to ask your students how and where they want to be assessed and what types of adjustments for accessibility issues they might require. Whilst students may not understand all assessment modalities or the pedagogy behind them, they will have a sense of the sorts of digital tools they expect to use, e.g., a typed examination as opposed to handwritten. Moreover, students will also benefit from the flexibility of being able to sit an assessment at a time or location that suits them. Whilst this may not be viable for all assessments, asking the question of where a student produces their best work or where is most convenient for them is a useful tool in improving student assessment outcomes.

Assessment and feedback are frequently low-scoring segments of NSS scores for UK Higher Education, and, whilst I am not going to dissect why this is the case now, I do want to highlight that actively listening to the student voice in this regard can have a tangible benefit in the overall success and ranking of an institution.

2. Understand your institution

A common tendency I have experienced is that institutions dive straight into looking at high-level functional platform requirements before having first analysed the assessment and business context at your institution. Below, I have set out some key questions to ask, along with the important considerations needed before considering your digital assessment requirements.

Question Consideration
What assessment types do we carry out? Most institutions conduct a variety of assessment types across different subject areas and faculties. Common formats of assessment are examinations, coursework submissions (both written and digital artefacts), observational assessments, MCQ quizzes, and portfolio assessments.

Each assessment mode will have its own unique processes and characteristics, which in turn will lead to different functional and non-functional system requirements.
How am I regulated? Many higher education courses are regulated by professional, statutory and regulatory bodies (PSRBs). These PSRBs themselves may have stringent requirements around the type of assessment that is mandated in order for the course to be accredited. Make sure to consider these and other quality assurance regulations your institution follows.
Are there any other digital projects or programmes that need to be considered? Digital transformation takes time and resources to get right, and digital assessment is no different. It is important to understand the wider digital strategy of the university and, in particular, whether other projects will require overlapping resources in order to complete. One of the most common reasons for the failure of digital transformation projects is the failure to ringfence the resources and expertise required.
What IT and learning technology resources do we have? Institutions of different sizes have differing levels and models for the support of technology. Some institutions have centralised support teams, whereas some teams have support at the faculty, school, or college levels. Before embarking on requirements gathering, consider who will support the digital assessment platform once implemented.
What facilities do we have (e.g. IT hardware, exam halls, computer rooms)? To ensure that you are ready to embrace new technology, institutions need to consider the hardware and facilities you have now. Whilst it is possible to invest in new hardware as part of a digital transformation programme, this can only be done in the confines of the institutions’ physical infrastructure and financial resources. Herein, it is important to ensure that your digital assessment delivery strategy – be it remote or on-campus, BYOD vs. centrally managed devices – is considered.

3. Understand your requirements

The requirements can be complex when you adopt any new digital system, and best practice would have you split these into functional and non-functional. Further, many institutions I work with then look to break down the functional requirements into the different activities carried out along the assessment lifecycle: from authoring a test to delivering the test, through to invigilating, marking, and providing feedback.

In order to do this, you need to understand your assessment lifecycle. Whilst every institution I have ever worked with certainly does follow a generic overarching assessment process, no two institutions are exactly the same. Furthermore, many schools, colleges, or faculties within a single institution may have slightly (or in some cases substantially) different processes for authoring an assessment or marking a student submission. One question this raises is that of standardisation. Once you are aware of these differences, you need to decide whether your digital transformation will enforce a degree of standardisation, or allow differentiation across faculties. Many platforms will allow for flexibility within the processes you implement, but standardisation is as much an administrative and experiential question as it is a technology one.

With this in mind, you will then need to weigh up the requirements and prioritise those that are most important to you. What you are likely to find at this point is that certain assessments are conducted far more often than others at your institutions. For example, you may be an institution that has largely eliminated summative examinations and instead opt for coursework-style assessments. Alternatively, you may be regulated by PSRBs, who in turn have specific requirements to conduct high-stakes examinations under strictly monitored conditions. Understanding your balance of assessments will allow you to weight certain requirements that are more heavily favoured by the modality of assessment you will conduct most frequently.

Alongside prioritisation sits compromise. It is unlikely that any one single technology platform will ever deliver every requirement from each corner of your institution. Compromise is often seen as a dirty word, but is an essential part of change management theory. The key is ensuring that you are compromising for the right reasons and that you have mitigation in place for the compromises that you make. For example, if you cannot meet the requirements of a certain faculty, you may want to consider allowing them to use a separate platform for those requirements – provided the requirements are justified, of course. Alternatively, it may be about providing a faculty with a longer time period to adopt a new platform to give them time to change the business processes they use within their assessment process or to build confidence.  Compromise does not have to mean forcing individuals to do something suboptimal for them; it is about finding solutions that allow progress to be made. Progress is too often blocked by the inability of individuals to find compromise, and it is not for the want of trying. In my experience, this can be the hardest part of any digital transformation.

4. Define your end goal

When it comes to a transformation project such as digital assessment, it pays to think big. Don’t be scared about setting ambitious goals at the beginning of your project. By doing this, you are not necessarily stating that you will achieve these goals immediately, but you are framing a purpose upon which the project can be defined against. It will allow you to develop a clear communication strategy, signposting to the rest of the institution the changes that will take place, and break down the project into manageable phases or stages, and then celebrate once the milestones are achieved.

Start with an end goal in mind and define the benefits you hope to achieve. Ask yourself how the platform will need to scale and flex, considering the maximum number of students likely to ever use it and which subject areas it will – and won’t – be used for in the future. How much time will you save, and what kind of tests will the platform be used for? Will it be used for exams, essay coursework, advanced multimedia coursework, adaptive testing, or something else? Answering these questions and linking them to your end goal will provide you with the project scope in its entirety.

Piloting is a great way to get started, but it is not a guarantee of success. A successful pilot looks beyond the initial scope of the pilot to the next two or three years. Getting some experience of a digital assessment platform is helpful for you to hone your requirements, but it will not define the goals of your digital assessment transformation. If you have set out a certain ambition or goal, then piloting within an individual faculty or across a range of different assessment types will allow you to break down your implementation into a manageable size, reduce risk and learn from your experience. Once the pilot is complete, you can evaluate it against the overarching goals. Without these set out in advance, the pilot is a one-off activity, and it may be difficult for you to plan your next steps. The further down the line, you can look from the beginning, the better.

Finally, digital transformation does not exist in a vacuum, so the goals or scope you set out at the start of the project could evolve over time. Do not be afraid to review your goals periodically to ensure that they still align with the overall strategy of the institution as well as the development of pedagogy and technology over time.

Over to you

Before you get to the pilot stage of a digital assessment project, ensure you have worked through these four recommendations. It can take months, even years, to get to the point of a successful digital transformation strategy, but the groundwork is essential and will directly impact your educator and student experience, not just in the immediate future but also years down the line.

To further understand how to successfully adapt a change management mindset when transforming to digital, take a look at our eBook or get in touch with me directly to discuss the topic further.

GET IN TOUCH

Written by Harvey Tayman

February 20, 2023
Online Assessments