Learning analytics is all about collecting, analysing and leveraging insights from learner data to make better-informed decisions about learning and teaching. And contrary to popular belief, with tools like our own Inspera Assessment, you don’t have to love maths or be a computer whizz in order to get started. Read on to get some inspiration with our tips.
Learning analytics offers an opportunity to reflect and optimise learning and assessment, through the collection, analysis, and reporting of data. All of these can be presented to the end-user as actionable insights in Inspera Assessment.
At Inspera, we distinguish between three kinds of learning analytics:
Combined, these three branches of learning analytics can be used to:
The more data that are collected on exam questions and stored in the item bank (a repository of questions used in an assessment program, and all information pertaining to those items), the richer the insights that can be generated. Therefore, it is worth investing in an organisational approach to item bank management. However, even if you don’t have a robust item bank or a learning technologist on your team, as an individual exam author user of Inspera Assessment, you could still gain benefits from one-off basic learning analytics per test.
Learning analytics are part of Inspera’s core offer, and we have a dedicated R&D team who are working on innovations in this field. We asked one of our data scientists, Dr Niels Goet, to share a few hands-on strategies on how to apply learning analytics on some of the teachers’ frequently asked questions.
First, it is good practice to have a look at the distribution of p-values (i.e. the proportion of correct answers for an item) across the entire test. Usually, your choice of items at the aggregate level should be informed by your knowledge of the group of learners that you want to test and the purpose of the test.
For example, including only very difficult items (low p-value) means that you are setting a very high bar for your students. This may or may not be your intended goal. And, if the purpose of the test is formative, that is, to help students learn in the process of being tested, you may want to consider the build-up of items from easy to difficult across the test, and include some easier items to start off with.
When using multiple-choice items, test writers need to choose effective “distractors”, that is, the other (incorrect) options that make up the item in addition to the correct response. Again, we can use data to inform our choices.
First, we want to make sure that we are choosing other answer options that allow us to discriminate well between low- and high-ability learners. When we want to improve and reuse items in a test, we can use historical response data to calculate the answer discrimination index. This index can be used to improve our selection of discriminators. It is defined as the proportion of examinees that select the answer option that is in the top 27 percent of the total score distribution, minus the proportion of examinees that select the answer option in the bottom 27 percent of the total score distribution. A negative answer discrimination index therefore tells us that lower-ability students are more likely to select a particular answer option and vice versa.
Second, we can evaluate a number of more straightforward statistics to inform our choice of distractors. The percentage of learners that selected each option, for example, gives us some idea of how well a distractor performs. An answer option that is never chosen may, for instance, be too implausible, while a distractor that attracts an equal number of responses as the correct option may reveal that the question is unclear, or that two answer options might be correct.
Do you want to learn more about how to conduct online exams and assessments, and how Inspera Assessment can make your assessments accessible, secure, valid and reliable? No worries, we wrote a Guide to Online Exams and Assessments.
Lorem ipsum dolor sit amet, consectetur adipiscing elit
These Stories on Why online assessment?