When choosing a digital assessment platform the decision for awarding bodies and exam boards often rests on features and costs. Both are important considerations for sure. They’re also easily measurable on paper.
However, four questions about how the candidate’s experience in the platform has influenced the design and implementation can tell you more about its actual value. Katherine Lancaster, RM’s Director of Product and Technology answers them in the latest of our Candidate Lens interviews.
Katherine explains the principles of candidate-centred digital assessment platform design in this video.
Prefer reading? The key insights from the conversation are below, with timestamped links so you can jump straight to the parts you’re interested in.
According to Katherine, the core principle of an effective digital assessment platform is that it measures knowledge and skills. The candidate’s ability to use the system should never be on test.
For Katherine, candidate experience sits at the centre of platform design. It is backed up by usability, security and accessibility. However, focusing on the candidate experience does not necessarily mean creating a design that delivers an enjoyable occasion for the person taking the test.
This is not a consumer product. Candidates value fairness, clarity and predictability. Awarding bodies and learners need a system where the technology is not front and centre. Candidates should be able to demonstrate their skills and knowledge within the parameters of the assessment rather than be tested on how well they can use the system itself.
To deliver a positive experience for test-takers assessment system providers must consider their entire journey up to and through the point of assessment. That journey encompasses preparation, the familiarisation process and understanding what to expect from the platform through to knowing how they will receive their result.
When evaluating an assessment platform, you should ask whether the vendor can show how the platform supports the whole journey, not just the exam screen.
Many awarding bodies are in a process of transforming from fully paper-based assessments towards the level of digital assessment that suits their circumstances. Some, often more common in professional qualifications, have embraced the benefits of using technology to more authentically assess their candidates. For example, accountancy awarding bodies can set questions that use embedded spreadsheet or other common financial software functionality, mirroring scenarios that candidates will encounter in the world of work,
In some school leaving subject exams, such as computer science, digital assessment improves authenticity for learners. It allows candidates to create code for answers in native applications instead of going through the strange experience of writing out by hand what they have spent years learning to create on screen.
Other awarding bodies are still relatively early in the digital journey. They may know that more authentic assessment is possible but implementing technology into their processes for its own sake is not the right approach. Understanding the desired outcome of the assessment will ensure that any technology deployment serves the objectives of the awarding body and its candidates.
Start with the objective of the assessment and then choose the digital approach that delivers it. The right partner will support progress at the appropriate pace.
It may seem that the ability to provide fair, reliable assessments, delivered to the right person is incompatible with preventing malpractice. The friction involved in effective counter-malpractice measures could be detrimental to a positive candidate experience.
Any analysis of how a digital assessment platforms handles the apparent dichotomy should begin with understanding the risk being managed. The context in which the candidate sits the exam is a vital consideration. Exams where candidates are paying to take them in a test centre or remotely have a different risk profile to a school-leaving exam with candidates sit in a school hall setting.
Technology partners can help awarding bodies establish processes that avoid negatively impacting candidate experience. Examples include:
The best controls to maintain positive candidate experience are ones that learners barely notice at the time, perhaps through familiarity but understand the value of.
When evaluating a digital assessment platform’s suitability, ask how its controls adapt to different settings.
It can be tempting to pigeonhole the majority of assessment candidates as of a relatively narrow age range, with associated assumptions about their facility with digital systems. However, digital literacy levels vary widely. In addition, it is now more common for people to change careers at a later stage in life, increasing the average age for some professional qualifications.
Accommodating different levels of digital literacy involves a deliberate approach to user experience design, which encompasses both the platform and process around it.
Accomplishing this goal involves going beyond accessibility considerations to achieve a design that is in essence invisible to candidates. Instructions should be clear; navigation uncomplicated and consistent.
RM’s approach involves extensive user testing with the learners who will be taking the test in our platform to ensure that they’re comfortable with the process. Outside the confines of the assessment platform, we consider test centre readiness and other elements of the candidate onboarding experience. This helps ensure that whatever their level of digital literacy candidates can just get on with taking their assessment.
As part of checking whether a digital assessment platform fits your needs, find out if the vendor trials the system with candidate groups and helps them get used to it.
The overarching principle is that the technology should only serve the purpose and objectives of the actual assessment. To help evaluate the extent to which any digital assessment platform achieves this, keep the four questions in mind:
Katherine Lancaster, RM’s Director of Product and Technology explains RM’s approach to answering these questions in the video at the top of this page.
Header photo by Haseeb Modi on Unsplash