Colleague Thomas Oran-Reubens and I attended the recent International Association for Educational Assessment (IAEA) conference in Toronto, Canada. The conference is a great opportunity to connect with the assessment community from across the world. Among the many presentations, workshops and seminars about all aspects of educational assessment, one panel discussion stood out.
Titled Global Perspectives on Assessment Modernisation, it featured contributions from six panellists representing diverse regions:
Dan Koenig (Education Quality and Accountability Office, Ontario),
Confidence Dikgole (Independent Examinations Board, South Africa)
Dr Wayne Wesley (Caribbean Examinations Council)
Dr Naveed Yousuf (CEO, Aga Khan University Examination Board)
Pascal Couture (Pascal Couture Professional Consulting, previously Ministry of Education and Childcare, Alberta)
Dr Mary Pitoniak (President, Pitoniak Educational Measurement Consulting)
Despite the different contexts, there was remarkable consistency in what they said. This is a summary of the panellists' observations and recommendations relating to digital assessment based on experience.
“It wasn't a technology project… it was a trust project.”
Dan described Ontario's move to digital assessment as fundamentally about building confidence with parents, students, teachers, schools and boards. His answer was to “communicate, communicate, communicate”. EQAO has tried to demystify assessment through material explaining how an assessment item is created, how adaptive testing works and why digital delivery is being used.
“We must not confuse technological advancement with assessment advancement.”
Confidence argued that learners do not experience technology equally. Device quality, connectivity, infrastructure, technical support and familiarity can all affect performance. If those factors begin determining success, we risk measuring privilege alongside the intended construct.
“Accessibility cannot begin with the accommodation. It has to begin with the assessment.”
Her test for modernisation was simple: technology should strengthen the promise that every learner gets a fair opportunity to demonstrate what they know and can do. If it weakens that promise, however sophisticated the technology is, don't proceed.
Naveed brought another reality check. Examination systems may technically support computer-based assessment, but if everyday teaching, learning and formative assessment remain paper-based, is it fair for a learner's first significant digital experience to be their high-stakes examination?
He also raised practical realities relating to electricity, connectivity, infrastructure and access to devices, alongside the challenge of establishing comparability and equivalence where paper and digital versions of the same qualification coexist.
“Technical readiness doesn't necessarily mean system readiness or learner readiness.”
Wayne brought the discussion back to results and reporting. With high-volume paper examinations, extracting granular information for individual centres is difficult. CXC is putting more information into its results reporting so educators can interpret patterns.
His example was simple but useful: if a whole class struggles with a particular module, that may point towards a teaching or curriculum intervention. If one learner struggles while the rest of the class performs well, that suggests a more individual intervention.
Mary brought the psychometric and validity perspective. She described organisations wanting to move to computer-based testing simply because everybody else was doing it. In her previous licensing/certification work, she estimated that around 80% of organisations she challenged on purpose, items, budget and readiness ultimately decided they weren't ready to make that transition.
“Slow down. Don't be so fast to jump on the AI train.”
Her point was that AI item generation may make creating questions quicker and cheaper, but it doesn't remove the need to define the domain, construct, evidence and reporting purpose. A well-designed paper assessment producing defensible interpretations can be preferable to an impressive AI-enabled digital assessment where we aren't confident what the results actually mean.
Pascal used a Niagara Falls tightrope analogy to describe Alberta's approximately five-year digital journey. On one side is strategy: why are we assessing and where do we want to go? On the other is operations: can schools, infrastructure and the platform actually deliver it? The tightrope itself is psychometric defensibility, validity, comparability and fairness.
Having made that transition, Alberta can now explore things that weren't previously as practical, including AI-supported marking, adaptive testing, embedded field testing and innovative item types. Digital transformation shouldn't simply reproduce the paper assessment on a screen; done properly, it can enable different and richer evidence of what learners can do.
Photo by Conor Samuel on Unsplash