Authentic Assessment: Feasible at Last Thanks to AI

This post is about summative assessment design. I know, I might have lost you right here, but bear with me. I suspect we might be at the brink of some very exciting assessment times that I want to share with you. Let's jump right in.

Typically, we want summative assessment to meet the following criteria:

1. Feasibility: Can this be done without a small army of assessment specialists and graders?
2. Fraud Resistance: Can we prevent students from outsourcing their tasks to AI?
3. Measurement Quality: Are the data and methods valid and reliable?
4. Alignment: Is it designed to test the intended learning outcomes?
5. Transparency: Do students understand what’s expected and how they’ll be assessed?
6. Fairness: Is the assessment unbiased and equitable for all students?

The Imperfect World of Present-Day Assessments

Unhappily, no single assessment method currently in use scores high on all these criteria. Oral exams, for instance, allow for immediate feedback and assessment of critical thinking and articulation skills. However, they are time-consuming and subjective. Multiple-choice quizzes, on the other hand, are easy to administer and grade and are highly objective. However, they often fail to measure deep understanding or the ability to apply knowledge creatively. Take-home assignments encourage deeper engagement with the material and the development of research skills. Yet, they come with a high potential for plagiarism and outsourcing to AI.

The Cinderella of Assessment Approaches

Lying in wait, there is an assessment approach that has been overlooked despite its numerous virtues. It is resistant to fraud, transparent, and allows for quality measurement. It has only one, but fatal, flaw - it is terribly costly. Let's talk about authentic assessment.

Authentic assessment is every assessment specialist's wet dream. It encourages students to think critically, not just answering questions but asking the right ones. It makes them navigate the social and interdisciplinary intricacies of working in a team. It requires them to apply abstract knowledge to concrete challenges and to integrate academic learning with practical skills. In short, it develops and tests the kind of competencies that higher education aims to instill. So how does it work?

The idea behind authentic assessment is for students to tackle real-world (authentic) problems, demonstrating their ability to apply theoretical knowledge in practical settings. For example, students might be tasked with solving a problem such as improving the design of sensors used to measure the vitals of newborns, which currently cause discomfort and often render the measurements incorrect. In this task, students would need to carefully articulate the problem, develop a blueprint for a solution, and present it to the class.

AI, the great enabler

You can perhaps already see why authentic assessment has mostly been an elusive dream. It is incredibly resource-intensive to implement, requiring instructors to design meaningful problems, involve industry and societal partners, and provide detailed feedback, every term. But this is where the devil that ruined our assignments in the first place, artificial intelligence, can be a blessing in disguise.

 AI can produce any number of realistic and varied problem scenarios in a whizz, reducing the workload on educators. It can also serve as a stand-in for industry and societal partners, with AI-powered chatbots responding to students' queries. Furthermore, AI tools can analyze student submissions and provide high-quality, detailed, and timely feedback. AI can perform bias and accessibility checks, identifying hidden curricula in assignment instructions.

The long-time-coming assessment revolution at last at our doorstep

Technology has a great record of making things that were previously impossible, possible. I hope the arrival of AI will enable us to replace current, suboptimal summative assessment methods with better ones. Authentic assessment, the excellent, yet currently impractical method for developing and testing higher-order thinking skills, may soon become not only feasible but the norm. How wonderful would that be?

This post and image were created with the help of ChatGPT Plus.