Overview

Led by Dr. Cristi Ford (D2L), with co-authors Dr. Angela Gunder (Opened Culture, University of Arizona) and Linda Feng (D2L), this experience paper takes up a problem the field has been slow to name: quality assurance frameworks are evolving far more slowly than the AI technologies they are meant to govern. As organizations adopt AI for course design, content generation, feedback, and assessment support, the standards that have long protected quality in digital learning are struggling to keep pace. The paper argues that quality assurance has always been a human practice grounded in shared values, peer expertise, and a culture of continuous improvement, and that AI makes that human core more essential rather than less.

To examine the problem at the levels where it actually unfolds, the authors conducted two complementary inquiries. The first studied quality assurance at the system level through a multimodal engagement with the Quality Matters AI Working Group, culminating in a facilitated design sprint at the QM Board Retreat in November 2025. The second studied quality assurance at the artifact level through a rubric evaluation study conducted in partnership with D2L, in which evaluators applied a prototype rubric to AI-generated and human-authored educational materials. Designed to speak to each other, the two studies offer a two-level portrait of what quality requires in AI-enabled learning.

At the system level, a consistent and urgent pattern emerged. Organizations are adopting AI faster than they are investing in the governance frameworks, professional learning pathways, and shared standards needed to ensure those efficiencies serve learners well, with fewer than a quarter of institutions reporting any formal AI policy. Participants were equally clear that human judgment must remain central to quality review, since the interpretive, empathic, and ethical dimensions of that work cannot be automated. They named context as the prerequisite for any quality determination, raised equity concerns about surveillance-based responses to AI, and called for layered, adaptable frameworks that preserve core quality principles while building in mechanisms for frequent, community-informed updates.

At the artifact level, the findings converged from a different direction. Across all evaluators, the clearest result was that meaningful evaluation requires pedagogical context before it can begin, because content can appear coherent and polished while remaining misaligned with the specific objectives, audience, and standards it is meant to serve. Evaluators identified the dimensions automated tools cannot assess, including organization and flow, disciplinary accuracy, and the voice and engagement that invite learners into ideas. They also pointed toward a productive division of labor, in which automated systems handle mechanical and rule-based criteria so human reviewers are freed to address the higher-order dimensions only they can reliably judge, and toward evaluation understood as an iterative dialogue with AI rather than a single output to be scored.

Read together, the two levels tell one story. Quality assurance in AI-enabled digital learning cannot be reduced to checklists, automated scores, or static rubrics, because artifact-level evaluation without system-level governance produces inconsistent judgments, while system-level governance without artifact-level evidence produces policy untethered from practice. Drawing on human-centered AI, TPACK-informed perspectives on educator knowledge, peer-review approaches to online course quality, and Isabelle Hau’s concept of Relational Quotient, the paper reframes quality assurance as an ecosystem problem whose relational dimensions form the load-bearing structure on which technical and pedagogical quality both depend. Its closing invitation is a practical one: to match every investment in AI adoption with investment in governance, professional learning, and the relational infrastructure through which quality is continuously made.