GenAI can transform educational assessments by improving efficiency, engagement, equity, and personalization, but without shared evaluation standards, AI-generated assessments risk low-quality outcomes. This project analyzes over 250 recent research studies to identify leading evaluation practices and create guidance for schools, EdTech developers, and measurement researchers.
Embedded Files
Goals and Objectives
Goals and Objectives
Build a consensus-based GenAI research analysis approach to evaluate validity, reliability and fairness.
Human-label at least 250 research studies using the approach.
Pilot large language models to scale rubric application.
Provide actionable guidance for developers, educators, and policymakers.
Beneficiaries
Beneficiaries
Students – Shorter, more engaging, equitable assessments.
Educators & System Leaders – Trusted guidance to select high-quality tools.
EdTech & Assessment Providers – Shared rubric and benchmarks to accelerate innovation.
Professional Communities – NCME, ISTE, 1EdTech members gain reusable evidence-based resources.
Methodology
Methodology
Research review → Human-in-the-loop labeling → LLM pilot → Synthesis & Recommendations
Key Outcomes and Deliverables
Key Outcomes and Deliverables
Publicly available evidence framework and evaluation rubric.
Annotated dataset of 250+ studies.
Final synthesis report summarizing key findings and gaps.
Web portal hosting all materials for broad accessibility.
Accessibility & Global Impact
Accessibility & Global Impact
Open-access materials via a dedicated portal with Creative Commons licensing.
No paywalls; designed for under-resourced schools and public institutions.
Dissemination through conferences, webinars, and professional networks.
The Coding Team
The Coding Team
John Whitmer, Learning Data Insights, Strategic Leadership
Maggie Beiting-Parrish, Learning Data Insights, Psychometric Expertise
Alexis Andres, Learning Data Insights, Research Design and AI Integration
Aaron Wong, University of Minnesota, Data Analysis and Reliability
Melanie Kurimchak, Learning Data Insights, Technical Product Management
Heeryung Choi
Chris Steadman, University of Minnesota - Twin Cities, Ph.D. Student
Alexander White
Nhidi Nasair, University of Pennslyvania, Ph.D. Candidate
The Advisory Committee
The Advisory Committee
Kristen DiCerbo, Khan Academy
Michael Feldstein, 1EdTech
Amy Hendrickson, College Board and NCME
Susan Lottridge, Pearson
Brandon Olszewski, ISTE and ASCD
Nancy Otero, Bill and Melinda Gates Foundation, ex officio
Page updated
Report abuse