Skip to content
“AI brings enormous opportunities, but has also created an explosion of education products with highly variable and largely untested quality. Independent quality assurance helps decision-makers tell the difference between what's just effectively promoted and what actually works, so that the right tools reach the students who need them most. ”
Marc Shotland, Chief Technical and Learning Officer, IDinsight

Adapting EdTech quality assurance for AI tools

Plenty of EdTech Quality Assurance already exists, but it doesn’t test for what AI introduces.

Our goal is to identify, align, and build the AI-specific pieces currently missing in the sector.

We do this through three key workstreams:

  1. Building a layered set of global benchmarks and evaluations: We are collating emerging AI standards and EdTech evaluation frameworks to help decision-makers determine what an impactful AI tool should deliver. 
  2. Enabling rapid evaluation: We evaluate tools on a timeline that matches how fast AI tools evolve. Instead of a single multi-year randomized control trial, we run multiple shorter, rapid-cycle evaluations with a keen focus on students and teachers as end users. 
  3. Synthesizing evidence as a digital public good: We publish everything we find openly to empower decision-makers to compare tools before committing to scale them.

Assessing quality at every stage

Since AI tools change substantially as programs evolve and mature, we treat evidence generation as sequential but iterative. We assess evidence across a product’s full lifecycle rather than making a one-off pass/fail judgment at the end.

  • We start with pedagogical foundations. We ask: Does the tool rest on a clear theory of change? Does its design reflect what research tells us about how children learn? Is its content adapted to the local curriculum and language?
  • Safety and ethics come next. We ensure that a product used by children poses no risk of harm.
  • Implementation and uptake follow. At this stage, we assess the quality of the interaction between the tool and end users, and whether it holds up under the conditions schools actually face.
  • Outcomes and cost-effectiveness come next. A tool that improves learning but cannot be afforded at scale needs to be revisited.

If an AI product changes substantially during the program implementation, we send our assessment back through earlier stages rather than carrying forward an old result.

Building the evidence needed to scale AI EdTech tools

Our work is deliberately product-agnostic. The QA Facility does not certify or endorse any specific tools. Its end goal is to provide governments and funders a credible way to tell which AI-EdTech products actually work before committing to scale them. 

Through our work, we also aim to shift developer incentives. Commercial developers have little economic reason to make AI EdTech tools usable in low-resource settings. Language coverage, curriculum alignment, and performance on offline or low-bandwidth connections are expensive and largely invisible to buyers. By measuring them publicly, we make that investment visible.

Looking forward

Hundreds of millions of children still don’t have access to quality education. AI is a promising tool that could reach them at a cost education budgets can bear. We are building this evidence base now, while AI EdTech is still emerging. Benchmarks set at this stage can influence what the sector and key stakeholders choose to build, not just judge what they have already shipped.

We will continue to share more about the QA Facility’s work and partners in future, so stay tuned.

“AI brings enormous opportunities, but has also created an explosion of education products with highly variable and largely untested quality. Independent quality assurance helps decision-makers tell the difference between what's just effectively promoted and what actually works, so that the right tools reach the students who need them most. ”
Marc Shotland, Chief Technical and Learning Officer, IDinsight

Adapting EdTech quality assurance for AI tools

Plenty of EdTech Quality Assurance already exists, but it doesn’t test for what AI introduces.

Our goal is to identify, align, and build the AI-specific pieces currently missing in the sector.

We do this through three key workstreams:

  1. Building a layered set of global benchmarks and evaluations: We are collating emerging AI standards and EdTech evaluation frameworks to help decision-makers determine what an impactful AI tool should deliver. 
  2. Enabling rapid evaluation: We evaluate tools on a timeline that matches how fast AI tools evolve. Instead of a single multi-year randomized control trial, we run multiple shorter, rapid-cycle evaluations with a keen focus on students and teachers as end users. 
  3. Synthesizing evidence as a digital public good: We publish everything we find openly to empower decision-makers to compare tools before committing to scale them.

Assessing quality at every stage

Since AI tools change substantially as programs evolve and mature, we treat evidence generation as sequential but iterative. We assess evidence across a product’s full lifecycle rather than making a one-off pass/fail judgment at the end.

  • We start with pedagogical foundations. We ask: Does the tool rest on a clear theory of change? Does its design reflect what research tells us about how children learn? Is its content adapted to the local curriculum and language?
  • Safety and ethics come next. We ensure that a product used by children poses no risk of harm.
  • Implementation and uptake follow. At this stage, we assess the quality of the interaction between the tool and end users, and whether it holds up under the conditions schools actually face.
  • Outcomes and cost-effectiveness come next. A tool that improves learning but cannot be afforded at scale needs to be revisited.

If an AI product changes substantially during the program implementation, we send our assessment back through earlier stages rather than carrying forward an old result.

Building the evidence needed to scale AI EdTech tools

Our work is deliberately product-agnostic. The QA Facility does not certify or endorse any specific tools. Its end goal is to provide governments and funders a credible way to tell which AI-EdTech products actually work before committing to scale them. 

Through our work, we also aim to shift developer incentives. Commercial developers have little economic reason to make AI EdTech tools usable in low-resource settings. Language coverage, curriculum alignment, and performance on offline or low-bandwidth connections are expensive and largely invisible to buyers. By measuring them publicly, we make that investment visible.

Looking forward

Hundreds of millions of children still don’t have access to quality education. AI is a promising tool that could reach them at a cost education budgets can bear. We are building this evidence base now, while AI EdTech is still emerging. Benchmarks set at this stage can influence what the sector and key stakeholders choose to build, not just judge what they have already shipped.

We will continue to share more about the QA Facility’s work and partners in future, so stay tuned.

“AI brings enormous opportunities, but has also created an explosion of education products with highly variable and largely untested quality. Independent quality assurance helps decision-makers tell the difference between what's just effectively promoted and what actually works, so that the right tools reach the students who need them most. ”
Marc Shotland, Chief Technical and Learning Officer, IDinsight

Adapting EdTech quality assurance for AI tools

Plenty of EdTech Quality Assurance already exists, but it doesn’t test for what AI introduces.

Our goal is to identify, align, and build the AI-specific pieces currently missing in the sector.

We do this through three key workstreams:

  1. Building a layered set of global benchmarks and evaluations: We are collating emerging AI standards and EdTech evaluation frameworks to help decision-makers determine what an impactful AI tool should deliver. 
  2. Enabling rapid evaluation: We evaluate tools on a timeline that matches how fast AI tools evolve. Instead of a single multi-year randomized control trial, we run multiple shorter, rapid-cycle evaluations with a keen focus on students and teachers as end users. 
  3. Synthesizing evidence as a digital public good: We publish everything we find openly to empower decision-makers to compare tools before committing to scale them.

Assessing quality at every stage

Since AI tools change substantially as programs evolve and mature, we treat evidence generation as sequential but iterative. We assess evidence across a product’s full lifecycle rather than making a one-off pass/fail judgment at the end.

  • We start with pedagogical foundations. We ask: Does the tool rest on a clear theory of change? Does its design reflect what research tells us about how children learn? Is its content adapted to the local curriculum and language?
  • Safety and ethics come next. We ensure that a product used by children poses no risk of harm.
  • Implementation and uptake follow. At this stage, we assess the quality of the interaction between the tool and end users, and whether it holds up under the conditions schools actually face.
  • Outcomes and cost-effectiveness come next. A tool that improves learning but cannot be afforded at scale needs to be revisited.

If an AI product changes substantially during the program implementation, we send our assessment back through earlier stages rather than carrying forward an old result.

Building the evidence needed to scale AI EdTech tools

Our work is deliberately product-agnostic. The QA Facility does not certify or endorse any specific tools. Its end goal is to provide governments and funders a credible way to tell which AI-EdTech products actually work before committing to scale them. 

Through our work, we also aim to shift developer incentives. Commercial developers have little economic reason to make AI EdTech tools usable in low-resource settings. Language coverage, curriculum alignment, and performance on offline or low-bandwidth connections are expensive and largely invisible to buyers. By measuring them publicly, we make that investment visible.

Looking forward

Hundreds of millions of children still don’t have access to quality education. AI is a promising tool that could reach them at a cost education budgets can bear. We are building this evidence base now, while AI EdTech is still emerging. Benchmarks set at this stage can influence what the sector and key stakeholders choose to build, not just judge what they have already shipped.

We will continue to share more about the QA Facility’s work and partners in future, so stay tuned.

“AI brings enormous opportunities, but has also created an explosion of education products with highly variable and largely untested quality. Independent quality assurance helps decision-makers tell the difference between what's just effectively promoted and what actually works, so that the right tools reach the students who need them most. ”
Marc Shotland, Chief Technical and Learning Officer, IDinsight

Adapting EdTech quality assurance for AI tools

Plenty of EdTech Quality Assurance already exists, but it doesn’t test for what AI introduces.

Our goal is to identify, align, and build the AI-specific pieces currently missing in the sector.

We do this through three key workstreams:

  1. Building a layered set of global benchmarks and evaluations: We are collating emerging AI standards and EdTech evaluation frameworks to help decision-makers determine what an impactful AI tool should deliver. 
  2. Enabling rapid evaluation: We evaluate tools on a timeline that matches how fast AI tools evolve. Instead of a single multi-year randomized control trial, we run multiple shorter, rapid-cycle evaluations with a keen focus on students and teachers as end users. 
  3. Synthesizing evidence as a digital public good: We publish everything we find openly to empower decision-makers to compare tools before committing to scale them.

Assessing quality at every stage

Since AI tools change substantially as programs evolve and mature, we treat evidence generation as sequential but iterative. We assess evidence across a product’s full lifecycle rather than making a one-off pass/fail judgment at the end.

  • We start with pedagogical foundations. We ask: Does the tool rest on a clear theory of change? Does its design reflect what research tells us about how children learn? Is its content adapted to the local curriculum and language?
  • Safety and ethics come next. We ensure that a product used by children poses no risk of harm.
  • Implementation and uptake follow. At this stage, we assess the quality of the interaction between the tool and end users, and whether it holds up under the conditions schools actually face.
  • Outcomes and cost-effectiveness come next. A tool that improves learning but cannot be afforded at scale needs to be revisited.

If an AI product changes substantially during the program implementation, we send our assessment back through earlier stages rather than carrying forward an old result.

Building the evidence needed to scale AI EdTech tools

Our work is deliberately product-agnostic. The QA Facility does not certify or endorse any specific tools. Its end goal is to provide governments and funders a credible way to tell which AI-EdTech products actually work before committing to scale them. 

Through our work, we also aim to shift developer incentives. Commercial developers have little economic reason to make AI EdTech tools usable in low-resource settings. Language coverage, curriculum alignment, and performance on offline or low-bandwidth connections are expensive and largely invisible to buyers. By measuring them publicly, we make that investment visible.

Looking forward

Hundreds of millions of children still don’t have access to quality education. AI is a promising tool that could reach them at a cost education budgets can bear. We are building this evidence base now, while AI EdTech is still emerging. Benchmarks set at this stage can influence what the sector and key stakeholders choose to build, not just judge what they have already shipped.

We will continue to share more about the QA Facility’s work and partners in future, so stay tuned.

“AI brings enormous opportunities, but has also created an explosion of education products with highly variable and largely untested quality. Independent quality assurance helps decision-makers tell the difference between what's just effectively promoted and what actually works, so that the right tools reach the students who need them most. ”
Marc Shotland, Chief Technical and Learning Officer, IDinsight

Adapting EdTech quality assurance for AI tools

Plenty of EdTech Quality Assurance already exists, but it doesn’t test for what AI introduces.

Our goal is to identify, align, and build the AI-specific pieces currently missing in the sector.

We do this through three key workstreams:

  1. Building a layered set of global benchmarks and evaluations: We are collating emerging AI standards and EdTech evaluation frameworks to help decision-makers determine what an impactful AI tool should deliver. 
  2. Enabling rapid evaluation: We evaluate tools on a timeline that matches how fast AI tools evolve. Instead of a single multi-year randomized control trial, we run multiple shorter, rapid-cycle evaluations with a keen focus on students and teachers as end users. 
  3. Synthesizing evidence as a digital public good: We publish everything we find openly to empower decision-makers to compare tools before committing to scale them.

Assessing quality at every stage

Since AI tools change substantially as programs evolve and mature, we treat evidence generation as sequential but iterative. We assess evidence across a product’s full lifecycle rather than making a one-off pass/fail judgment at the end.

  • We start with pedagogical foundations. We ask: Does the tool rest on a clear theory of change? Does its design reflect what research tells us about how children learn? Is its content adapted to the local curriculum and language?
  • Safety and ethics come next. We ensure that a product used by children poses no risk of harm.
  • Implementation and uptake follow. At this stage, we assess the quality of the interaction between the tool and end users, and whether it holds up under the conditions schools actually face.
  • Outcomes and cost-effectiveness come next. A tool that improves learning but cannot be afforded at scale needs to be revisited.

If an AI product changes substantially during the program implementation, we send our assessment back through earlier stages rather than carrying forward an old result.

Building the evidence needed to scale AI EdTech tools

Our work is deliberately product-agnostic. The QA Facility does not certify or endorse any specific tools. Its end goal is to provide governments and funders a credible way to tell which AI-EdTech products actually work before committing to scale them. 

Through our work, we also aim to shift developer incentives. Commercial developers have little economic reason to make AI EdTech tools usable in low-resource settings. Language coverage, curriculum alignment, and performance on offline or low-bandwidth connections are expensive and largely invisible to buyers. By measuring them publicly, we make that investment visible.

Looking forward

Hundreds of millions of children still don’t have access to quality education. AI is a promising tool that could reach them at a cost education budgets can bear. We are building this evidence base now, while AI EdTech is still emerging. Benchmarks set at this stage can influence what the sector and key stakeholders choose to build, not just judge what they have already shipped.

We will continue to share more about the QA Facility’s work and partners in future, so stay tuned.