AI for Learning: Oak National Academy
How Oak National Academy is experimenting with GenAI to give teachers back their Sunday nights.
👋 This week we have the second of our two-parter on Oak National Academy, which is also the second in our new series on AI for Learning.
We’re running an online meetup next Tuesday, on the topic of measuring impact in learning: sign up here.
And we’re also recruiting for our October cohorts. Find out more about our programmes here.
“We want to give teachers their Sunday nights back,” says John Roberts, Director of Product and Engineering for Oak National Academy. This is the goal for their new lesson planning assistant, Aila.
Oak was set up in April 2020 as an emergency response to the closure of schools as a result of the Covid-19 pandemic. Since then, they have evolved into an arm’s length body owned by the Department for Education and are now used by a third of the teachers in the UK (see our recent case study telling the extraordinary story of their launch).
Over the last couple of years, Oak has been experimenting with GenAI to understand how it might help them further their Theory of Change and help teachers deliver better lessons and reduce their workload. Last Thursday, after a pilot with 10,000 teachers, Oak released Aila.
“We know that around three-quarters of teachers adapt the content that they download from Oak,” says John. “So we’ve been exploring things that will support them to do it. Whether it's to make it more difficult. Or increase or decrease the reading age. Or give a task some additional challenge. That kind of thing.”
The goal was to bring their unique assets to bear on the public purpose they’ve been set up to make an impact on.
“Once the whole curriculum is complete, we will have around 350,000 minutes of transcribed video content and essentially all of the content in the curriculum aged four to 16. Plus, we have about 100,000 quiz questions,” explains John. “So how can we use that as a corpus of work as an anchor for a large language model?”
Three goals
After a period of experimentation around the potential of genAI - “Can we improve the efficiency of our own content creation process? Can we just make it easier for our curriculum partners?” - Oak focused on creating a lesson planning assistant for teachers.
They had three goals:
Increase the quality of the output provided by out-of-the-box tools so that it was “as close to what good teachers can make as possible”
Increase the safety of out-of-the-box tools. “The bar is very high if you want to deliver in a classroom.”
Keep the teachers in the driving seat. “To deliver a good lesson, you need to have gone through the planning process and rehearsed it in your head.”
“We’ve been exploring how we can use this technology in pedagogically sound ways, and give reasonable guarantees that it's safe to use in the classroom,” says John.
Aila is built on top of GPT4o, although they have also been experimenting with Anthropic’s Claude and other models.
To achieve these goals, they’ve taken three approaches:
Prompt engineering: They’ve developed prompts totalling 9000 words that codify what they see as really good practice in lesson planning.
Retrieval-Augmented Generation (RAG): This combines the results from the LLM with Oak’s corpus of National Curriculum aligned content.
Content anchoring: The starting point is suggesting similar quality assured lessons that are age-appropriate and then adapting them.
John simplifies this as, “How do we give you great results that are relevant to the UK’s National Curriculum in a way that you can’t do yourself with Chat GPT? At least without lots of very careful prompting.”
The user experience
Teachers can tell Aila what they want to teach, starting with the learning outcome. For example, they might want to plan a lesson on flooding.
Aila will start by suggesting a quality-assured lesson plan from Oak’s catalogue as a starting point to begin the planning process if there’s one available or allow you to start from scratch and instead it suggests appropriate learning outcomes.
“There's no point reinventing the wheel for something like a lesson on Pythagoras,” says John. “The first lesson on Pythagoras for most pupils, is a reasonably similar lesson, the learning objectives are reasonably similar. Instead, we help them to easily adapt it or make it more specific to their needs.”
At each stage, they can add suggestions. Chatting to Aila keeps teachers in the driving seat, forcing them to think through the lesson in individual steps, rather than a single process. “It feels like you’re planning a lesson, despite the support,” he says.
Teachers can ask to add or remove content, ask for more challenging practice tasks, adjust the reading age or even make it more contextual to the school’s location. “For example,” says John, “if the teacher planning the flooding lesson was in Cumbria, they could ask it to rewrite the plan to include Storm Desmond and flooding that destroyed Pooley Bridge.”
Finally, Aila will then review the lesson for coherence, ensure all the keywords have definitions and check it for UK spelling and grammar. “Chat GPT loves to put stray ‘z’s in,” smiles John.
Aila will then produce a lesson plan, slide deck, starter and exit quiz, practice tasks and worksheet for the teachers to download.
Teachers can also ask it to produce additional materials, for example, a briefing for the teacher they job share with who might be unfamiliar with the local area.
Early results
The results from the closed pilot have been encouraging. On average, teachers are taking around 10 to 15 mins to plan a lesson that would normally take 45 to 50 mins. Most teachers plan around 18-22 lessons a week and a small group of teachers evaluating the product have said they are saving 3-4 hours of planning time on average, each week. Or, to put it another way, their Sunday night.
They’ve been working with teachers to help them evaluate their goals, including blind testing the results of quiz question generation. “We did some evaluation of getting teachers to kind of say which one is better, the AI-generated or the teacher-made question, " says John. “We’ve seen some interesting results.”
“Maths is a known challenge and may need other non-AI approaches. It's actually quite hard to generate questions in Key Stage 4 science because there's very specific vocabulary and definitions required. For example, what is a mitochondria? It's really important that you are very precise in the definition to meet the needs of the exam specifications.” He gives another example: “what does the verb ‘to push’ mean? It’s really good here, this is where LLMs thrive, and it can be a little more creative.”
This is one of the areas where, with continuous evaluation, and subject focussed improvements, Oak reckon they can do much better than the out-of-the-box technology.
As with their code and content, Oak is taking an open-source approach to development. They are keen to share their learnings and collaborate with others in the sector. You can access the code for Aila, and the evaluation tools, on their GitHub.
Organising the team
I ask if they’ve thought differently about how they have organised their product team to tackle this opportunity.
“It's a similar-sized squad,” he says. AI is one of five product teams. “It's similar roles: education experts, excellent engineers, product designer and product manager. In total, eight people, plus a couple of people inputting from the wider team.” However, Oak has invested in training. “Some folks have done some machine learning apprenticeships, and brought that expertise into the team.”
The biggest challenge has been around evaluating the outcomes. “It's an emerging technology. The benchmarking and quality measurement, the moderation, and the safety elements are really important. That's what we focused on quite a lot. And this is something we hear is actually very unique to Oak.” These tools are open source too.
Key takeaways
Reflecting on the conversation, there are a few key takeaways that are likely to be relevant to others in EdTech experimenting with AI.
Problem-first: Oak have focused on using GenAI as a new way to tackle the core problem they set out to impact, within their strategy and theory of change, for a specific audience (UK teachers).
Build on your unique advantage: Oak have leveraged their unique assets - their vast corpus of National Curriculum aligned content - to produce something that the general technology can’t do. High-quality transcripts of lesson videos have been highly valuable.
Keep the human in the loop: Oak has put the teacher in the driving seat and made them do the thinking that they need to do to deliver a great lesson. The planning process remains important and integral.
Focus on adding value: it’s not about replacing teachers, it’s about assisting them, and freeing them up to spend more time focusing on pupils in the classroom and helping increase the lesson quality.
Evaluation and co-creation: Oak have put a significant amount of effort into testing and evaluating the results and working with teachers to ensure the results are high quality and safe to use in the classroom.
John is keen to point out that this is just the start. “We can’t wait to see the feedback we get when we have more teachers using it,” he says. “We know that there are some subjects that Aila will not perform as well in, in particular Maths, Science, and Music, we’re keen to highlight this, and we’re going to be looking specifically into ways to improve them.”
This embraces the spirit of Oak’s first release at the start of the pandemic. “Let’s ship it and trust teachers to tell us how to make it better.”
Go and try Aila out for yourself by registering here.
Oak National Academy provides openly licensed access to all of their National Curriculum aligned lessons and resources. They are interested in working with other EdTech providers who can see potential in using it, providing feedback that can help them further their mission and theory of change. Oak National Academy’s code is also openly licensed. You can get in touch here. They are also hiring.