AI Tutors Provide 'Reliable' Instruction with Human Assistance, Study Reveals

A recent study indicates that an AI-driven tutor, supervised by a human and guided by detailed data on a students skill level, can surpass traditional human tutoring in effectiveness. The findings could reshape the debate on AIs role in classrooms and the degree of human monitoring required when interacting with students.

In a controlled experiment involving 165 British secondary school students aged 1315, the educational technology company Eedi.com tested a hybrid tutoring system. Expert human tutors oversaw a large language model (LLM) provided by Googles LearnLM. The AI generated responses to students math questions on Eedis platform, while humans reviewed and revised the messages before sending them, ensuring accuracy and appropriateness.

Students were unaware whether responses came from a human or AI, yet they engaged in longer conversations with the supervised AI than with human tutors alone, according to Bibi Groot, Eedis chief impact officer. Ultimately, students using the AI-human combination slightly outperformed those tutored solely by humans, solving new problem types correctly 66.2% of the time, compared to 60.7% for human-only tutoring. Researchers deemed the AI a reliable source of instruction. Human tutors approved roughly 75% of AI-generated messages with minimal edits.

When supported by both human and AI tutors, students corrected misconceptions and provided accurate answers over 90% of the time, versus 65% when using pre-written responses. The AI produced factual errors only 0.1% of the timefive errors out of 3,617 messagesand none raised safety concerns.

The study highlights the potential for AI, carefully tailored to pedagogy, to deliver scalable and individualized instruction. Students with AI assistance also showed stronger ability to tackle new topics.

Groot explained that the AIs advantage stemmed from access to highly personalized information about each students learning history over 20 weeks, including mastered topics and areas of difficulty. This allowed the AI to adapt its guidance dynamically, a capability standard LLMs lack. The AI processed all this context instantly, avoiding the cognitive overload human tutors experience while trying to track extensive student data in real time.

Despite their expertise, human tutors cannot be available around the clock. Eedi employs about 25 tutors across multiple time zones, covering 9 a.m. to 10 p.m. daily. Using AI as a front line tutor could expand student access, with humans stepping in only when persistent misconceptions arise or the conversation veers off track.

This study differs from earlier research, such as Stanfords October 2024 work, where AI only supported the tutor. In contrast, the Eedi study placed AI at the center of instruction, with humans supervising.

Experts see the findings as significant. Robin Lake of the Center on Reinventing Public Education noted that AI, when properly trained, can be an incredibly powerful tool and has the potential to scale tutoring in unprecedented ways. A previous Harvard study also found that AI tutors helped undergraduates learn more efficiently and stay engaged.

However, limitations remain. Liz Cohen of 50CAN pointed out that the study focused on 13-to-15-year-olds, leaving questions about younger students. She also highlighted concerns about student persistence and motivation, noting that AI tutors may favor self-driven learners, while those who become frustrated or disengaged might abandon the sessions more quickly.

Author: Natalie Monroe