Update Blog (Insights) “2025-07-29-how-can-we-…-examine-the-impact-of-interactions-with-llm-chatbot-tutors-on-students’-learning-psychology-and-behaviours”

Author: bp374

_posts/2025-07-29-how-can-we-…-examine-the-impact-of-interactions-with-llm-chatbot-tutors-on-students’-learning-psychology-and-behaviours.md

@@ -23,11 +23,11 @@ While AI development should be evidence-based, tutoring tools are increasingly b
 
 To overcome this lack of evidence, my project focused on examining the effect of interacting with LLM tutors on student learning psychology and key behaviours: effort, perseverance, resilience, and challenge-seeking. These behaviours are linked not only to academic achievement, but also to broader life outcomes, such as improved health and greater life satisfaction. The study aimed to provide a robust and informative comparison of behavioural differences among students supported by LLM tutors, human tutors, and those working independently, as well as exploring mechanisms underlying any observed behavioural changes.
 
-I conducted the experiment in a sixth form college with 150 students. Each student was presented with a series of eight challenging mathematics questions, delivered via a newly devised computer programme. The questions were divided into two sections, designed to be above the students’ expected A level capabilities and to increase gradually in difficulty. The task was voluntary, and students could decide how many questions to attempt. After completing the main task, students took part in a challenge-seeking exercise, in which they selected from a set of hypothetical future questions based on their preferred level of difficulty.
+I conducted the experiment in a sixth form college with 150 students. Each student was presented with a series of eight challenging mathematics questions, delivered via a newly devised computer-based task sequence. The questions were divided into two sections, designed to be above the students’ expected A level capabilities and to increase gradually in difficulty. The task was voluntary, and students could decide how many questions to attempt. After completing the main task, students took part in a challenge-seeking exercise, in which they selected from a set of hypothetical future questions based on their preferred level of difficulty.
 
 Each student was randomly assigned to one of the three support conditions: one group received help from a human teacher, another used an LLM AI tutor called Tutor Me (a simplified version of Khan Academy’s *Khanmigo*), and the third had access to open internet resources as a form of unsupported study. They were free to engage with their assigned support in any way they chose while working through the tasks.
 
-Throughout the session, the computer programme recorded participants’ behavioural data, allowing for detailed analysis of effort, perseverance, resilience, and challenge-seeking. It used indicators such as number of questions attempted, self-reported difficulty and effort, response to the second set of questions, and the difficulty level of questions chosen. For students in the AI tutor and human tutor conditions, interactions were also recorded to support further analysis of behaviours and student-tutor dialogue. After completing the tasks, students reflected on their experience in a qualitative questionnaire, which explored how they felt during the session.
+Throughout the session, behavioural data were recorded via the computer-based task sequence, allowing for detailed analysis of effort, perseverance, resilience, and challenge-seeking. It used indicators such as number of questions attempted, self-reported difficulty and effort, response to the second set of questions, and the difficulty level of questions chosen. For students in the AI tutor and human tutor conditions, interactions were also recorded to support further analysis of behaviours and student-tutor dialogue. After completing the tasks, students reflected on their experience in a qualitative questionnaire, which explored how they felt during the session.
 
 Under the supervision of Dr Ros McLellan, Associate Professor in Teacher Education and Development / Pedagogical Innovation, I was awarded a grant from the Accelerate Programme in collaboration with the Cambridge Centre for Data-Driven Discovery (C2D3) to fund the field work, including a human teacher to offer support in the ‘human teacher’ condition. Without the grant, we would not have been able to explore the difference between the human relationship and the machine relationship with the students.