A Real-Time Model to Assess Student Engagement during Interaction with Intelligent Educational Agents

LaVonda N. Brown; Ayanna M. Howard

Download Paper | Permalink

Conference: 2014 ASEE Annual Conference & Exposition
Location: Indianapolis, Indiana
Publication Date: June 15, 2014
Start Date: June 15, 2014
End Date: June 18, 2014
ISSN: 2153-5965
Conference Session: Computer-Based Learning Models
Tagged Division: Computers in Education
Page Count: 11
Page Numbers: 24.95.1 - 24.95.11
DOI: 10.18260/1-2--19987
Permanent URL: https://strategy.asee.org/19987
Download Count: 464

Request a correction

Paper Authors

biography

LaVonda N. Brown Georgia Institute of Technology

visit author page

LaVonda Brown received her B.S. (2010) in Electronics Engineering from Norfolk State University and M.S. (2012) in Electrical and Computer Engineering from Georgia Institute of Technology. She is currently pursuing a Ph.D. at the GT Human-Automation Systems (HumAnS) Lab. Her research interests include engagement, educational robotics, and socially interactive robots.

visit author page

author page

Ayanna M. Howard Georgia Institute of Technology

Download Paper | Permalink

Abstract

A Real-Time Model to Assess Student Engagement during Interaction with Intelligent Educational AgentsAdaptive learning is an educational method that utilizes computers as an interactive teachingdevice. Intelligent tutoring systems, or educational agents, use adaptive learning techniques toadapt to each student’s needs and learning styles in order to individualize learning. Effectiveeducational agents should accomplish two essential goals during the learning process – 1)monitor engagement of the student during the interaction and 2) apply behavioral strategies tomaintain the student’s attention when engagement decreases. In this paper, we focus on the firstobjective of monitoring student engagement. Most educational agents do not monitorengagement explicitly, but rather assume engagement and adapt their interaction based on thestudent’s responses to questions and tasks. A few advanced methods have begun to incorporatemodels of engagement through vision-based algorithms that assess behavioral cues such as eyegaze, head pose, gestures, and facial expressions. Unfortunately, these methods require a heavycomputation load, memory/storage constraints, and high power consumption. In addition, thesebehavioral cues do not correlate well with achievement of high-cognitive tasks, as we willdiscuss in this paper. As an alternative, our proposed model of engagement uses physical events,such as keyboard and mouse events. This approach requires fewer resources and lower powerconsumption, which is also ideally suited for mobile educational agents such as handheld tabletsand robotic platforms.In this paper, we discuss our engagement model which uses techniques that determine behavioraluser state and correlate these findings to mouse and keyboard events. In particular, we observethree event processes: total time required to answer a question; accuracy of responses; and properfunction executions. We evaluate the correctness of our model based on an investigationinvolving a middle-school after-school program in which a 15-question math exam that increasedin cognitive difficulty is used for assessment. The first 5 questions of the exam are categorized assimple 1-step problems; the next 5 questions require multiple steps; the last 5 questions arecategorized as very difficult (i.e. requiring a high-cognitive load). Eye gaze and head posetechniques are referenced for the baseline metric of engagement. We then conclude theinvestigation with a survey to gather the subject’s perspective of their mental state throughout theexam.We found that our model of engagement is comparable to the eye gaze and head pose techniques.When high-level cognitive thinking is required, our model is more accurate than the eye gaze andhead pose techniques due to the use of outside variables for assistance and non-focused gazesduring questions requiring deep thought. The large time delay associated with the lack of eyecontact between the student and the computer screen causes the aforementioned algorithms toincorrectly declare the subjects as being disengaged. Furthermore, speed and validity ofresponses can help to determine how well the student understands the material, and this isconfirmed through the survey responses and video observations. This additional information willbe used in the future to better integrate instructional scaffolding and adaptation with theeducational agent.

Citation
Format

Brown, L. N., & Howard, A. M. (2014, June), A Real-Time Model to Assess Student Engagement during Interaction with Intelligent Educational Agents Paper presented at 2014 ASEE Annual Conference & Exposition, Indianapolis, Indiana. 10.18260/1-2--19987

TY - CPAPER
AB - A Real-Time Model to Assess Student Engagement during Interaction with Intelligent Educational AgentsAdaptive learning is an educational method that utilizes computers as an interactive teachingdevice. Intelligent tutoring systems, or educational agents, use adaptive learning techniques toadapt to each student’s needs and learning styles in order to individualize learning. Effectiveeducational agents should accomplish two essential goals during the learning process – 1)monitor engagement of the student during the interaction and 2) apply behavioral strategies tomaintain the student’s attention when engagement decreases. In this paper, we focus on the firstobjective of monitoring student engagement. Most educational agents do not monitorengagement explicitly, but rather assume engagement and adapt their interaction based on thestudent’s responses to questions and tasks. A few advanced methods have begun to incorporatemodels of engagement through vision-based algorithms that assess behavioral cues such as eyegaze, head pose, gestures, and facial expressions. Unfortunately, these methods require a heavycomputation load, memory/storage constraints, and high power consumption. In addition, thesebehavioral cues do not correlate well with achievement of high-cognitive tasks, as we willdiscuss in this paper. As an alternative, our proposed model of engagement uses physical events,such as keyboard and mouse events. This approach requires fewer resources and lower powerconsumption, which is also ideally suited for mobile educational agents such as handheld tabletsand robotic platforms.In this paper, we discuss our engagement model which uses techniques that determine behavioraluser state and correlate these findings to mouse and keyboard events. In particular, we observethree event processes: total time required to answer a question; accuracy of responses; and properfunction executions. We evaluate the correctness of our model based on an investigationinvolving a middle-school after-school program in which a 15-question math exam that increasedin cognitive difficulty is used for assessment. The first 5 questions of the exam are categorized assimple 1-step problems; the next 5 questions require multiple steps; the last 5 questions arecategorized as very difficult (i.e. requiring a high-cognitive load). Eye gaze and head posetechniques are referenced for the baseline metric of engagement. We then conclude theinvestigation with a survey to gather the subject’s perspective of their mental state throughout theexam.We found that our model of engagement is comparable to the eye gaze and head pose techniques.When high-level cognitive thinking is required, our model is more accurate than the eye gaze andhead pose techniques due to the use of outside variables for assistance and non-focused gazesduring questions requiring deep thought. The large time delay associated with the lack of eyecontact between the student and the computer screen causes the aforementioned algorithms toincorrectly declare the subjects as being disengaged. Furthermore, speed and validity ofresponses can help to determine how well the student understands the material, and this isconfirmed through the survey responses and video observations. This additional information willbe used in the future to better integrate instructional scaffolding and adaptation with theeducational agent.
AU - LaVonda N. Brown
AU - Ayanna M. Howard
CY - Indianapolis, Indiana
DA - 2014/06/15
PB - ASEE Conferences
TI - A Real-Time Model to Assess Student Engagement during Interaction with Intelligent Educational Agents
UR - https://strategy.asee.org/19987
DO - 10.18260/1-2--19987
ER -