Asee peer logo

Work in Progress: Finding the Right Questions: Using Data Science to Close the Loop with Classroom Response Systems

Download Paper |

Conference

2019 ASEE Annual Conference & Exposition

Location

Tampa, Florida

Publication Date

June 15, 2019

Start Date

June 15, 2019

End Date

June 19, 2019

Conference Session

ERM Technical Session 6: Technology-enhanced Instruction and Assessment

Tagged Division

Educational Research and Methods

Page Count

13

DOI

10.18260/1-2--33619

Permanent URL

https://strategy.asee.org/33619

Download Count

327

Request a correction

Paper Authors

biography

Asuman Cagla Acun Sener University of Louisville

visit author page

Asuman Cagla Acun Sener holds B.S. and M.S. degrees in Computer Science and Computer Engineering. She is currently pursuing a doctoral degree in Computer Science at Knowledge Discovery & Web Mining Lab, Department of Computer Science and Computer Engineering, University of Louisville. She is also working as a graduate assistant. Her research interests are educational data mining, visualization, predictive modeling and recommender systems.

visit author page

biography

Olfa Nasraoui University of Louisville

visit author page

Olfa Nasraoui is Professor of Computer Engineering and Computer Science, Endowed Chair of e-commerce, and the founding director of the Knowledge Discovery and Web Mining Lab at the University of Louisville. She received her Ph.D. in Computer Engineering and Computer Science from the University of Missouri-Columbia in 1999. From 2000 to 2004, she was an Assistant Professor at the University of Memphis. Her research activities include Data Mining/ Machine Learning, Web Mining, Information Retrieval and Personalization, in particular in problems involving large multiple domain, high dimensional data, such as text, transactions, and social network data. She is the recipient of the National Science Foundation CAREER Award, and the winner of two Best Paper Awards, a Best Paper Award in theoretical developments in computational intelligence at the Artificial Neural Networks In Engineering conference (ANNIE 2001) and a Best Paper Award at the Knowledge Discovery and Information Retrieval conference in Seville, Spain (KDIR 2018). She has more than 200 refereed publications, including over 47 journal papers and book chapters and 12 edited volumes. Her research has been funded notably by NSF and NASA. Between 2004 and 2008, she has co-organized the yearly WebKDD workshops on User Profiling and Web Usage Mining at the ACM KDD conference. She has served on the program committee member, track chair, or senior program committee of several Data mining, Big Data, and Artificial Intelligence conferences, including ACM KDD, WWW, RecSys, IEEE Big Data, ICDM, SDM, AAAI, etc. In summer 2015, she served as Technical Mentor/Project Lead at the Data Science for Social Good Fellowship, in the Center for Data Science and Public Policy at the University of Chicago. She is a member of ACM, ACM SigKDD, senior member of IEEE and IEEE-WIE. She is also on the leadership team of the Kentucky Girls STEM collaborative network.

visit author page

biography

Jeffrey Lloyd Hieb University of Louisville

visit author page

Jeffrey L. Hieb is an Associate Professor in the Department of Engineering Fundamentals at the University of Louisville. He graduated from Furman University in 1992 with degrees in Computer Science and Philosophy. After 10 years working in industry, he returned to school, completing his Ph.D. in Computer Science Engineering at the University of Louisville’s Speed School of Engineering in 2008. Since completing his degree, he has been teaching engineering mathematics courses and continuing his dissertation research in cyber security for industrial control systems. In his teaching, Dr. Hieb focuses on innovative and effective use of tablets, digital ink, and other technology and is currently investigating the use of the flipped classroom model and collaborative learning. His research in cyber security for industrial control systems is focused on high assurance field devices using microkernel architectures.

visit author page

Download Paper |

Abstract

This is a work in progress paper. Classroom response systems (CRS) are educational technology tools that when paired with an appropriate pedagogy, provide increased classroom engagement in support improved teaching and learning. They do this by leveraging technology to allow every student to respond to instructor posed questions. Many of these systems, such as Learning Catalytics and clickers, collect and store a wealth of individual student response data that is aggregated to provide instructors with real time student response data. One open challenge in this setting is how to glean insights from all of the the collected response data to identify activities, specific questions, and combinations of questions that associate with student performance. A data driven analysis of student response data collected by a CRS combined with student performance data will enable individual instructors to refine and adapt their use of a CRS, thus closing the loop from an instructional design perspective. This paper presents a data science methodology and preliminary results of analyzing CRS data accumulated from daily activities in two sections of an engineering mathematics course. The data is collected from the CRS Learning Catalytics where students respond to questions in two rounds following the team-based learning model. In the first round, students answer questions individual; in the second round, they answer the same questions as a team after reviewing each other’s answers from round 1. The CRS stores each student’s responses from both rounds along with a timestamp. This study serves two purposes, 1) examine the effect of disagreement between individual responses on student performance and 2) identify which activity question scores, individual versus team-based, are associated with better exam performance, thus allowing the reduction of the number of questions. Using a methodology based on data mining and data exploration, we found three trends between round 1 and round 2 scores: students who have no score change in both rounds, students who increase their scores, and student who decrease their scores after the team-based collaboration. Our plan is to cluster class activity questions depending on the predicted performance, thus paving the way toward an improved data driven design of in class activities.

Acun Sener, A. C., & Nasraoui , O., & Hieb, J. L. (2019, June), Work in Progress: Finding the Right Questions: Using Data Science to Close the Loop with Classroom Response Systems Paper presented at 2019 ASEE Annual Conference & Exposition , Tampa, Florida. 10.18260/1-2--33619

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2019 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015