An Initial Investigation of Design Cohesion as a IDE-based Learning Analytic for Measuring Introductory Programming Metacognition
- Conference
- Location
-
Portland, Oregon
- Publication Date
-
June 23, 2024
- Start Date
-
June 23, 2024
- End Date
-
July 12, 2024
- Conference Session
- Tagged Division
-
Computers in Education Division (COED)
- Tagged Topic
-
Diversity
- Permanent URL
-
https://peer.asee.org/46561
Paper Authors
Phyllis J. Beck
Mississippi State University
orcid.org/0009-0000-8699-5771
Phyllis Beck is a blend of art and science having completed an undergraduate degree in Fine Arts at MSU and a Ph.D. in Computer Science, where she focused on applying Artificial Intelligence, Natural language Processing, and Machine Learning techniques to the engineering education space. Currently, she is working as an Assistant Research Professor at Mississippi State University in the Bagley College of Electrical and Computer Engineering. She has worked for companies such as the Air Force Research Laboratory in conjunction with Oak Ridge National Labs and as an R & D Computer Science Inter for Sandia National Labs conducting Natural Language Processing and AI research and was inducted into the Bagley College of Engineering Hall of Fame in 2021.
Mahnas Jean Mohammadi-Aragh
Mississippi State University
orcid.org/0000-0002-3094-3734
Jean Mohammadi-Aragh is the Director of Bagley College of Engineering Office of Inclusive Excellence and Associate Professor in the Department of Electrical and Computer Engineering at Mississippi State University. Through her interdependent roles in research, teaching, and service, Jean is actively breaking down academic and social barriers to foster an environment where diverse and creative people are successful in the pursuit of engineering and computing degrees. Jean’s efforts have been recognized with numerous awards including the National Science Foundation Faculty Early Career Development award, the American Society for Engineering Education John A. Curtis Lecturer award, and the Bagley College of Engineering Service award. Jean earned her B.S. and M.S. in computer engineering from Mississippi State University, and her Ph.D. in engineering education from Virginia Tech.
Abstract
In this full paper, we describe our case study approach to initially characterize Design Cohesion as a new IDE-based learning metrics through an exploratory coding process. We developed a new alignment notation to generate two new qualitative metrics: Design Cohesion (High, Medium, Low) and Granularity Level (High, Medium, Low) Design Cohesion is the level of alignment between flowchart and code, which accounts for the order of intended program execution, the internal data of a flowchart node, and the number of nodes in the flowchart that map to the code. We define Granularity Level as an additional characterization of cohesion that labels the level of detail in the flowchart. Our primary objective is to use these new metrics to understand how a flowchart can be aligned with its code implementation to understand introductory students’ current level of programming metacognition.
In the context of our case study, we discuss the exploratory coding process and alignment notation developed to generate the features for the newly proposed metrics. Next, we explore two cases to illustrate the diversity of characteristics found in various feature combinations. Each case study compares two examples from the same participant, one with High Cohesion and High Granularity, the other with High Cohesion and Low Granularity. Our initial investigation into design cohesion has led to the hypothesis that High-level Design Cohesion paired with low levels of flowchart granularity demonstrates high levels of abstraction in the initial flowchart design, which may point to under-designing by participants and/or lower levels of metacognition. Comparatively, having high cohesion and granularity may point to over-designing by the participant and often stems from a one-to-one mapping of flowchart nodes to lines of code. Our results point toward a logical relationship between Design Cohesion and students’ level of self-estimated skill, and we are confident that Design Cohesion will serve as viable metric for understanding introductory programming metacognition.
Beck, P. J., & Mohammadi-Aragh, M. J. (2024, June), An Initial Investigation of Design Cohesion as a IDE-based Learning Analytic for Measuring Introductory Programming Metacognition Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/46561
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2024 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015