Individual Design Experiences Improve Students’ Self-Efficacy on Team-Based Engineering Design Projects

Amy Trauth; Marcia Gail Headley; Sara Grajeda; Dustyn Roberts; Jenni Buckley

Download Paper | Permalink

Conference: 2020 ASEE Virtual Annual Conference Content Access
Location: Virtual On line
Publication Date: June 22, 2020
Start Date: June 22, 2020
End Date: June 26, 2021
Conference Session: Design Teams 1
Tagged Division: Design in Engineering Education
Tagged Topic: Diversity
Page Count: 15
DOI: 10.18260/1-2--34821
Permanent URL: https://strategy.asee.org/34821
Download Count: 369

Request a correction

Paper Authors

biography

Amy Trauth University of Delaware orcid.org/0000-0002-5146-592X

visit author page

Amy Trauth, Ph.D., is the Senior Associate Director of Science Education at the University of Delaware's Professional Development Center for Educators. In her role, Amy works collaboratively with K-12 science and engineering teachers to develop and implement standards-based curricula and assessments. She also provides mentoring and coaching and co-teaching support to K-12 teachers across the entire trajectory of the profession. Her research focuses on teacher education, classroom assessment, and P-16 environmental and engineering education.

visit author page

biography

Marcia Gail Headley University of Delaware orcid.org/0000-0003-3017-2834

visit author page

Dr. Headley is a Research Associate at the Center for Research in Education and Social Policy (CRESP) at the University of Delaware. She specializes in the development of mixed methods research designs and strategies for integrating quantitative and qualitative research approaches. Her work has been published in the prestigious Journal of Mixed Method Research. In her current role, she uses her methodological expertise to support a variety of CRESP projects. Dr. Headley is devoted to designing effective research studies with the potential to generate well-justified answers to complex questions about how students learn given variations in their health, homes, classrooms, and schools.

visit author page

biography

Sara Grajeda University of Delaware

visit author page

Dr. Grajeda's research interests lie in applied measurement work and policy analyses in education and public health areas. Her measurement work has involved developing and analyzing observational rubrics and surveys in both K12 and higher education settings in various content areas.

visit author page

biography

Dustyn Roberts P.E. University of Pennsylvania

visit author page

Dustyn is a Philadelphia-based engineer, Senior Lecturer at Penn, and co-founder of Sage Smart Garden, LLC. After an early career putting robots on mars and teaching engineering to artists, she now teaches engineering primarily at the undergraduate level with a focus on design, mechanics, materials, systems, and prototyping. Her research includes work in engineering education, entrepreneurial mindset, and developing new engineering educators. Dustyn received her B.S. in Mechanical and Biomedical Engineering (2003) from Carnegie Mellon University, her M.S. in Biomechanics & Movement Science (2004) from the University of Delaware, and her Ph.D. in Mechanical Engineering (2014) from New York University.

visit author page

biography

Jenni Buckley University of Delaware

visit author page

Dr. Buckley is an Associate Professor of Mechanical Engineering at University of Delaware. She received her Bachelor’s of Engineering (2001) in Mechanical Engineering from the University of Delaware, and her MS (2004) and PhD (2006) in Mechanical Engineering from the University of California, Berkeley, where she worked on computational and experimental methods in spinal biomechanics. Since 2006, her research efforts have focused on the development and mechanical evaluation of medical and rehabilitation devices, particularly orthopaedic, neurosurgical, and pediatric devices. She teaches courses in design, biomechanics, and mechanics at University of Delaware and is heavily involved in K12 engineering education efforts at the local, state, and national levels.

visit author page

Download Paper | Permalink

Abstract

This Complete Research study involves a case-controlled investigation of Individual Design Experiences (IDEs) to improve students' self-efficacy in team-based engineering design activities.

Team-based projects are widely used in engineering courses, particularly product or process design courses in disciplines such as mechanical, chemical, civil, and biomedical engineering. While the intention of team-based design projects is to provide all students with a diversity of technical and non-technical mastery experiences, students enter into these experiences with differences - whether real or perceived – in relevant technical skills that undermine individuals’ learning objectives on team-based work. Prior research by our group and others indicates that majority (white) male engineering students are more confident than females and traditionally under-represented minorities (URMs) in their math and science abilities, open-ended problem solving, and hands-on prototyping skills. These disparities lead to behavioral differences on team-based projects that in turn reinforce students’ beliefs about their own skill set and others’, often on a gendered or racial basis.

Mastery experiences, which are task-related experiences that culminate in a performance accomplishment, are a well-established mechanism to bolster students’ self-efficacy. Mastery experiences have been studied previously in undergraduate engineering settings, frequently in the context of self-paced mini-modules and micro-certifications; however, the purpose of these studies have been to boost a student’s performance in individual rather than team-based tasks. The purpose of this study is to evaluate the strategic use of specially designed mastery experiences, which we term Individual Design Experiences (IDEs), embedded within team-based engineering design challenges. We posit that the use of IDEs will mitigate disparities in self-efficacy for hands-on prototyping tasks conducted in team settings, which have been shown to disproportionately persist for women and URMs.

A cohort-based, mixed methods study design was used to determine whether IDEs affect student self-efficacy for hands-on prototyping tasks. The setting for the IRB-approved study was the first mechanical engineering design course taken by all mechanical engineering majors at a mid-sized (ca. 150 students/year), ABET-accredited program at a US land grant university. The course was taught by a single professor in multiple sections, with students in half of the sections receiving the IDE (n=81 total) and the other half as control (n=50 total). The IDE consisted of a single, two-week assignment during which students were tasked with designing and building a child’s pull-toy using introductory-level carpentry techniques, e.g., drill, saw, router. The course also had a semester-long (13-week) team-based design project that involved similar design and prototyping techniques. The control group completed both the pull-toy and semester-long projects in their assigned teams, while the IDE treatment group completed the pull-toy individually and the semester-long project in their assigned teams. Self-efficacy was assessed pre and post-course with a validated instrument developed and previously reported by our team that included five factors that encompass most skills necessary for team-based engineering design, namely: (1) Math and Science Skills; (2) Engineering Application; (3) Professional and Interpersonal Skills; (4) Hands-On Prototyping (“Tinkering”); and (5) Open-Ended Problem Solving. Ordinary Least Squares (OLS) regression was used separately for each of the five factors to predict post-treatment scores on treatment assignment, while controlling for students’ pre-treatment scores. Focus group interviews were also conducted post-course, with responses subjected to thematic analysis consistent with the survey instrument.

Students in the IDE treatment group showed greater gains in self-efficacy for Hands-On Prototyping than those in the control group (p=0.01, d=0.37 on 0-4 pt factor scale). Gains were isolated to this one factor, although there was a modest but not statistically significant increase in Math and Science Abilities for IDE vs. Control (d=0.25, p=0.06). Due to loss of subjects to follow-up (pre vs. post-course response drop-off), the study was under-powered to disaggregate self-efficacy gains by gender and race. Results of the focus group interviews suggested that students of all races and genders who had little prototyping experience appreciated the individual mastery experience with carpentry skills, while there was no indication that those with prior experiences were adversely affected.

Our study clearly shows that IDEs can lead to gains in self-efficacy that persist during team-based activity. Given that our IDE intervention was focused almost exclusively on hands-on prototyping, it is not surprising that the effects were more pronounced for this factor. Although our study was ultimately under-powered to definitively determine whether IDE treatment is particularly beneficial for women and URMs, our quantitative and qualitative results strongly suggest that IDEs mitigate disparities in prototyping self-efficacy due to prior experience, which differs for majority vs. under-represented populations in engineering. These results suggest that IDEs may be an effective intervention to address disparities in self-efficacy that otherwise persist throughout or are exacerbated by team-based design experiences.

Citation
Format

Trauth, A., & Headley, M. G., & Grajeda, S., & Roberts, D., & Buckley, J. (2020, June), Individual Design Experiences Improve Students’ Self-Efficacy on Team-Based Engineering Design Projects Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line . 10.18260/1-2--34821

TY - CPAPER
AB - This Complete Research study involves a case-controlled investigation of Individual Design Experiences (IDEs) to improve students&#39; self-efficacy in team-based engineering design activities.

Team-based projects are widely used in engineering courses, particularly product or process design courses in disciplines such as mechanical, chemical, civil, and biomedical engineering. While the intention of team-based design projects is to provide all students with a diversity of technical and non-technical mastery experiences, students enter into these experiences with differences - whether real or perceived – in relevant technical skills that undermine individuals’ learning objectives on team-based work. Prior research by our group and others indicates that majority (white) male engineering students are more confident than females and traditionally under-represented minorities (URMs) in their math and science abilities, open-ended problem solving, and hands-on prototyping skills. These disparities lead to behavioral differences on team-based projects that in turn reinforce students’ beliefs about their own skill set and others’, often on a gendered or racial basis.

Mastery experiences, which are task-related experiences that culminate in a performance accomplishment, are a well-established mechanism to bolster students’ self-efficacy. Mastery experiences have been studied previously in undergraduate engineering settings, frequently in the context of self-paced mini-modules and micro-certifications; however, the purpose of these studies have been to boost a student’s performance in individual rather than team-based tasks. The purpose of this study is to evaluate the strategic use of specially designed mastery experiences, which we term Individual Design Experiences (IDEs), embedded within team-based engineering design challenges. We posit that the use of IDEs will mitigate disparities in self-efficacy for hands-on prototyping tasks conducted in team settings, which have been shown to disproportionately persist for women and URMs.

A cohort-based, mixed methods study design was used to determine whether IDEs affect student self-efficacy for hands-on prototyping tasks. The setting for the IRB-approved study was the first mechanical engineering design course taken by all mechanical engineering majors at a mid-sized (ca. 150 students/year), ABET-accredited program at a US land grant university. The course was taught by a single professor in multiple sections, with students in half of the sections receiving the IDE (n=81 total) and the other half as control (n=50 total). The IDE consisted of a single, two-week assignment during which students were tasked with designing and building a child’s pull-toy using introductory-level carpentry techniques, e.g., drill, saw, router. The course also had a semester-long (13-week) team-based design project that involved similar design and prototyping techniques. The control group completed both the pull-toy and semester-long projects in their assigned teams, while the IDE treatment group completed the pull-toy individually and the semester-long project in their assigned teams. Self-efficacy was assessed pre and post-course with a validated instrument developed and previously reported by our team that included five factors that encompass most skills necessary for team-based engineering design, namely: (1) Math and Science Skills; (2) Engineering Application; (3) Professional and Interpersonal Skills; (4) Hands-On Prototyping (“Tinkering”); and (5) Open-Ended Problem Solving. Ordinary Least Squares (OLS) regression was used separately for each of the five factors to predict post-treatment scores on treatment assignment, while controlling for students’ pre-treatment scores. Focus group interviews were also conducted post-course, with responses subjected to thematic analysis consistent with the survey instrument.

Students in the IDE treatment group showed greater gains in self-efficacy for Hands-On Prototyping than those in the control group (p=0.01, d=0.37 on 0-4 pt factor scale). Gains were isolated to this one factor, although there was a modest but not statistically significant increase in Math and Science Abilities for IDE vs. Control (d=0.25, p=0.06). Due to loss of subjects to follow-up (pre vs. post-course response drop-off), the study was under-powered to disaggregate self-efficacy gains by gender and race. Results of the focus group interviews suggested that students of all races and genders who had little prototyping experience appreciated the individual mastery experience with carpentry skills, while there was no indication that those with prior experiences were adversely affected.

Our study clearly shows that IDEs can lead to gains in self-efficacy that persist during team-based activity. Given that our IDE intervention was focused almost exclusively on hands-on prototyping, it is not surprising that the effects were more pronounced for this factor. Although our study was ultimately under-powered to definitively determine whether IDE treatment is particularly beneficial for women and URMs, our quantitative and qualitative results strongly suggest that IDEs mitigate disparities in prototyping self-efficacy due to prior experience, which differs for majority vs. under-represented populations in engineering. These results suggest that IDEs may be an effective intervention to address disparities in self-efficacy that otherwise persist throughout or are exacerbated by team-based design experiences.
AU - Amy Trauth
AU - Marcia Gail Headley
AU - Sara Grajeda
AU - Dustyn Roberts P.E.
AU - Jenni Buckley
CY - Virtual On line
DA - 2020/06/22
PB - ASEE Conferences
TI - Individual Design Experiences Improve Students’ Self-Efficacy on Team-Based Engineering Design Projects
UR - https://strategy.asee.org/34821
DO - 10.18260/1-2--34821
ER -