REU Program Evaluation: A Valuable Tool for Studying Undergraduate Socialization in Engineering

Caitlin D. Wylie; Kent A. Wayland; Andy Wang

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: July 12, 2024
Conference Session: Educational Research and Methods Division (ERM) Technical Session 18
Tagged Division: Educational Research and Methods Division (ERM)
Tagged Topic: Diversity
Permanent URL: https://peer.asee.org/47947

Request a correction

Paper Authors

biography

Caitlin D. Wylie University of Virginia orcid.org/0000-0002-0214-7837

visit author page

Caitlin D. Wylie is an associate professor of Science, Technology and Society in the University of Virginia's School of Engineering and Applied Science.

visit author page

biography

Kent A. Wayland University of Virginia

visit author page

Kent earned his PhD in Anthropology at the University of Virginia and is now an Assistant Professor in the Department of Engineering and Society, School of Engineering and Applied Science, at the University of Virginia

visit author page

biography

Andy Wang University of Virginia

visit author page

Andy Wang is an undergraduate student at the University of Virginia.

visit author page

Download Paper | Permalink

Abstract

This Work in Progress paper introduces a project investigating whether and how a summer research experience relates to undergraduate students’ sense of identity and belonging in engineering, understanding of research as a process, and research-related academic and professional skills. We draw from theories of situated learning and socialization into professional communities to ask what and how students learn during an NSF-funded Research Experience for Undergraduates (REU) summer program in materials science and engineering.

In addition, we propose that REU program evaluation data can offer valuable insights into student learning, but that these data are rarely analyzed with regards to research questions. Typically, they are collected, used to evaluate the extent to which a program is meeting its goals for internal improvement and to satisfy funders’ requirements, and then discarded. This is a missed opportunity. Furthermore, REU sites typically evaluate their programs with quantitative surveys, even though each site tends to serve only a few dozen students. Surveys designed for large-scale participant pools cannot capture nuances of students’ experiences, especially with comparatively low participant numbers. As a result, the effects of research experience on these students’ learning and identity are difficult for engineering education researchers to access. This is particularly problematic because many REU sites are designed to serve marginalized populations in engineering and science. Not studying these students’ experiences because their sample size is small is inequitable, and contributes to the existing knowledge gap about marginalized populations’ experiences and success in engineering. Designing evaluation methods to also produce rich data for research on these small student cohorts is a powerful way to address this inequity and provide important insights into student learning and identity formation.

As the program evaluators for a three-year REU site, we pushed the boundaries of traditional program evaluation to generate data that can also be used to address research questions, in addition to conducting cumulative and summative evaluation of the program’s effectiveness. To access broader questions about engineering identity, belonging, and comprehension of knowledge production practices among undergraduates, we designed new quantitative and qualitative instruments. These instruments are pre/post surveys and interviews that draw from existing instruments commonly used for REU evaluation (e.g., the SURE, the URSSA, the URES) as well as instruments that capture students’ views of the nature of science. In this paper, we share our design process for these instruments, our research methodology (including how we achieved IRB approval for evaluation data), and preliminary results from one summer cohort’s survey and interview responses.

Citation
Format

Wylie, C. D., & Wayland, K. A., & Wang, A. (2024, June), REU Program Evaluation: A Valuable Tool for Studying Undergraduate Socialization in Engineering Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/47947

TY - CPAPER
AB - This Work in Progress paper introduces a project investigating whether and how a summer research experience relates to undergraduate students’ sense of identity and belonging in engineering, understanding of research as a process, and research-related academic and professional skills. We draw from theories of situated learning and socialization into professional communities to ask what and how students learn during an NSF-funded Research Experience for Undergraduates (REU) summer program in materials science and engineering.

In addition, we propose that REU program evaluation data can offer valuable insights into student learning, but that these data are rarely analyzed with regards to research questions. Typically, they are collected, used to evaluate the extent to which a program is meeting its goals for internal improvement and to satisfy funders’ requirements, and then discarded. This is a missed opportunity. Furthermore, REU sites typically evaluate their programs with quantitative surveys, even though each site tends to serve only a few dozen students. Surveys designed for large-scale participant pools cannot capture nuances of students’ experiences, especially with comparatively low participant numbers. As a result, the effects of research experience on these students’ learning and identity are difficult for engineering education researchers to access. This is particularly problematic because many REU sites are designed to serve marginalized populations in engineering and science. Not studying these students’ experiences because their sample size is small is inequitable, and contributes to the existing knowledge gap about marginalized populations’ experiences and success in engineering. Designing evaluation methods to also produce rich data for research on these small student cohorts is a powerful way to address this inequity and provide important insights into student learning and identity formation.

As the program evaluators for a three-year REU site, we pushed the boundaries of traditional program evaluation to generate data that can also be used to address research questions, in addition to conducting cumulative and summative evaluation of the program’s effectiveness. To access broader questions about engineering identity, belonging, and comprehension of knowledge production practices among undergraduates, we designed new quantitative and qualitative instruments. These instruments are pre/post surveys and interviews that draw from existing instruments commonly used for REU evaluation (e.g., the SURE, the URSSA, the URES) as well as instruments that capture students’ views of the nature of science. In this paper, we share our design process for these instruments, our research methodology (including how we achieved IRB approval for evaluation data), and preliminary results from one summer cohort’s survey and interview responses.
AU - Caitlin D. Wylie
AU - Kent A. Wayland
AU - Andy Wang
CY - Portland, Oregon
DA - 2024/06/23
PB - ASEE Conferences
TI - REU Program Evaluation: A Valuable Tool for Studying Undergraduate Socialization in Engineering
UR - https://peer.asee.org/47947
ER -