Defining an Evaluation Framework for Undergraduate Research Experiences
- Conference
- Location
-
Vancouver, BC
- Publication Date
-
June 26, 2011
- Start Date
-
June 26, 2011
- End Date
-
June 29, 2011
- ISSN
-
2153-5965
- Conference Session
- Tagged Division
-
Educational Research and Methods
- Page Count
-
31
- Page Numbers
-
22.419.1 - 22.419.31
- DOI
-
10.18260/1-2--17700
- Permanent URL
-
https://strategy.asee.org/17700
- Download Count
-
403
Paper Authors
Lisa Massi University of Central Florida
Dr. Lisa Massi is the Director of Operations Analysis in the UCF College of Engineering & Computer Science. Her primary responsibilities include accreditation, assessment, and data administration. She is a Co-PI of the NSF-funded S-STEM program at UCF entitled the "Young Entrepreneur & Scholar (YES) Scholarship Program." Her research interests include program evaluation and predictors of career intentions.
Michael Georgiopoulos University of Central Florida
Michael Georgiopoulos is a Professor in the UCF Department of Electrical Engineering and Computer Science and the PI of the NSF-funded S-STEM program at UCF entitled the "Young Entrepreneur and Scholar (YES) Scholarship Program" as well as the NSF-funded STEP program entitled "EXCEL:UCF-STEP Pathways to STEM: From Promise to Prominence." Dr. Georgiopoulos' research interests lie in the areas of machine learning, neural networks, pattern
recognition and applications in signal/image processing, communications, medical field, manufacturing, transportation engineering, amongst others. Dr. Georgiopoulos is a Director of the
Machine Learning Laboratory at UCF.
Cynthia Y. Young University of Central Florida
Dr. Cynthia Y. Young is a Professor of Mathematics at the University of Central Florida where she holds a secondary joint appointment in the College of Optics and Photonics. She holds a B.A. in Mathematics Education from the University of North Carolina, M.S. in Mathematical Science from the University of Central Florida, and M.S. in Electrical Engineering and Ph.D. in Applied Mathematics from the University of Washington, Seattle, WA. In 1997 she joined the faculty at UCF where she conducts research in the area of atmospheric effects on laser propagation. She has worked on laser satellite communication projects and laser radar projects for the Boeing Company, the US Navy, and the US Air Force.
In 2001 Dr. Young was the recipient of the Young Investigator Award from the Office of Naval Research for a project entitled “Turbulence Effects on Lidar”. She has collaborated with optical scientists from the Australian Defense, Science, and Technology Office on experiments in Adelaide Australia and Kennedy Space Center, Fl. In 2005 she did a sabbatical at the Naval Research Laboratory in which both theoretical and experimental studies were conducted with NRL scientists and engineers. In 2007 Dr. Young was named a fellow of the International Society for Optical Engineers. Dr. Young has received the UCF Research Incentive Award, Teaching Incentive Award, and Scholarship of Teaching and Learning award. Dr. Young is currently the co-director of the UCF EXCEL program.
Cherie Geiger University of Central Florida
Dr. Cherie Geiger is an Associate Professor of Chemistry at the University of Central Florida. She earned a B.S. in Chemistry from University of Central Florida in 1990 and a Ph.D. in Inorganic and Environmental Chemistry from the University of South Florida in 1994 and joined the faculty at UCF in 1996. Dr. Geiger has won several national and international awards including the Tech Museum of Innovation Intel Environmental Laureate, the Government Invention of the Year Award, and the Government Commercial Invention of the Year Award. She and her research team were inducted into the Space Technology Hall of Fame in 2007 for their work with environmental remediation of groundwater and aquifer materials. She has given over 150 professional presentations and has over 80 scientific publications. She served as President of the Florida Academy of Sciences from 2003 - 2005 and as Associate Chair of the Chemistry Department from 2005 - 2008. Her teaching accomplishments have been acknowledged by winning the Teaching Incentive Program Award at UCF twice. Dr. Geiger currently has seven doctoral students and two undergraduate student working in her laboratories on several environmental chemistry research studies.
Patrice Lancey University of Central Florida
Dr. Patrice Lancey is the Director of the UCF Office of Operational Excellence and Assessment Support. She holds a B.A. in Psychology from Brooklyn College and an M.A. and Ph.D. in Clinical Psychology from Wayne State University. She joined the University of Central Florida in 2001 where she coordinates the university wide Institutional Effectiveness Assessment process and supports assessment of academic programs and administrative departments. She also designs statistical studies to provide information about student engagement, institutional conditions that enhance student learning outcomes, progression, and retention to provide reports to decision makers to include upper administration, faculty and staff. Dr. Lancey serves as the outside evaluator for the NSF STEP funded program “EXCEL: UCF STEP Pathways to STEM: From Promise to Prominence” and as an assessment specialist for NSF S-STEM program entitled, “Young Entrepreneur and Scholar (YES) Scholarship Program.” Prior to this, she held positions at The Johns Hopkins School of Public Health, Westat, Inc., University of Alabama, and Palm Beach Community College. She presents papers and workshops for faculty and administrators on educational assessment topics at national and regional conferences and acts as a consultant to other universities. Dr. Lancey serves as a reviewer for proposals in the area of assessment for the Association for Institutional Research and Southern Association for Institutional Research. She regularly works with faculty to develop research methodology and student learning assessment.
Divya Bhati University of Central Florida
Divya Bhati serves as Assistant Director of Operational Excellence and Assessment Support where she supports all assessment efforts for academic programs and administrative units. Dr. Bhati is an assessment specialist for the the NSF-funded S-STEM program at UCF entitled the "Young Entrepreneur and Scholar (YES) Scholarship Program" as well as the NSF-funded STEP program entitled "EXCEL:UCF-STEP Pathways to STEM: From Promise to Prominence." Dr. Bhati's research interests include assessment of student learning outcomes, teaching and learning, survey design and research methodology, application of technology in classroom settings, and research related to human performance.
Abstract
Defining an Evaluation Framework for Undergraduate Research ExperiencesAbstractThe literature on undergraduate research experiences (UREs) makes, in many instances,unsubstantiated claims (e.g., claims based on small sample sizes). Furthermore, quite often thereis a lack of a control group to make comparisons with, data on how students develop a sense ofidentity, and the non-cognitive benefits gained by students (Seymour et al., 2004).In an effort to improve the evaluation framework for the authors’ NSF-funded S-STEM program(Scholarships in Science, Technology, Engineering, and Mathematics) that supports researchexperiences for its participating students, we analyzed quantitative and qualitative data from the2009-10 senior exit survey for graduating students in the College of Engineering and ComputerScience at a large, metropolitan, research university. The survey data offered a large sample size(n=687) of which 104 or 15% of students reported participation in a URE; non-URE studentswere used as a control group. First, we looked for patterns in the data that would provide someinsight into three hypothesized claims (see below) for our NSF program, which we called theYoung Entrepreneur and Scholar (YES) scholarship program. We then compared our findingswith a study of 76 rising seniors in eight science disciplines at four liberal arts schools bySeymour et al. (2004) and a study of 1,135 students (primarily in engineering and the sciences)surveyed at 41 institutions by Lopatto (2004).Claim 1. The URE is one pathway by which students explore their sense of “becoming” i.e.,establishing a career identity which is often inextricably bound up with personal identity (Zunkeras cited in Okocha, 2008), .Forty three percent of our URE students expressed interest in continuing on to graduate schoolcompared with 17% of non-URE students. In our survey, we found 41% of gains in“clarification or confirmation of career/education paths” compared with 20% of gains reported inthe Seymour et al. study. The difference is probably attributable to the timing of both studies:we surveyed graduating seniors whereas Seymour interviewed rising seniors.Claim 2. The URE is a transformational experience (Hunter et al., 2007).Fifty one percent of gains in the personal/professional domain (which includes “thinking andworking like a scientist”) were found in both our study and the Seymour study.Claim 3. The URE is an attractive proposition to recruit and retain under-represented groups.In our survey data, males and females showed the same level of interest in continuing on tograduate school as did Lopatto (2004). In our data, Caucasians and Hispanics showed higherlevels of interest in continuing on to graduate school whereas Lopatto (2004) found nodifferences among ethnic groups. This finding may be unique to our institution.The comparative analysis of our data with the two studies reaffirms the hypothesized claims anddefines an appropriate evaluation framework upon which our future YES program assessmentswill rely. We believe these assessments will be stronger since we expect to recruit two moreYES cohorts of 25 URE students (currently, 23 students are in the Research Path). ReferencesHunter, A, Laursen, S. L., Seymour, E. (2007). Becoming a scientist: The role of undergraduate research in students’ cognitive, personal, and professional development. Science Education 91(1), 36-74.Lopatto, D. (2004, Winter). Survey of undergraduate research experiences (SURE): First findings. Cell Biology Education 3, 270-277.Okocha, A.A.G. (2008, July). Racial ethnic identity and career development concerns of college students from immigrant African and Hmong families. Paper based on a program presented at the National Career Development Association Global Conferences, July 9- 11, 2008, Washington, D.C.Seymour, E., Hunter, A., Laursen, S.L., & Deantoni, T. (2004). Establishing the benefits of research experiences for undergraduates in the sciences: First findings from a three-year study. Science Education 88(4), 493-534.
Massi, L., & Georgiopoulos, M., & Young, C. Y., & Geiger, C., & Lancey, P., & Bhati, D. (2011, June), Defining an Evaluation Framework for Undergraduate Research Experiences Paper presented at 2011 ASEE Annual Conference & Exposition, Vancouver, BC. 10.18260/1-2--17700
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2011 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015