Development and Application of a Questionnaire to Measure Student Attitudes Toward and Understanding of Engineering

Jan DeWaters; John C. Moosbrugger; Pankaj Sharma

Download Paper | Permalink

Conference: 2017 ASEE Annual Conference & Exposition
Location: Columbus, Ohio
Publication Date: June 24, 2017
Start Date: June 24, 2017
End Date: June 28, 2017
Conference Session: The Best of First-Year Programs Division
Tagged Division: First-Year Programs
Tagged Topic: Diversity
Page Count: 19
DOI: 10.18260/1-2--28153
Permanent URL: https://peer.asee.org/28153
Download Count: 614

Request a correction

Paper Authors

biography

Jan DeWaters Clarkson University

visit author page

Jan DeWaters is an Assistant Professor in the Wallace H. Coulter School of Engineering at Clarkson University, in Potsdam, New York. She is part of the development team for Clarkson’s First Year Engineering/Interdisciplinary course that was the motivation for the work that is described in this paper. Her current research interests include the implementation and evaluation of evidence-based effective learning practices in STEM education, environmental education, and energy education.

visit author page

biography

John C. Moosbrugger Clarkson University

visit author page

John C. Moosbrugger, PhD, is a Professor of Mechanical and Aeronautical Engineering and Associate Dean for Academic Programs for the Wallace H. Coulter School of Engineering at Clarkson University.

visit author page

biography

Pankaj Sharma Clarkson University

visit author page

Graduate Student, M.S. Data Analytics

visit author page

Download Paper | Permalink

Abstract

Abstract –

Motivation and Background. This complete research paper will describe the development of a questionnaire that has been applied to measure student attitudes toward and understanding of studies and careers in engineering. The work was motivated by the desire to gain a better understanding of our students’ sense of fit, or belonging, within the engineering profession, and to monitor changes among our first year engineering (FYE) students as they encounter the rigors of the first year program.

A course was designed and implemented in Fall 2011 to engage FYE students with engineering faculty and expose them to topics that demonstrate the confluence of engineering practice with societal needs. The course, Engineering and Society, became part of the required FYE core curriculum in Fall 2014. Prior to its introduction all FYE students were enrolled in a 2-course sequence in Calculus, Physics, and Chemistry, two humanities/writing courses, and a computing course. Now, approximately half of all incoming FYE students delay Physics I, largely by virtue of their performance on a pre-enrollment math readiness exam. These students are scheduled into a fall section of Engineering and Society, while the remaining FYE cohort enrolls in spring semester.

Engineering and Society contains elements that are common among FYE courses such as the study of engineering disciplines, ethics, and a team-based design project, yet it uniquely focuses on the connections among engineering/technology and society and the development of technology within a societal context. This allows us to integrate ethics and the engineering design experience with the technology and society content, which provides a platform for analyzing current technological systems and exposes students to the breadth and diversity of engineering. Aside from meeting ABET and University-level outcomes, Engineering and Society strives to clarify students’ perceptions of the broad nature of engineering problem solving, and to positively impact their attitudes toward engineering studies and careers.

Since its inception in 2011 we have administered a self-constructed questionnaire to measure student attitudes before and after taking the course. Our findings have consistently shown a significant increase in students’ understanding of the broad nature of engineering and engineering problem solving, self-confidence with respect to engineering problem solving and design, and their sense of fit within the engineering profession. These increases were also significantly greater than those measured among the fall semester control group, consisting of freshmen enrolled in physics. In light of a significant body of research demonstrating, among a variety of reasons, the importance of students’ sense of belonging to their persistence in engineering studies and careers [e.g. 1,2], and – for women in particular – the evidence that negative perceptions toward engineering can be dispelled by framing engineering studies and careers within a broader context to demonstrate societal relevance [3,4], our preliminary findings have inspired us to (1) look more closely at male/female differences in our students’ experiences; and (2) subject our questionnaire to a rigorous psychometric analysis to ensure its validity and reliability, thereby making it more widely available to other institutions wishing to evaluate their student population. The second objective is the subject of this paper.

Methods/Assessment. The Engineering Attitudes Survey was developed largely using items that were adapted by permission from existing instruments [5,6,7,8], with additional items created to measure course objectives related to students’ understanding of the breadth of engineering and interactions with society. The questionnaire contains 22 items that use a Likert-type format with five options ranging from strongly disagree (1) to strongly agree (5).

The survey is being analyzed according to psychometric principles from educational and social psychology research. Preliminary analysis included an evaluation of the homogeneity/heterogeneity among student responses, as well as male/female differences, for each of the 22 survey items. Subsequent analysis will use the current database of responses in a series of statistical item- and instrument-analyses combined with a qualitative evaluation of each item’s contribution to the overall objectives of the survey and the relationship between items in various factor groupings.

Preliminary Results. Analysis of the survey is currently underway, using data from over 2000 FYE students collected over 10 semesters. Preliminary analysis of survey responses from the pre-college cohort indicates that it adequately captures the psychometric factors and the students’ sense of belonging, acceptance, and association with his/her choice of engineering as a potential field of study. In-depth analysis of the questionnaire is currently underway, and will be completed in time for the paper submission in February. It is anticipated that this in-depth analysis, and subsequent refinement of the questionnaire, will result in a valid, reliable instrument that we can then make available to other institutions as a measure for student characterization or for program evaluation.

References: [1] French, B.F., Immekus, J.C., and Oakes, W.C. (2005). An examination of indicators of engineering students’ success and persistence. Journal of Engineering Education, 94(4):419-425. [2] Clewell, B.C., and Campbell, P.B. (2002). Taking stock: Where we've been, where we are, where we're going. Journal of Women and Minorities in Science and Engineering, 8:255-284. [3] Isaacs, B. (2001). Mystery of the missing women engineers: A Solution. Journal of Professional Issues in Engineering Education and Practice, 127(2): p. 85-91. [4] University of Michigan (2005). Why Women Shy Away from Careers in Science and Math. University of Michigan Press Release, Thursday April 7, 2005. <http://www.newswise.com/articles/view/510997/>. [5] Tuan, H., Vhin, C., and Shich, S. (2005). The development of a questionnaire to measure students’ motivation toward science learning. International Journal of Science Education, 27(6): 639-654. [6] Rosenberg, M. (1965). Society and the adolescent self-image, Princeton University Press, Princeton, NJ, Instrument: Rosenberg Self-Esteem Scale. [7] APPLES (Academic Pathways of People Learning Engineering Survey), created by the CAEE (Center for the Advancement of Engineering Education) project and available online at http://caee-aps.stanford.edu/phpESP/admin/manage.php. [8] LAESE (Longitudinal Assessment of Engineering Self-Efficacy) survey versions 3.0 (copyright 2006) and 3.1 (copyright 2007), which are products of AWE (Assessing Women and Men in Engineering), available online at www.aweonline.org.

Citation
Format

DeWaters, J., & Moosbrugger, J. C., & Sharma, P. (2017, June), Development and Application of a Questionnaire to Measure Student Attitudes Toward and Understanding of Engineering Paper presented at 2017 ASEE Annual Conference & Exposition, Columbus, Ohio. 10.18260/1-2--28153

TY - CPAPER
AB - Abstract –

Motivation and Background. This complete research paper will describe the development of a questionnaire that has been applied to measure student attitudes toward and understanding of studies and careers in engineering. The work was motivated by the desire to gain a better understanding of our students’ sense of fit, or belonging, within the engineering profession, and to monitor changes among our first year engineering (FYE) students as they encounter the rigors of the first year program.

A course was designed and implemented in Fall 2011 to engage FYE students with engineering faculty and expose them to topics that demonstrate the confluence of engineering practice with societal needs. The course, Engineering and Society, became part of the required FYE core curriculum in Fall 2014. Prior to its introduction all FYE students were enrolled in a 2-course sequence in Calculus, Physics, and Chemistry, two humanities/writing courses, and a computing course. Now, approximately half of all incoming FYE students delay Physics I, largely by virtue of their performance on a pre-enrollment math readiness exam. These students are scheduled into a fall section of Engineering and Society, while the remaining FYE cohort enrolls in spring semester.

Engineering and Society contains elements that are common among FYE courses such as the study of engineering disciplines, ethics, and a team-based design project, yet it uniquely focuses on the connections among engineering/technology and society and the development of technology within a societal context. This allows us to integrate ethics and the engineering design experience with the technology and society content, which provides a platform for analyzing current technological systems and exposes students to the breadth and diversity of engineering. Aside from meeting ABET and University-level outcomes, Engineering and Society strives to clarify students’ perceptions of the broad nature of engineering problem solving, and to positively impact their attitudes toward engineering studies and careers.

Since its inception in 2011 we have administered a self-constructed questionnaire to measure student attitudes before and after taking the course. Our findings have consistently shown a significant increase in students’ understanding of the broad nature of engineering and engineering problem solving, self-confidence with respect to engineering problem solving and design, and their sense of fit within the engineering profession. These increases were also significantly greater than those measured among the fall semester control group, consisting of freshmen enrolled in physics. In light of a significant body of research demonstrating, among a variety of reasons, the importance of students’ sense of belonging to their persistence in engineering studies and careers [e.g. 1,2], and – for women in particular – the evidence that negative perceptions toward engineering can be dispelled by framing engineering studies and careers within a broader context to demonstrate societal relevance [3,4], our preliminary findings have inspired us to (1) look more closely at male/female differences in our students’ experiences; and (2) subject our questionnaire to a rigorous psychometric analysis to ensure its validity and reliability, thereby making it more widely available to other institutions wishing to evaluate their student population. The second objective is the subject of this paper.

Methods/Assessment. The Engineering Attitudes Survey was developed largely using items that were adapted by permission from existing instruments [5,6,7,8], with additional items created to measure course objectives related to students’ understanding of the breadth of engineering and interactions with society. The questionnaire contains 22 items that use a Likert-type format with five options ranging from strongly disagree (1) to strongly agree (5).

The survey is being analyzed according to psychometric principles from educational and social psychology research. Preliminary analysis included an evaluation of the homogeneity/heterogeneity among student responses, as well as male/female differences, for each of the 22 survey items. Subsequent analysis will use the current database of responses in a series of statistical item- and instrument-analyses combined with a qualitative evaluation of each item’s contribution to the overall objectives of the survey and the relationship between items in various factor groupings.

Preliminary Results. Analysis of the survey is currently underway, using data from over 2000 FYE students collected over 10 semesters. Preliminary analysis of survey responses from the pre-college cohort indicates that it adequately captures the psychometric factors and the students’ sense of belonging, acceptance, and association with his/her choice of engineering as a potential field of study. In-depth analysis of the questionnaire is currently underway, and will be completed in time for the paper submission in February. It is anticipated that this in-depth analysis, and subsequent refinement of the questionnaire, will result in a valid, reliable instrument that we can then make available to other institutions as a measure for student characterization or for program evaluation.

References:
[1] French, B.F., Immekus, J.C., and Oakes, W.C. (2005). An examination of indicators of engineering students’ success and persistence. Journal of Engineering Education, 94(4):419-425.
[2] Clewell, B.C., and Campbell, P.B. (2002). Taking stock: Where we&#39;ve been, where we are, where we&#39;re going. Journal of Women and Minorities in Science and Engineering, 8:255-284.
[3] Isaacs, B. (2001). Mystery of the missing women engineers: A Solution. Journal of Professional Issues in Engineering Education and Practice, 127(2): p. 85-91.
[4] University of Michigan (2005). Why Women Shy Away from Careers in Science and Math. University of Michigan Press Release, Thursday April 7, 2005. &lt;http://www.newswise.com/articles/view/510997/&gt;.
[5] Tuan, H., Vhin, C., and Shich, S. (2005). The development of a questionnaire to measure students’ motivation toward science learning. International Journal of Science Education, 27(6): 639-654.
[6] Rosenberg, M. (1965). Society and the adolescent self-image, Princeton University Press, Princeton, NJ, Instrument: Rosenberg Self-Esteem Scale.
[7] APPLES (Academic Pathways of People Learning Engineering Survey), created by the CAEE (Center for the Advancement of Engineering Education) project and available online at http://caee-aps.stanford.edu/phpESP/admin/manage.php.
[8] LAESE (Longitudinal Assessment of Engineering Self-Efficacy) survey versions 3.0 (copyright 2006) and 3.1 (copyright 2007), which are products of AWE (Assessing Women and Men in Engineering), available online at www.aweonline.org.

AU - Jan DeWaters
AU - John C. Moosbrugger
AU - Pankaj Sharma
CY - Columbus, Ohio
DA - 2017/06/24
PB - ASEE Conferences
TI - Development and Application of a Questionnaire to Measure Student Attitudes Toward and Understanding of Engineering
UR - https://peer.asee.org/28153
DO - 10.18260/1-2--28153
ER -