Development and Validation of the Draw-an-Engineer and Applications of Mathematics and Science Instrument (Work in Progress)

Rebekah J. Hammack; Toni Ivey; Juliana Utley

Download Paper | Permalink

Conference: 2021 ASEE Virtual Annual Conference Content Access
Location: Virtual Conference
Publication Date: July 26, 2021
Start Date: July 26, 2021
End Date: July 19, 2022
Conference Session: Pre-College Engineering Education Division Poster Session
Tagged Division: Pre-College Engineering Education
Page Count: 9
DOI: 10.18260/1-2--36954
Permanent URL: https://strategy.asee.org/36954
Download Count: 322

Request a correction

Paper Authors

biography

Rebekah J. Hammack Montana State University orcid.org/0000-0002-8621-1006

visit author page

Rebekah Hammack is an Assistant Professor of K-8 Science Education at Montana State University. Prior to joining the faculty at MSU, she served as an Albert Einstein Distinguished Educator Fellow in the Division of Research on Learning in Formal and Informal Settings at the National Science Foundation. She holds a BS in Animal Science from The Ohio State University, a MS in Animal Science from Oklahoma State University, and a PhD in Science Education from Oklahoma State University. She spent 12 years teaching secondary science and engineering in Oklahoma, and is a 2014 recipient of the Presidential Award for Excellence in Mathematics and Science Teaching.

visit author page

biography

Toni Ivey Oklahoma State University orcid.org/0000-0002-3725-7649

visit author page

Dr. Toni Ivey is an Associate Professor of Science Education in the School of Teaching and Curriculum Leadership at Oklahoma State University. She serves as the co-director for the Center for Research on STEM Teaching and Learning and the program coordinator for the Science and Mathematics Education program. Her research interests include science teacher professional development, science teacher preparation, engineering education, and geoscience education.

visit author page

biography

Juliana Utley Oklahoma State University orcid.org/0000-0002-2850-6148

visit author page

Juliana Utley is a Professor of Mathematics Education and Director for the Center for Research on STEM Teaching and Learning (CRSTL) at Oklahoma State University. Her research interests include affective issues in mathematics education, professional development of preservice and in-service teachers, and engineering education.

visit author page

Download Paper | Permalink

Abstract

A Framework for K-12 Science Education (NRC, 2012) and the Next Generation Science Standards ([NGSS], 2013) highlight the importance of including engineering in the K-12 curriculum. The implementation of NGSS requires that teachers understand engineers’ use of design in their work as well as the ways that engineering is connected to science, technology, and society. Researchers have used drawing tasks, such as the Draw and Engineer (DAE) test to identify teachers’ perceptions of engineers. While DAE instruments can provide researchers with important information about teachers perceptions, current instruments fall short of eliciting information about how teachers view the role of science and mathematics within the scope of engineers’ work. We sought to develop a modified version of DAE that allows researchers to identify how teachers perceive the use of mathematics and science within engineering work. The aim of the current study was to examine the use of the Draw-An-Engineer and Applications of Mathematics and Science (DEAMS) instrument and associated scoring rubric (DEAMS-R) for eliciting elementary teachers’ perceptions about the work of engineers and the ways that engineers use mathematics and science. DEAMS instructs participants to draw an engineer or engineers at work and then explain what the engineer is doing, as well as how the engineer is using mathematics and science in the work depicted. This studies documents the validation of DEAMS-R with elementary pre-service teachers at two large lang grant universities, one in the western US and one in the mid-western US. All participants were enrolled in an elementary science methods course at the time they completed the instrument. All participants completed the DEAMS twice, consecutively, during a single class meeting, resulting is a total of 170 drawings for analysis. The instrument was given twice consecutively to determine if a second drawing provided additional insight into participants’ perceptions. The current study documents the steps taken to ensure content validity and reliability, including expert panel review and inter-rater reliability. The first version of the rubric has three constructs: work of an engineer, use of science, and use of mathematics. Each construct has possible values of 1, 2, 3, or 4. This version was used by two science education faculty to independently score a random sample of 24 drawings. Using the first version of the rubric, the two faculty agreed on 67% of work of an engineer, 82% of use of science, and 77% of use of mathematics, with K=0.534, which indicates moderate agreement. Areas of disagreement were discussed extensively, and these discussions were used to develop a second version of the rubric. The second version is currently being tested and results from this second version of the rubric will be shared in the final paper.

Citation
Format

Hammack, R. J., & Ivey, T., & Utley, J. (2021, July), Development and Validation of the Draw-an-Engineer and Applications of Mathematics and Science Instrument (Work in Progress) Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--36954

TY - CPAPER
AB - A Framework for K-12 Science Education (NRC, 2012) and the Next Generation Science Standards ([NGSS], 2013) highlight the importance of including engineering in the K-12 curriculum. The implementation of NGSS requires that teachers understand engineers’ use of design in their work as well as the ways that engineering is connected to science, technology, and society. Researchers have used drawing tasks, such as the Draw and Engineer (DAE) test to identify teachers’ perceptions of engineers. While DAE instruments can provide researchers with important information about teachers perceptions, current instruments fall short of eliciting information about how teachers view the role of science and mathematics within the scope of engineers’ work. We sought to develop a modified version of DAE that allows researchers to identify how teachers perceive the use of mathematics and science within engineering work. The aim of the current study was to examine the use of the Draw-An-Engineer and Applications of Mathematics and Science (DEAMS) instrument and associated scoring rubric (DEAMS-R) for eliciting elementary teachers’ perceptions about the work of engineers and the ways that engineers use mathematics and science. DEAMS instructs participants to draw an engineer or engineers at work and then explain what the engineer is doing, as well as how the engineer is using mathematics and science in the work depicted. This studies documents the validation of DEAMS-R with elementary pre-service teachers at two large lang grant universities, one in the western US and one in the mid-western US. All participants were enrolled in an elementary science methods course at the time they completed the instrument. All participants completed the DEAMS twice, consecutively, during a single class meeting, resulting is a total of 170 drawings for analysis. The instrument was given twice consecutively to determine if a second drawing provided additional insight into participants’ perceptions. The current study documents the steps taken to ensure content validity and reliability, including expert panel review and inter-rater reliability. The first version of the rubric has three constructs: work of an engineer, use of science, and use of mathematics. Each construct has possible values of 1, 2, 3, or 4. This version was used by two science education faculty to independently score a random sample of 24 drawings. Using the first version of the rubric, the two faculty agreed on 67% of work of an engineer, 82% of use of science, and 77% of use of mathematics, with K=0.534, which indicates moderate agreement. Areas of disagreement were discussed extensively, and these discussions were used to develop a second version of the rubric. The second version is currently being tested and results from this second version of the rubric will be shared in the final paper.
AU - Rebekah J. Hammack
AU - Toni Ivey
AU - Juliana Utley
CY - Virtual Conference
DA - 2021/07/26
PB - ASEE Conferences
TI - Development and Validation of the Draw-an-Engineer and Applications of Mathematics and Science Instrument (Work in Progress)
UR - https://strategy.asee.org/36954
DO - 10.18260/1-2--36954
ER -