Testing a Reflective Judgement Scale for Suitability with  First-Year Student Reflective Responses

Natalie C.T. Van Tyne; Lisa D. McNair

Download Paper | Permalink

Conference: 2019 ASEE Annual Conference & Exposition
Location: Tampa, Florida
Publication Date: June 15, 2019
Start Date: June 15, 2019
End Date: June 19, 2019
Conference Session: First-Year Programs: Focusing on Student Success
Tagged Division: First-Year Programs
Page Count: 13
DOI: 10.18260/1-2--33364
Permanent URL: https://strategy.asee.org/33364
Download Count: 871

Request a correction

Paper Authors

biography

Natalie C.T. Van Tyne Virginia Tech orcid.org/0000-0002-7058-9098

visit author page

Natalie Van Tyne is an Associate Professor of Practice at Virginia Polytechnic Institute and State University, where she teaches first year engineering design as a foundation courses for Virginia Tech's undergraduate engineering degree programs. She holds bachelors and masters degrees from Rutgers University, Lehigh University and Colorado School of Mines, and studies best practices in pedagogy, reflective learning and critical thinking as aids to enhanced student learning.

visit author page

biography

Lisa D. McNair Virginia Tech

visit author page

Lisa D. McNair is a Professor of Engineering Education at Virginia Tech, where she also serves as Director of the Center for Research in SEAD Education at the Institute for Creativity, Arts, and Technology (ICAT). Her research interests include interdisciplinary collaboration, design education, communication studies, identity theory and reflective practice. Projects supported by the National Science Foundation include exploring disciplines as cultures, liberatory maker spaces, and a RED grant to increase pathways in ECE for the professional formation of engineers.

visit author page

Download Paper | Permalink

Abstract

Testing a Reflective Judgement Scale for Suitability with First Year Student Reflective Responses

This Complete Evidence-Based Practice paper describes the use of reflection in a first-year engineering design course. Reflection is an essential part of learning, but it is not widely used in engineering curricula. Using reflective learning techniques in the classroom can help students develop critical thinking skills, which are highly valued in the modern workplace. Critical thinking consists of an objective analysis and reconstruction of available information, often from multiple sources, before deciding what to accept as valid. While we expect that the ability to think critically develops with practice and time, it would be useful to discern how well our students could learn to reflect and think critically during their first year of college.

Two theoretical models are useful when evaluating student performance on reflective learning tasks. The Perry Model of intellectual development (1996) suggests that students with extensive practice in open-ended problem solving involving reflection will be more successful than their peers. Similarly, the King and Kitchener Reflective Judgement Stages model (1994) contains a scale which is useful for measuring increased complexity of reflective thinking over time, another indicator of future success. Both models emphasize that knowledge is largely contextual, meaning that it is sometimes true, and sometimes not true, depending on the applicable context. Thus, context is another important factor to consider when evaluating students reflective learning performance or ability.

In the present study, we assess the extent to which students use reflective judgement and critical thinking skills when writing about their experiences in a semester long introductory design course. Based on our theoretical models, we hypothesized that students whose reflections acknowledge the role of context in learning will additionally show more critical thinking development over the course of the semester, as well as earn a higher grade on successive assignments. Therefore, we wanted to determine whether students’ ability to reflect on their experiences in the course improved over time.

Participants were approximately 100 first year engineering students who completed a series of five graded short essays. Four essays incorporated reflection about a specific three-week period in the course, while the fifth essay prompted reflection about the entire course content. In each case, students answered three questions whereby they identified the most important item of knowledge that they learned, why it was important, and where else they could use this knowledge, outside of the course. Assignment prompts also stipulated that reflection was to extend beyond the specific needs of the course, such as contributions to homework assignments.

We operationalized the ability to exercise critical thinking by reflecting on multiple sources of knowledge and recognizing alternative contexts by using the Perry Model and King and Kitchener’s reflective judgement stages in parallel with an internally-developed grading rubric. Rubric criteria included the level of relevant detail in describing items of knowledge and their context, and the application of specific knowledge to aspects of lifelong learning. By comparing the grades for each assignment over time, in conjunction with corresponding Perry Model and King and Kitchener stages, we expect to find a mixture of continual, sporadic or lack of progress over time. These results will help further develop this course in the future and can provide new insights as to how students view the importance of reflection and critical thinking for their learning.

Citation
Format

Van Tyne, N. C., & McNair, L. D. (2019, June), Testing a Reflective Judgement Scale for Suitability with First-Year Student Reflective Responses Paper presented at 2019 ASEE Annual Conference & Exposition , Tampa, Florida. 10.18260/1-2--33364

TY - CPAPER
AB - Testing a Reflective Judgement Scale for Suitability with
First Year Student Reflective Responses

This Complete Evidence-Based Practice paper describes the use of reflection in a first-year engineering design course. Reflection is an essential part of learning, but it is not widely used in engineering curricula. Using reflective learning techniques in the classroom can help students develop critical thinking skills, which are highly valued in the modern workplace. Critical thinking consists of an objective analysis and reconstruction of available information, often from multiple sources, before deciding what to accept as valid. While we expect that the ability to think critically develops with practice and time, it would be useful to discern how well our students could learn to reflect and think critically during their first year of college.

Two theoretical models are useful when evaluating student performance on reflective learning tasks. The Perry Model of intellectual development (1996) suggests that students with extensive practice in open-ended problem solving involving reflection will be more successful than their peers. Similarly, the King and Kitchener Reflective Judgement Stages model (1994) contains a scale which is useful for measuring increased complexity of reflective thinking over time, another indicator of future success. Both models emphasize that knowledge is largely contextual, meaning that it is sometimes true, and sometimes not true, depending on the applicable context. Thus, context is another important factor to consider when evaluating students reflective learning performance or ability.

In the present study, we assess the extent to which students use reflective judgement and critical thinking skills when writing about their experiences in a semester long introductory design course. Based on our theoretical models, we hypothesized that students whose reflections acknowledge the role of context in learning will additionally show more critical thinking development over the course of the semester, as well as earn a higher grade on successive assignments. Therefore, we wanted to determine whether students’ ability to reflect on their experiences in the course improved over time.

Participants were approximately 100 first year engineering students who completed a series of five graded short essays. Four essays incorporated reflection about a specific three-week period in the course, while the fifth essay prompted reflection about the entire course content. In each case, students answered three questions whereby they identified the most important item of knowledge that they learned, why it was important, and where else they could use this knowledge, outside of the course. Assignment prompts also stipulated that reflection was to extend beyond the specific needs of the course, such as contributions to homework assignments.

We operationalized the ability to exercise critical thinking by reflecting on multiple sources of knowledge and recognizing alternative contexts by using the Perry Model and King and Kitchener’s reflective judgement stages in parallel with an internally-developed grading rubric. Rubric criteria included the level of relevant detail in describing items of knowledge and their context, and the application of specific knowledge to aspects of lifelong learning. By comparing the grades for each assignment over time, in conjunction with corresponding Perry Model and King and Kitchener stages, we expect to find a mixture of continual, sporadic or lack of progress over time. These results will help further develop this course in the future and can provide new insights as to how students view the importance of reflection and critical thinking for their learning.

AU - Natalie C.T. Van Tyne
AU - Lisa D. McNair
CY - Tampa, Florida
DA - 2019/06/15
PB - ASEE Conferences
TI - Testing a Reflective Judgement Scale for Suitability with First-Year Student Reflective Responses
UR - https://strategy.asee.org/33364
DO - 10.18260/1-2--33364
ER -