Instrumentation for Evaluating Design-learning and Instruction Within Courses and Across Programs

Steven Santana

Download Paper | Permalink

Conference: 2021 ASEE Virtual Annual Conference Content Access
Location: Virtual Conference
Publication Date: July 26, 2021
Start Date: July 26, 2021
End Date: July 19, 2022
Conference Session: Faculty Perspectives of Active Learning, Inequity, and Curricular Change
Tagged Division: Educational Research and Methods
Tagged Topic: Diversity
Page Count: 23
DOI: 10.18260/1-2--37351
Permanent URL: https://strategy.asee.org/37351
Download Count: 307

Request a correction

Paper Authors

author page

Steven Santana Harvey Mudd College

Download Paper | Permalink

Abstract

This work in progress describes an instrumentation plan, applied through action research, to evaluate student learning of engineering design. The instruments described herein will be applied to an entry-level design course. This course situates teams as engineers contracted to work on a project provided by an external client. The instructors solicit, select, and refine projects that present engineering problems requiring conceptual design and tangible, mechanical solutions. In this course, the overarching learning outcome is to implement and practice a design process to identify, frame, and solve open-ended and ill-structured engineering problems. This outcome emphasizes the inherent complexity, ambiguity, and nonlinearity of the design process and is, as a result, a challenge to teach and difficult to learn. Students often find the reality of engineering design overwhelming the first time they experience it. Instructor observations indicate that students proceed through the design process in a linear fashion, spend insufficient time scoping problems, and minimally implement design activities to advance their work and learning. This trend is well documented. A primary learning outcome of this course is for students to internalize the design process as a set of activities that engineers deploy with adaptive expertise to create high-quality deliverables that satisfy needs while developing their identity as engineers. Thus, this work seeks to develop a mixed-methods, instrument suite to (1) assess students implementation of design activities within authentic design projects, (2) evaluate final deliverable quality, (3) survey students’ engineering design abilities, and (4) record students’ evolving engineering identity and sense of belonging in the engineering community. Future work includes developing prototypes of the aforementioned instruments and assessing their validity and reliability. To quantify design activities, this instrument includes design diaries to capture engagement with design phases and activities. To assess the quality of final designs, this suite includes a rubric with the goal of connecting processes to outcomes. To measure students’ perceptions of their abilities, engineering identity, and sense of belonging within engineering this work includes surveys to capture development, on an individual level, in each of these areas. This work will extend prior scholarship by lengthening the timeline of instrument use to a nearly semester-long project; by applying the instrument to a student team’s deliverable, as opposed to an individual’s work; and, by using a standard rubric to assess unique designs created by students within similar settings. Once developed and validated, these combined measures will be deployed to understand students’ learning as a function of course pedagogy, activities, structures, and climate and to iteratively improve student outcomes and instruction. A future goal is to apply this tool and analysis framework to improve instruction across courses and programs, wherein design is, or could be, central to student learning. This work is done in service to equipping engineering graduates with expertise, fluency, and with the capacity to lead in engineering and design.

Citation
Format

Santana, S. (2021, July), Instrumentation for Evaluating Design-learning and Instruction Within Courses and Across Programs Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--37351

TY - CPAPER
AB - This work in progress describes an instrumentation plan, applied through action research, to evaluate student learning of engineering design. The instruments described herein will be applied to an entry-level design course. This course situates teams as engineers contracted to work on a project provided by an external client. The instructors solicit, select, and refine projects that present engineering problems requiring conceptual design and tangible, mechanical solutions.
In this course, the overarching learning outcome is to implement and practice a design process to identify, frame, and solve open-ended and ill-structured engineering problems. This outcome emphasizes the inherent complexity, ambiguity, and nonlinearity of the design process and is, as a result, a challenge to teach and difficult to learn. Students often find the reality of engineering design overwhelming the first time they experience it. Instructor observations indicate that students proceed through the design process in a linear fashion, spend insufficient time scoping problems, and minimally implement design activities to advance their work and learning. This trend is well documented. A primary learning outcome of this course is for students to internalize the design process as a set of activities that engineers deploy with adaptive expertise to create high-quality deliverables that satisfy needs while developing their identity as engineers.
Thus, this work seeks to develop a mixed-methods, instrument suite to (1) assess students implementation of design activities within authentic design projects, (2) evaluate final deliverable quality, (3) survey students’ engineering design abilities, and (4) record students’ evolving engineering identity and sense of belonging in the engineering community. Future work includes developing prototypes of the aforementioned instruments and assessing their validity and reliability. To quantify design activities, this instrument includes design diaries to capture engagement with design phases and activities. To assess the quality of final designs, this suite includes a rubric with the goal of connecting processes to outcomes. To measure students’ perceptions of their abilities, engineering identity, and sense of belonging within engineering this work includes surveys to capture development, on an individual level, in each of these areas. This work will extend prior scholarship by lengthening the timeline of instrument use to a nearly semester-long project; by applying the instrument to a student team’s deliverable, as opposed to an individual’s work; and, by using a standard rubric to assess unique designs created by students within similar settings.
Once developed and validated, these combined measures will be deployed to understand students’ learning as a function of course pedagogy, activities, structures, and climate and to iteratively improve student outcomes and instruction. A future goal is to apply this tool and analysis framework to improve instruction across courses and programs, wherein design is, or could be, central to student learning. This work is done in service to equipping engineering graduates with expertise, fluency, and with the capacity to lead in engineering and design.

AU - Steven Santana
CY - Virtual Conference
DA - 2021/07/26
PB - ASEE Conferences
TI - Instrumentation for Evaluating Design-learning and Instruction Within Courses and Across Programs
UR - https://strategy.asee.org/37351
DO - 10.18260/1-2--37351
ER -