Developing an Observation Protocol to Categorize Formative Assessment in Engineering Courses

Max William Blackburn; Aaron W. Johnson; Cynthia J. Finelli

Download Paper | Permalink

Conference: 2018 ASEE Annual Conference & Exposition
Location: Salt Lake City, Utah
Publication Date: June 23, 2018
Start Date: June 23, 2018
End Date: July 27, 2018
Conference Session: Research Methods
Tagged Division: Educational Research and Methods
Page Count: 6
DOI: 10.18260/1-2--30300
Permanent URL: https://strategy.asee.org/30300
Download Count: 588

Request a correction

Paper Authors

biography

Max William Blackburn University of Michigan

visit author page

Max Blackburn is a fourth year undergraduate Electrical Engineering student at the University of Michigan, focusing in Power systems and Energy. He is currently assisting Dr. Cynthia Finelli with research concerning the effects of flexible learning spaces and formative assessment techniques.

visit author page

biography

Aaron W. Johnson University of Michigan

visit author page

Aaron W. Johnson is a postdoctoral research fellow at the University of Michigan. He received his Ph.D. in Aeronautics and Astronautics from the Massachusetts Institute of Technology in 2014, after which he served as a postdoctoral research fellow at the Tufts University Center for Engineering Education and Outreach. Aaron also obtained a master's degree from MIT in 2010 and a bachelor's degree from the University of Michigan in 2008, both in aerospace engineering.

visit author page

biography

Cynthia J. Finelli University of Michigan orcid.org/0000-0001-9148-1492

visit author page

Dr. Cynthia Finelli is Associate Professor of Electrical Engineering and Computer Science, Associate Professor of Education, and Director and Graduate Chair for Engineering Education Research Programs at University of Michigan (U-M). Dr. Finelli is a fellow in the American Society of Engineering Education, a Deputy Editor of the Journal for Engineering Education, an Associate Editor of the IEEE Transactions on Education, and past chair of the Educational Research and Methods Division of ASEE. She founded the Center for Research on Learning and Teaching in Engineering at U-M in 2003 and served as its Director for 12 years. Prior to joining U-M, Dr. Finelli was the Richard L. Terrell Professor of Excellence in Teaching, founding director of the Center for Excellence in Teaching and Learning, and associate professor of electrical engineering at Kettering University.

Dr. Finelli's current research interests include student resistance to active learning, faculty adoption of evidence-based teaching practices, the use of technology and innovative pedagogies on student learning and success, and the impact of a flexible classroom space on faculty teaching and student learning. She also led a project to develop a taxonomy for the field of engineering education research, and she was part of a team that studied ethical decision-making in engineering students.

visit author page

Download Paper | Permalink

Abstract

Developing An Observation Protocol to Categorize Formative Assessment in Engineering Courses - Blackburn, Johnson, & Finelli

This work-in-progress paper details the development of an observation protocol to categorize student and instructor behavior during different types of formative assessment in the engineering classroom. Formative assessment is any formal or informal procedure that instructors use to elicit information about student comprehension in order to modify their teaching practices in real-time. It is in contrast to summative assessment, in which quizzes, exams, or project reports are used to gauge student understanding at the end of an instructional unit or semester. Different types of formative assessment give instructors a range of information about student understanding. For example, some types—such as “clicker” questions—provide a quantitative measure of how many students can answer a question correctly, while other types—such as impromptu “in the moment” responses to nonverbal cues, questions, or statements from students—can provide a deeper understanding of how students are conceptualizing different class topics or approaching problems. Our goal for this work-in-progress paper is to present the development of a classroom observation protocol to allow researchers to categorize the type of formative assessment used in undergraduate engineering courses. The protocol will be framed around the following research questions: 1) How do instructors implement formative assessment in large undergraduate engineering courses?, 2) How do students respond to different instances of formative assessment?, and 3) How do instructors change their teaching based on the positive or negative feedback they receive from students through formative assessment? Answering these research questions will allow us to assess which formative assessment strategies give instructors “deeper” understanding of student thinking. As the first step in developing our classroom observation protocol, we are currently reviewing 50 hours of video recordings from 7 undergraduate engineering courses at a large Midwestern university. We are identifying instances of formative assessment in these recordings and then coding the instances according to three categories: 1) the type of formative assessment, 2) students’ response to this formative assessment, and 3) changes made by the instructor based on students’ response. After developing our initial set of codes, we will combine similar codes in each category to develop a final list of codes for our classroom observation protocol. In this work-in-progress paper we will present our methodology and initial list of codes in each category, and we will seek feedback from the community in refining our list.

This research is part of a larger project to ascertain how different classroom settings can affect student-instructor interaction in large undergraduate engineering courses. The classroom observation protocol developed within this project will be used to study flexible classroom spaces that have movable tables and chairs and that, as a result, may allow the instructor to interact better with students and may ultimately lead to “deeper” types of formative assessment.

Citation
Format

Blackburn, M. W., & Johnson, A. W., & Finelli, C. J. (2018, June), Developing an Observation Protocol to Categorize Formative Assessment in Engineering Courses Paper presented at 2018 ASEE Annual Conference & Exposition , Salt Lake City, Utah. 10.18260/1-2--30300

TY  - CPAPER
AB  - Developing An Observation Protocol to Categorize Formative Assessment in Engineering Courses - Blackburn, Johnson, &amp; Finelli

This work-in-progress paper details the development of an observation protocol to categorize student and instructor behavior during different types of formative assessment in the engineering classroom. Formative assessment is any formal or informal procedure that instructors use to elicit information about student comprehension in order to modify their teaching practices in real-time. It is in contrast to summative assessment, in which quizzes, exams, or project reports are used to gauge student understanding at the end of an instructional unit or semester.
 
Different types of formative assessment give instructors a range of information about student understanding. For example, some types—such as “clicker” questions—provide a quantitative measure of how many students can answer a question correctly, while other types—such as impromptu “in the moment” responses to nonverbal cues, questions, or statements from students—can provide a deeper understanding of how students are conceptualizing different class topics or approaching problems. Our goal for this work-in-progress paper is to present the development of a classroom observation protocol to allow researchers to categorize the type of formative assessment used in undergraduate engineering courses. The protocol will be framed around the following research questions: 1) How do instructors implement formative assessment in large undergraduate engineering courses?, 2) How do students respond to different instances of formative assessment?, and 3) How do instructors change their teaching based on the positive or negative feedback they receive from students through formative assessment? Answering these research questions will allow us to assess which formative assessment strategies give instructors “deeper” understanding of student thinking.
 
As the first step in developing our classroom observation protocol, we are currently reviewing 50 hours of video recordings from 7 undergraduate engineering courses at a large Midwestern university. We are identifying instances of formative assessment in these recordings and then coding the instances according to three categories: 1) the type of formative assessment, 2) students’ response to this formative assessment, and 3) changes made by the instructor based on students’ response. After developing our initial set of codes, we will combine similar codes in each category to develop a final list of codes for our classroom observation protocol. In this work-in-progress paper we will present our methodology and initial list of codes in each category, and we will seek feedback from the community in refining our list.

This research is part of a larger project to ascertain how different classroom settings can affect student-instructor interaction in large undergraduate engineering courses. The classroom observation protocol developed within this project will be used to study flexible classroom spaces that have movable tables and chairs and that, as a result, may allow the instructor to interact better with students and may ultimately lead to “deeper” types of formative assessment.

AU  - Max William Blackburn
AU  - Aaron W. Johnson
AU  - Cynthia J. Finelli
CY  - Salt Lake City, Utah
DA  - 2018/06/23
PB  - ASEE Conferences
TI  - Developing an Observation Protocol to Categorize Formative Assessment in Engineering Courses
UR  - https://strategy.asee.org/30300
DO  - 10.18260/1-2--30300
ER  -