Board 135: DIY Classroom Observations: A Toolkit for Novice Classroom Observers

Adrian Leos; Kaitlin Harlan; Brooke A. Merrill; Sin-Ning Cindy Liu; Cynthia K. Lang; Robert C. Martin; Carolyn L. Sandoval; Mindy Bergman

Download Paper | Permalink

Conference: 2019 ASEE Annual Conference & Exposition
Location: Tampa, Florida
Publication Date: June 15, 2019
Start Date: June 15, 2019
End Date: June 19, 2019
Conference Session: NSF Grantees Poster Session
Tagged Topic: NSF Grantees Poster Session
Page Count: 9
DOI: 10.18260/1-2--32243
Permanent URL: https://strategy.asee.org/32243
Download Count: 413

Request a correction

Ph.D. student in Industrial/Organizational Psychology at Texas A&M University.
M.A. Educational Psychology, 2016 - Baylor University
B.A. Psychology, 2014 - Baylor University

visit author page

biography

Cynthia K. Lang Texas A&M University

visit author page

Cynthia Lang is a third-year graduate student in the School Psychology Ph.D program at Texas A&M University. She earned her BA in Psychology from The University of Texas at Austin in May 2016.

visit author page

author page

Robert C. Martin Texas A&M University

biography

Carolyn L. Sandoval University of California, San Diego

visit author page

Dr. Sandoval is the Associate Director of the Teaching + Learning Commons at the University of California, San Diego. She earned a PhD in Adult Education-Human Resource Development. Her research interests include adult learning and development, faculty development, qualitative methods of inquiry, and social justice education.

visit author page

biography

Mindy Bergman Texas A&M University

visit author page

Dr. Bergman is a Professor in the Department of Psychological and Brain Sciences and Executive Director of Interdisciplinary Critical Studies at Texas A&M University. She earned her PhD in industrial-organizational psychology at the University of Illinois at Urbana-Champaign. Her research interests include workplace safety, occupational health, and fairness and mistreatment in the workplace and in STEM classrooms and programs.

visit author page

Download Paper | Permalink

Abstract

The Improving Student Experiences to Increase Student Engagement (ISE-2) grant was awarded to Texas A&M University by the National Science Foundation, through EEC-Engineering Diversity Activities (Grant No. 1648016) with the goal of increasing student engagement and retention in the College of Engineering. ISE-2 deployed a faculty development program in the Summers of 2017 and 2018, with the goal of increasing active learning, improving classroom climates, and decreasing implicit bias and deficit thinking in first- and second-year Engineering classes. In order to capture whether faculty have changed their instructional practices prior to and following the faculty development program, the ISE-2 team conducted classroom observations both before and after the faculty participants attended the trainings. To date, the team has observed 55 total classes in Spring 2017, Fall 2017, and Spring 2018, with two observers attending most of the classes.

The ISE-2 observation team consists of faculty, instructional consultants from the Center for Teaching Excellence (CTE), graduate students, and undergraduate research assistants. Thus, there was a wide range of classroom observation experiences in our team, ranging from expert to novice. In order to ensure the quality of the classroom observations and improve interrater reliability, the novice classroom observers were trained to observe undergraduate classrooms by the expert observers at the beginning of each semester. These training sessions became the basis of this classroom observation toolkit. This classroom observation toolkit can be used by instructors or researchers who are interested in doing classroom observations, but have limited experience with qualitative coding and observational research, to train themselves and their research assistants. This paper will serve as an introduction and a handbook for the classroom observation toolkit. The observation toolkit includes: a) an annotated bibliography introducing articles that would be helpful to understanding classroom observations, b) a series of training videos teaching viewers to conduct classroom observations using an environmental scan and the Classroom Observation Protocol for Undergraduate STEM (COPUS; Smith et al., 2013), one of the peer-reviewed coding schemes that is available in the engineering education literature, c) a series of sample classroom videos that document a typical first- and second-year Engineering class, d) validation keys for each of the sample videos (i.e., consensus coding of the sample videos from our team, against which a new team can compare their codes and discuss their matches and mismatches as a group), and e) guidance from the qualitative coding literature on how to determine whether emergent codes (i.e., additional codes beyond the initially adopted coding scheme) are needed and how to operationalize them.

Reference Smith, M. K., Jones, F. H. M., Gilbert, S. L., & Wieman, C. E. (2013). The Classroom Observation Protocol for Undergraduate STEM (COPUS): A new instrument to characterize university STEM classroom practices. CBE—Life Sciences Education, 12, 618-627. DOI: 10.1187/cbe.13-08-0154.

Citation
Format

Leos, A., & Harlan, K., & Merrill, B. A., & Liu, S. C., & Lang, C. K., & Martin, R. C., & Sandoval, C. L., & Bergman, M. (2019, June), Board 135: DIY Classroom Observations: A Toolkit for Novice Classroom Observers Paper presented at 2019 ASEE Annual Conference & Exposition , Tampa, Florida. 10.18260/1-2--32243

TY - CPAPER
AB - The Improving Student Experiences to Increase Student Engagement (ISE-2) grant was awarded to Texas A&amp;M University by the National Science Foundation, through EEC-Engineering Diversity Activities (Grant No. 1648016) with the goal of increasing student engagement and retention in the College of Engineering. ISE-2 deployed a faculty development program in the Summers of 2017 and 2018, with the goal of increasing active learning, improving classroom climates, and decreasing implicit bias and deficit thinking in first- and second-year Engineering classes. In order to capture whether faculty have changed their instructional practices prior to and following the faculty development program, the ISE-2 team conducted classroom observations both before and after the faculty participants attended the trainings. To date, the team has observed 55 total classes in Spring 2017, Fall 2017, and Spring 2018, with two observers attending most of the classes.

The ISE-2 observation team consists of faculty, instructional consultants from the Center for Teaching Excellence (CTE), graduate students, and undergraduate research assistants. Thus, there was a wide range of classroom observation experiences in our team, ranging from expert to novice. In order to ensure the quality of the classroom observations and improve interrater reliability, the novice classroom observers were trained to observe undergraduate classrooms by the expert observers at the beginning of each semester. These training sessions became the basis of this classroom observation toolkit. This classroom observation toolkit can be used by instructors or researchers who are interested in doing classroom observations, but have limited experience with qualitative coding and observational research, to train themselves and their research assistants. This paper will serve as an introduction and a handbook for the classroom observation toolkit. The observation toolkit includes: a) an annotated bibliography introducing articles that would be helpful to understanding classroom observations, b) a series of training videos teaching viewers to conduct classroom observations using an environmental scan and the Classroom Observation Protocol for Undergraduate STEM (COPUS; Smith et al., 2013), one of the peer-reviewed coding schemes that is available in the engineering education literature, c) a series of sample classroom videos that document a typical first- and second-year Engineering class, d) validation keys for each of the sample videos (i.e., consensus coding of the sample videos from our team, against which a new team can compare their codes and discuss their matches and mismatches as a group), and e) guidance from the qualitative coding literature on how to determine whether emergent codes (i.e., additional codes beyond the initially adopted coding scheme) are needed and how to operationalize them.

Reference
Smith, M. K., Jones, F. H. M., Gilbert, S. L., &amp; Wieman, C. E. (2013). The Classroom Observation Protocol for Undergraduate STEM (COPUS): A new instrument to characterize university STEM classroom practices. CBE—Life Sciences Education, 12, 618-627. DOI: 10.1187/cbe.13-08-0154.
AU - Adrian Leos
AU - Kaitlin Harlan
AU - Brooke A. Merrill
AU - Sin-Ning Cindy Liu
AU - Cynthia K. Lang
AU - Robert C. Martin
AU - Carolyn L. Sandoval
AU - Mindy Bergman
CY - Tampa, Florida
DA - 2019/06/15
PB - ASEE Conferences
TI - Board 135: DIY Classroom Observations: A Toolkit for Novice Classroom Observers
UR - https://strategy.asee.org/32243
DO - 10.18260/1-2--32243
ER -