Work in Progress: Can In-Class Peer Reviews of Written Assignments Improve Problem Solving and Scientific Writing in a Standard-Based, Sophomore Laboratory Course?

Ken Gentry; Philippa Eshun; Casey Jane Ankeny; David P. O'Neill

Download Paper | Permalink

Conference: 2023 ASEE Annual Conference & Exposition
Location: Baltimore , Maryland
Publication Date: June 25, 2023
Start Date: June 25, 2023
End Date: June 28, 2023
Conference Session: Biomedical Engineering Division (BED): Best of Works in Progress
Tagged Division: Biomedical Engineering Division (BED)
Page Count: 11
DOI: 10.18260/1-2--44182
Permanent URL: https://strategy.asee.org/44182
Download Count: 136

Request a correction

Paper Authors

biography

Casey Jane Ankeny Northwestern University

visit author page

Casey J. Ankeny, PhD is an Associate Professor of Instruction at Northwestern University. Casey received her bachelor’s degree in Biomedical Engineering from the University of Virginia and her doctorate degree in Biomedical Engineering from Georgia Institute of Technology and Emory University where she studied the role of shear stress in aortic valve disease. Currently, she is investigating equitable student engagement strategies in flipped and traditional biomedical engineering courses. She aspires to understand and improve student attitude, achievement, and persistence in student-centered courses.

visit author page

biography

Ken Gentry Northwestern University

visit author page

Ken Gentry is a Professor of Instruction and Adviser working mainly with first-year students. He teaches cornerstone design and courses in the biomedical engineering department.

visit author page

biography

David P. O'Neill Northwestern University

visit author page

David O'Neill is an Associate Professor of Instruction and the Michael Jaharis Director of Experiential Learning for the Biomedical Engineering Department at Northwestern University. David read Engineering Science at University College, Oxford, receiving his M.Eng. and D.Phil. before undertaking a post-doc in the Department of Physiology, Anatomy and Genetics. During these years, he taught undergraduate tutorials for Keble, New, University, and Harris Manchester Colleges, was College Lecturer for New College and a Senior College Lecturer in Engineering Science for Keble College. He taught approximately 75% of the core degree topics, as well as human physiological measurement laboratory classes for medical students.

visit author page

author page

Philippa Eshun

Download Paper | Permalink

Abstract

Introduction: This work-in-progress study assesses the impact of reflective practices, including peer review, on written assignments in a sophomore-level, biomedical engineering laboratory course. The course serves as an introduction to experimentation and covers statistical design of experiments as well as how to quantify the quality of experimental measurement data. Topics include problem-solving skills, scientific writing, hypothesis generation, amongst other research-related topics. Course pedagogy includes standards-based grading and reflection.

Our previous work identified weaknesses in the “problem identification” and “interpretation” components of problem solving [1]. As a result, we implemented the evidence-based strategy of reflective practices [2] and noticed a trend suggesting that increased reflective practices, namely the addition of reflective engineering notebooks, may have improved student perception of standards-based grading as well as may increase their engagement with mastery of course standards [3]. To address limitations of our previous study and build upon encouraging results, we will implement the reflective, equitable strategy of peer review [4] on individual written assignments. We hypothesize that the implementation of peer review on individually written extended abstracts will result in increased mastery in course standards, namely “problem identification” and “interpretation”, and result in favorable student attitude with regards to the peer review process.

Methods: In-class: Written assignments will be assessed by peer reviewers and also by the instructional team, using an enhanced Likert-based, problem-solving rubric [5] which expands on “problem identification” and “interpretation”. Both students and the instructional team will undergo grader calibration. During the peer review process, peer reviewers will only provide written feedback (no Likert scores) to follow the evidenced-based process of ungrading [6].

Methods: Study Analysis: Methodologies used in the study will include quantitative and qualitative survey data analysis and analysis of written lab assignments. More specifically, the study team will grade the draft and final submissions using the Likert scale and providing comments. Only the grades for the final submission will be released; however, the researchers will analyze pre- and post-submissions to assess the impact of the peer review process on standards mastery. Further, the researchers will grade the quality of the peer reviews and investigate correlations with mastery levels of both the critics and critiqued. Lastly, the study team will investigate student attitude using surveys.

Results: The study team has developed two instruments for assessment: 1) the assessment of peer review quality and 2) an attitudinal survey regarding the peer review process. More specifically, the peer review quality instrument (based on [7]) will assess the appropriateness and specificity of the criticism as well as the justification for why it is included. Lastly, each review will be assessed for the presence and usefulness of any suggestions. The attitudinal survey (based on [8]-[10]) contains two sections, one for the critic and one for the critiqued, and covers areas such as utility, assessment of training/grader calibration, impact on future work, and emotion.

Discussion and Conclusion: The use of reflective practices improves the implementation of standards-based grading [2]. Our work has seen a trend suggesting improved engagement and standard-based grading perception [3]; however, further work is needed to assess without previous study limitations. More specifically, this work-in-progress assesses the implementation of an additional reflective practice (peer review) with the goal of improved individual mastery in problem solving and written communication while retaining high student favor.

References: [1] blinded [2] H. A. Diefes-Dux and A. R. Carberry, “Cases of Student Reflection within a Course Using Standards-Based Grading,” in 2019 IEEE Frontiers in Education Conference (FIE), 2019, pp. 1–9. doi: 10.1109/FIE43999.2019.9028501. [3] blinded [4] Feldman, J. (2019). Grading for equity: What it is, why it matters, and how it can transform schools and classrooms. [5] blinded [6] Kohn, A., & Blum, S.D. (2020). Ungrading: Why Rating Students Undermines Learning (and What to Do Instead). (First edition. ed.). Morgantown: West Virginia University Press. [7] S. Gielen, E. Peeters, F. Dochy, P. Onghena, and K. Struyven, “Improving the effectiveness of peer feedback for learning,” Learning and Instruction, vol. 20, no. 4, pp. 304–315, Aug. 2010, doi: 10.1016/j.learninstruc.2009.08.007. [8] Andersson, Magnus, and Maria Weurlander. “Peer Review of Laboratory Reports for Engineering Students.” European journal of engineering education. 44.3 (2019): 417–428. Web. [9] Conde, Miguel A. “Application of Peer Review Techniques in Engineering Education.” The International journal of engineering education. 33.2 (2017): 918–926. Print. [10] Larsen, Katarina, and Johan Gärdebo. “Retooling Engineering for Social Justice: The Use of Explicit Models for Analytical Thinking, Critical Reflection, and Peer-Review in Swedish Engineering Education.” International journal of engineering, social justice and peace. 5.1-2 (2018): 13–29. Web.

Citation
Format

Ankeny, C. J., & Gentry, K., & O'Neill, D. P., & Eshun, P. (2023, June), Work in Progress: Can In-Class Peer Reviews of Written Assignments Improve Problem Solving and Scientific Writing in a Standard-Based, Sophomore Laboratory Course? Paper presented at 2023 ASEE Annual Conference & Exposition, Baltimore , Maryland. 10.18260/1-2--44182

TY - CPAPER
AB - Introduction:
This work-in-progress study assesses the impact of reflective practices, including peer review, on written assignments in a sophomore-level, biomedical engineering laboratory course. The course serves as an introduction to experimentation and covers statistical design of experiments as well as how to quantify the quality of experimental measurement data. Topics include problem-solving skills, scientific writing, hypothesis generation, amongst other research-related topics. Course pedagogy includes standards-based grading and reflection.

Our previous work identified weaknesses in the “problem identification” and “interpretation” components of problem solving [1]. As a result, we implemented the evidence-based strategy of reflective practices [2] and noticed a trend suggesting that increased reflective practices, namely the addition of reflective engineering notebooks, may have improved student perception of standards-based grading as well as may increase their engagement with mastery of course standards [3]. To address limitations of our previous study and build upon encouraging results, we will implement the reflective, equitable strategy of peer review [4] on individual written assignments. We hypothesize that the implementation of peer review on individually written extended abstracts will result in increased mastery in course standards, namely “problem identification” and “interpretation”, and result in favorable student attitude with regards to the peer review process.

Methods: In-class:
Written assignments will be assessed by peer reviewers and also by the instructional team, using an enhanced Likert-based, problem-solving rubric [5] which expands on “problem identification” and “interpretation”. Both students and the instructional team will undergo grader calibration. During the peer review process, peer reviewers will only provide written feedback (no Likert scores) to follow the evidenced-based process of ungrading [6].

Methods: Study Analysis:
Methodologies used in the study will include quantitative and qualitative survey data analysis and analysis of written lab assignments. More specifically, the study team will grade the draft and final submissions using the Likert scale and providing comments. Only the grades for the final submission will be released; however, the researchers will analyze pre- and post-submissions to assess the impact of the peer review process on standards mastery. Further, the researchers will grade the quality of the peer reviews and investigate correlations with mastery levels of both the critics and critiqued. Lastly, the study team will investigate student attitude using surveys.

Results:
The study team has developed two instruments for assessment: 1) the assessment of peer review quality and 2) an attitudinal survey regarding the peer review process. More specifically, the peer review quality instrument (based on [7]) will assess the appropriateness and specificity of the criticism as well as the justification for why it is included. Lastly, each review will be assessed for the presence and usefulness of any suggestions. The attitudinal survey (based on [8]-[10]) contains two sections, one for the critic and one for the critiqued, and covers areas such as utility, assessment of training/grader calibration, impact on future work, and emotion.

Discussion and Conclusion:
The use of reflective practices improves the implementation of standards-based grading [2]. Our work has seen a trend suggesting improved engagement and standard-based grading perception [3]; however, further work is needed to assess without previous study limitations. More specifically, this work-in-progress assesses the implementation of an additional reflective practice (peer review) with the goal of improved individual mastery in problem solving and written communication while retaining high student favor.

References:
[1] blinded
[2] H. A. Diefes-Dux and A. R. Carberry, “Cases of Student Reflection within a Course Using Standards-Based Grading,” in 2019 IEEE Frontiers in Education Conference (FIE), 2019, pp. 1–9. doi: 10.1109/FIE43999.2019.9028501.
[3] blinded
[4] Feldman, J. (2019). Grading for equity: What it is, why it matters, and how it can transform schools and classrooms.
[5] blinded
[6] Kohn, A., &amp;amp; Blum, S.D. (2020). Ungrading: Why Rating Students Undermines Learning (and What to Do Instead). (First edition. ed.). Morgantown: West Virginia University Press.
[7] S. Gielen, E. Peeters, F. Dochy, P. Onghena, and K. Struyven, “Improving the effectiveness of peer feedback for learning,” Learning and Instruction, vol. 20, no. 4, pp. 304–315, Aug. 2010, doi: 10.1016/j.learninstruc.2009.08.007.
[8] Andersson, Magnus, and Maria Weurlander. “Peer Review of Laboratory Reports for Engineering Students.” European journal of engineering education. 44.3 (2019): 417–428. Web.
[9] Conde, Miguel A. “Application of Peer Review Techniques in Engineering Education.” The International journal of engineering education. 33.2 (2017): 918–926. Print.
[10] Larsen, Katarina, and Johan Gärdebo. “Retooling Engineering for Social Justice: The Use of Explicit Models for Analytical Thinking, Critical Reflection, and Peer-Review in Swedish Engineering Education.” International journal of engineering, social justice and peace. 5.1-2 (2018): 13–29. Web.

AU - Casey Jane Ankeny
AU - Ken Gentry
AU - David P. O'Neill
AU - Philippa Eshun
CY - Baltimore , Maryland
DA - 2023/06/25
PB - ASEE Conferences
TI - Work in Progress: Can In-Class Peer Reviews of Written Assignments Improve Problem Solving and Scientific Writing in a Standard-Based, Sophomore Laboratory Course?
UR - https://strategy.asee.org/44182
DO - 10.18260/1-2--44182
ER -