A New Assessment Method to Easily Identify Areas Needing Improvement in Course-level Learning Outcomes

Thomas Allen Knotts; W. Vincent Wilding; William G. Pitt; Morris D. Argyle

Download Paper | Permalink

Conference: 2012 ASEE Annual Conference & Exposition
Location: San Antonio, Texas
Publication Date: June 10, 2012
Start Date: June 10, 2012
End Date: June 13, 2012
ISSN: 2153-5965
Conference Session: New Classrooms, New Challenges II: Assessing Non-traditional Approaches
Tagged Division: Chemical Engineering
Page Count: 26
Page Numbers: 25.78.1 - 25.78.26
DOI: 10.18260/1-2--20838
Permanent URL: https://peer.asee.org/20838
Download Count: 312

Request a correction

Paper Authors

biography

Thomas Allen Knotts IV Brigham Young University

visit author page

Thomas Knotts became a faculty member in the Department of Chemical Engineering at Brigham Young University in 2006 after receiving his Ph.D. from the University of Wisconsin, Madison. He teaches a variety of courses, including thermodynamics, computer tools, unit operations lab, and molecular modeling. He enjoys teaching and discovering ways to improve student learning through problem-based and inductive learning strategies. With his research group, Knotts seeks to understand the physics of proteins and DNA at the molecular level with particular emphasis on the behavior of these molecules in “non-native” environments such as those often found in biotechnology. His research efforts have earned him the NSF CAREER Award and the Young Faculty Award from the Defense Advanced Research Projects Agency (DARPA). As part of his research efforts, Knotts creates outreach programs to help teachers improve K-12 STEM education.

visit author page

author page

W. Vincent Wilding Brigham Young University

biography

William G. Pitt Brigham Young University

visit author page

William G. Pitt received a Ph.D. in chemical engineering in 1987 from the University of Wisconsin, Madison. He obtained a faculty position at Brigham Young University in the Chemical Engineering Department, where he has served since 1987. He is currently the Pope Professor of chemical engineering at BYU, and is an Adjunct Research Professor in the Bioengineering Department of the University of Utah. During his 24 years at BYU, his teaching has been in the areas of materials, polymers, and transport phenomena. His research has spanned many disciplines, ranging from biomedical material surfaces and composite materials to his current work in controlled drug and gene delivery. With colleagues and students at BYU and other institutions, he has more than 110 peer-reviewed journal publications.

visit author page

author page

Morris D. Argyle Brigham Young University orcid.org/0000-0001-9430-9145

Download Paper | Permalink

Abstract

ANewAssessmentMethodtoEasilyIdentifyAreasNeedingImprovementinCourse‐levelLearningOutcomesThomas A. Knotts IV, W. Vincent Wilding, William G. Pitt, and Morris D. Argyle Department of Chemical Engineering, Brigham Young University Assessment of student proficiency in expected outcomes, whether on the course or program level, is an important aspect of curriculum development in engineering programs. The reasons for such assessment range from desires to improve student learning to fulfilling requirements of various accreditation bodies. But regardless of the reasons, the challenge is to develop suitable metrics which can clearly identify areas which need improvement. In order to assess student learning, the Department of Chemical Engineering at Brigham Young University has outlined multiple objectives (termed competencies) for each course in the curriculum. Each competency is designed to correspond to a specific program outcome such that assessment of mastery of the course competencies demonstrates achievement of the Program Outcomes. For several years, mastery of the competencies has been measured by direct methods by the faculty and indirectly using surveys by both faculty and students. The student surveys required each pupil to assess his or her mastery of each competency on a scale of 1‐5. Though this approach has provided a numerical evaluation of the abilities of each class as a whole, and has ensured minimum standards are kept, it has proven difficult to glean opportunities for specific improvement from these data, and changes to the curriculum have been largely prompted by the faculty surveys. In an effort to improve the student surveys, changes were recently made to the assessment methods for two courses: Chemical Engineering Thermodynamics and Plant Design. The numerical rating was removed and replaced with a simple yes/no question asking if the student felt proficient in each competency. In addition, the students were asked to select two of the competencies which were given “no” and explain the reason for the weakness. These simple adjustments greatly increased the effectiveness of the student surveys with no additional overhead cost. The data readily identify competencies which are problematic for students and (more importantly) the reasons for the struggles. This facilitates making of precise plans to improve student learning the next time the course is taught. This paper will explain this new assessment process in detail. To illustrate the value of the new procedures, the results of the new method will be compared with those of the traditional method (numerical 1‐5 scale). Emphasis will be placed on showing how the new method not only provides better data, but does so in a time‐efficient manner and makes “closing‐the‐loop” easy. Taken as a whole, the process is such an advancement over the previous method that it instills a sense of excitement for the assessment process that is not usually present in discussions on the topic.

Citation
Format

Knotts, T. A., & Wilding, W. V., & Pitt, W. G., & Argyle, M. D. (2012, June), A New Assessment Method to Easily Identify Areas Needing Improvement in Course-level Learning Outcomes Paper presented at 2012 ASEE Annual Conference & Exposition, San Antonio, Texas. 10.18260/1-2--20838

TY  - CPAPER
AB  - ANewAssessmentMethodtoEasilyIdentifyAreasNeedingImprovementinCourse‐levelLearningOutcomesThomas A. Knotts IV, W. Vincent Wilding, William G. Pitt, and Morris D. Argyle Department of Chemical Engineering, Brigham Young University  Assessment of student proficiency in expected outcomes, whether on the course or program level, is an important aspect of curriculum development in engineering programs.   The reasons for such assessment range from desires to improve student learning to fulfilling requirements of various accreditation bodies.  But regardless of the reasons, the challenge is to develop suitable metrics which can clearly identify areas which need improvement. In order to assess student learning, the Department of Chemical Engineering at Brigham Young University has outlined multiple objectives (termed competencies) for each course in the curriculum.  Each competency is designed to correspond to a specific program outcome such that assessment of mastery of the course competencies demonstrates achievement of the Program Outcomes. For several years, mastery of the competencies has been measured by direct methods by the faculty and indirectly using surveys by both faculty and students.  The student surveys required each pupil to assess his or her mastery of each competency on a scale of 1‐5.  Though this approach has provided a numerical evaluation of the abilities of each class as a whole, and has ensured minimum standards are kept, it has proven difficult to glean opportunities for specific improvement from these data, and changes to the curriculum have been largely prompted by the faculty surveys.   In an effort to improve the student surveys, changes were recently made to the assessment methods for two courses: Chemical Engineering Thermodynamics and Plant Design.  The numerical rating was removed and replaced with a simple yes/no question asking if the student felt proficient in each competency.  In addition, the students were asked to select two of the competencies which were given “no” and explain the reason for the weakness.   These simple adjustments greatly increased the effectiveness of the student surveys with no additional overhead cost.  The data readily identify competencies which are problematic for students and (more importantly) the reasons for the struggles.  This facilitates making of precise plans to improve student learning the next time the course is taught.  This paper will explain this new assessment process in detail.  To illustrate the value of the new procedures, the results of the new method will be compared with those of the traditional method (numerical 1‐5 scale).  Emphasis will be placed on showing how the new method not only provides better data, but does so in a time‐efficient manner and makes “closing‐the‐loop” easy.  Taken as a whole, the process is such an advancement over the previous method that it instills a sense of excitement for the assessment process that is not usually present in discussions on the topic. 
AU  - Thomas Allen Knotts IV
AU  - W. Vincent Wilding
AU  - William G. Pitt
AU  - Morris D. Argyle
CY  - San Antonio, Texas
DA  - 2012/06/10
PB  - ASEE Conferences
TI  - A New Assessment Method to Easily Identify Areas Needing Improvement in Course-level Learning Outcomes
UR  - https://peer.asee.org/20838
DO  - 10.18260/1-2--20838
ER  -