Work in Progress: A Holistic PhD Admissions Rubric--Design & Implementation

Shannon Barker; Amy Clobes

Download Paper | Permalink

Conference: 2021 ASEE Virtual Annual Conference Content Access
Location: Virtual Conference
Publication Date: July 26, 2021
Start Date: July 26, 2021
End Date: July 19, 2022
Conference Session: Graduate Studies Division Technical Session 3
Tagged Division: Graduate Studies
Tagged Topic: Diversity
Page Count: 19
DOI: 10.18260/1-2--38117
Permanent URL: https://strategy.asee.org/38117
Download Count: 2222

Request a correction

Paper Authors

biography

Shannon Barker University of Virginia

visit author page

Dr. Shannon Barker completed her PhD at the University of Alabama at Birmingham, and completed two post-doctoral fellowships at the University of Washington and Ecole Polytechnique Federale de Lausanne, specializing in gene delivery. Shannon has been in graduate higher education leadership for seven years both at the Georgia Institute of Technology and the University of Virginia, and is currently the Undergraduate Program Director for the University of Virginia's Department of Biomedical Engineering.

visit author page

biography

Amy Clobes University of Virginia orcid.org/0000-0001-7095-270X

visit author page

Dr. Amy M. Clobes is committed to supporting current and future graduate students as Director of Graduate Programs for the University of Virginia School of Engineering and Applied Science. In her current role, Dr. Clobes collaborates to support existing programs and develops new initiatives in graduate student recruitment, training, education, and career and professional development. Dr. Clobes holds a B.S. in Biology from the University of Michigan and Ph.D. in Biomedical Engineering from the University of Virginia. Her combined experience in STEM research and education, program development, and student advising are key to her dedication and success in creating opportunities for graduate students to achieve their education and career goals.

visit author page

Download Paper | Permalink

Abstract

Addressing today’s complex engineering challenges requires that all perspectives and lived experiences are brought to the table. The NSF and other organizations have stressed the importance of bringing a more representative proportion of the national population into the professional STEM research pipeline. By investing in building a robust and diverse community of engineers, we will have access to a workforce with the insight into societal needs needed to solve today’s problems

A comprehensive graduate student recruitment program must focus on increasing the diversity of its applicant pool and matriculating cohorts. Critical to this goal is the use of evidence-based best practices during the graduate admissions process. The admissions process should ensure that every student comes into the school with the same potential for success, regardless of background and experiences. It is therefore crucial that the process rely on true indicators of graduate school success, such as research interest and potential, motivation and persistence, and potential for leadership; while de-emphasizing poorer predictors, including traditional academic preparation indicators and GRE scores.

To this end, a Holistic PhD Admissions Rubric was developed and implemented for use by all graduate programs in the engineering school of a large R1 University. This rubric is an evaluation tool built into the central admissions system that aids faculty reviewers through best practices in PhD admissions. The rubric facilitates the use of these practices while still permitting a degree of uniformity to the review process, albeit one that further ensures excellence, access, and equity. The rubric utilizes nine criteria, including metrics such as letters of evaluation scores, demonstration of research potential/vision, evidence of motivation/persistence/ability to overcome obstacles, and potential for leadership. Each criterion includes suggestions on where in the application materials to seek out that information, as well as descriptors for each criterion’s numerical rating. A score is given for each criterion and then all criteria scores are totaled and used as part of the overall admissions evaluation.

Faculty dialogues, meetings, and educational emails were used to increase implementation of the rubric across the school beginning with the admissions cycle for students matriculating in the Fall of 2019. The rubric was updated for the admissions cycle for students matriculating in the Fall of 2020 based on continued feedback of faculty users, and review by premier experts in the field of holistic graduate admissions. In the admissions cycle for students matriculating in the Fall of 2020, 57% of admitted PhD students had a completed rubric in their application file, and we expect this number to continue to increase.

Our research team plans to conduct a survey of the faculty who review PhD applications and a series of focus groups with graduate program admission directors in order to better understand the role the rubric takes in the admissions process. We want to understand how faculty are using the rubric and whether they find it helpful and easy to use. Additionally, we want to know if the rubric has changed how faculty approach the PhD application evaluation process and the value they place on various application metrics. We will also assess faculty perspective on the quality of student being admitted into their programs both before and after implementation of the rubric. This review of faculty perspective will then be compared to admissions data in the two years prior to rubric implementation and two years after, concentrating specifically on GRE scores and GPA, as well as demographic data. We will also assess the first year GPA and first year retention rate of the class matriculating in the Fall of 2019 (the first year of rubric implementation) as a work-in-progress. Future work will focus on longer-term retention rates, as well as other metrics of graduate school success, including time-to-degree and research productivity.

Citation
Format

Barker, S., & Clobes, A. (2021, July), Work in Progress: A Holistic PhD Admissions Rubric--Design & Implementation Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--38117

TY - CPAPER
AB - Addressing today’s complex engineering challenges requires that all perspectives and lived experiences are brought to the table. The NSF and other organizations have stressed the importance of bringing a more representative proportion of the national population into the professional STEM research pipeline. By investing in building a robust and diverse community of engineers, we will have access to a workforce with the insight into societal needs needed to solve today’s problems

A comprehensive graduate student recruitment program must focus on increasing the diversity of its applicant pool and matriculating cohorts. Critical to this goal is the use of evidence-based best practices during the graduate admissions process. The admissions process should ensure that every student comes into the school with the same potential for success, regardless of background and experiences. It is therefore crucial that the process rely on true indicators of graduate school success, such as research interest and potential, motivation and persistence, and potential for leadership; while de-emphasizing poorer predictors, including traditional academic preparation indicators and GRE scores.

To this end, a Holistic PhD Admissions Rubric was developed and implemented for use by all graduate programs in the engineering school of a large R1 University. This rubric is an evaluation tool built into the central admissions system that aids faculty reviewers through best practices in PhD admissions. The rubric facilitates the use of these practices while still permitting a degree of uniformity to the review process, albeit one that further ensures excellence, access, and equity. The rubric utilizes nine criteria, including metrics such as letters of evaluation scores, demonstration of research potential/vision, evidence of motivation/persistence/ability to overcome obstacles, and potential for leadership. Each criterion includes suggestions on where in the application materials to seek out that information, as well as descriptors for each criterion’s numerical rating. A score is given for each criterion and then all criteria scores are totaled and used as part of the overall admissions evaluation.

Faculty dialogues, meetings, and educational emails were used to increase implementation of the rubric across the school beginning with the admissions cycle for students matriculating in the Fall of 2019. The rubric was updated for the admissions cycle for students matriculating in the Fall of 2020 based on continued feedback of faculty users, and review by premier experts in the field of holistic graduate admissions. In the admissions cycle for students matriculating in the Fall of 2020, 57% of admitted PhD students had a completed rubric in their application file, and we expect this number to continue to increase.

Our research team plans to conduct a survey of the faculty who review PhD applications and a series of focus groups with graduate program admission directors in order to better understand the role the rubric takes in the admissions process. We want to understand how faculty are using the rubric and whether they find it helpful and easy to use. Additionally, we want to know if the rubric has changed how faculty approach the PhD application evaluation process and the value they place on various application metrics. We will also assess faculty perspective on the quality of student being admitted into their programs both before and after implementation of the rubric. This review of faculty perspective will then be compared to admissions data in the two years prior to rubric implementation and two years after, concentrating specifically on GRE scores and GPA, as well as demographic data. We will also assess the first year GPA and first year retention rate of the class matriculating in the Fall of 2019 (the first year of rubric implementation) as a work-in-progress. Future work will focus on longer-term retention rates, as well as other metrics of graduate school success, including time-to-degree and research productivity.

AU - Shannon Barker
AU - Amy Clobes
CY - Virtual Conference
DA - 2021/07/26
PB - ASEE Conferences
TI - Work in Progress: A Holistic PhD Admissions Rubric--Design & Implementation
UR - https://strategy.asee.org/38117
DO - 10.18260/1-2--38117
ER -