ACT Preparation and the Percent of Variability in First-Year Engineering Student GPA Explained by ACT Scores

Teresa Lee Tinnell; Nora Honken; Patricia A. Ralston

Download Paper | Permalink

Conference: 2020 ASEE Virtual Annual Conference Content Access
Location: Virtual On line
Publication Date: June 22, 2020
Start Date: June 22, 2020
End Date: June 26, 2021
Conference Session: First-Year Programs: Cornucopia #2
Tagged Division: First-Year Programs
Page Count: 13
DOI: 10.18260/1-2--34089
Permanent URL: https://peer.asee.org/34089
Download Count: 445

Request a correction

Paper Authors

biography

Teresa Lee Tinnell University of Louisville orcid.org/0000-0002-2768-919X

visit author page

Terri Tinnell is a STEM Education PhD Candidate and Research Assistant with the Engineering Fundamentals Department in JB Speed School of Engineering at the University of Louisville.
Research interests include: interdisciplinary faculty development, first-year engineering student retention, STEM teacher education, and academic experiences within environments that encourage collaborative, active, and team-based learning (such as Makerspaces).

visit author page

biography

Nora Honken University of Cincinnati

visit author page

Nora is an Assistant Professor in the Engineering Education Department at The University of Cincinnati. She holds a PhD in Educational Leadership and Organizational Development for the University of Louisville, a MS in Industrial Engineering from Arizona State University and a BS in Industrial Engineering from Virginia Tech. She also has extensive industrial experience.

visit author page

biography

Patricia A. Ralston University of Louisville

visit author page

Dr. Patricia A. S. Ralston is Professor and Chair of the Department of Engineering Fundamentals at the University of Louisville. She received her B.S., MEng, and PhD degrees in chemical engineering from the University of Louisville. Dr. Ralston teaches undergraduate engineering mathematics and is currently involved in educational research on the effective use of technology in engineering education, the incorporation of critical thinking in undergraduate engineering education, and retention of engineering students. She leads a research group whose goal is to foster active interdisciplinary research which investigates learning and motivation and whose findings will inform the development of evidence-based interventions to promote retention and student success in engineering. Her fields of technical expertise include process modeling, simulation, and process control.

visit author page

Download Paper | Permalink

Abstract

This Complete Research paper investigates if the amount and type of preparation for the ACT is related to the amount of variability in a first year GPA of engineering students that can be explained by ACT scores. Looking specifically at engineering student college success, many studies examining engineering student’s first-year performance have used scores on either the ACT or the Scholastic Aptitude Test (SAT) as either a predictor or covariate (DeBerard, Spielmans, & Julka, 2004; French, Immekus, & Oakes, 2005). In some studies of general college student performance, ACT/SAT scores have been shown to not be statistically significant when related to college GPA (Pritchard & Wilson, 2003). Other research has shown a decreasing trend in the amount of variability in first-year engineering student’s GPA that can be explained by ACT scores (Maruyama, 2012). Since 2012 research at our institution has also shown a decreasing trend in the amount of variability in first year GPA of engineering students that can be explained by ACT scores from nearly 20% in 2012 to 5% in 2017. While some universities have stopped using standardized entrance exams, due to the belief that they do not accurately predict success (PBS, 2019), other universities use scores on these exams as a criteria for entrance to the university and honors programs, as well as for scholarship awards.

There are multiple methods of ACT/SAT test preparation available to students with costs ranging from free to thousands of dollars. In 2017 after analyzing scores from over 200,000 students, the College Board concluded the students who spent at least 20 hours of practice and student with the free official SAT practice on Khan Academy increased their score on average 115 points, which was twice the increase of students who did not use the free service (College Board, 2017). Research looking at other methods of preparation have also shown an increase in test scores related to test preparation (Moore et al, 2018).

The purpose of this research is start to understand why students’ ACT scores might be losing predictive ability. More specifically we seek to understand if the ACT scores of engineering students who complete more hours or different types of test preparation might explain less variability in their first year GPA, than scores earned by students who did less or no preparation. This study is relevant not only due to the high stakes impact of ACT scores at many universities, but also the potential to disadvantage students who do not have the means or are not encouraged to participate in costly and time consuming ACT/SAT preparation; student, who might have equal probability of success as someone who received a higher score due to their method of test preparation.

Data on ACT/SAT preparation methods that students used and estimates of time spent in each method of preparation were self-reported on a survey administered to first-year engineering students the first week of class in the fall semester of 2017. Students were asked for each of the following methods of preparation: self- study, courses, one on one tutoring: When preparing for your standardized College Entrance Exam (e.g., SAT, ACT), please estimate how many hours total you spent in each activity listed. Possible responses were 0, 1-5 hours, over 5 hours.

The same survey questions were administered at two large metropolitan universities. At one institution, class time was given for students to complete the survey. At the other institution students were given the survey as a part of a homework assignment. The resulting sample size was over 1,500 and the response rate was over 90%. Initial analysis of the responses to survey questions show variability in the methods and the time students spent preparing for the ACT or SAT. Demographic and performance data were extracted from university records. The data will be analyzed using regression analysis. If the subsamples are large enough to ensure statistical validity, gender, race and Pell grant eligibility will be considered in the analysis. If the results of this study show a significant difference in the predictive ability of ACT scores based on student’s preparation methods and time, this could have a major impact on engineering education research. Specifically researchers trying to explain variability in engineering student performance using ACT scores would have to also consider the variability in preparation for the test.

Citation
Format

Tinnell, T. L., & Honken, N., & Ralston, P. A. (2020, June), ACT Preparation and the Percent of Variability in First-Year Engineering Student GPA Explained by ACT Scores Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line . 10.18260/1-2--34089

TY - CPAPER
AB - This Complete Research paper investigates if the amount and type of preparation for the ACT is related to the amount of variability in a first year GPA of engineering students that can be explained by ACT scores. Looking specifically at engineering student college success, many studies examining engineering student’s first-year performance have used scores on either the ACT or the Scholastic Aptitude Test (SAT) as either a predictor or covariate (DeBerard, Spielmans, &amp;amp; Julka, 2004; French, Immekus, &amp;amp; Oakes, 2005). In some studies of general college student performance, ACT/SAT scores have been shown to not be statistically significant when related to college GPA (Pritchard &amp;amp; Wilson, 2003). Other research has shown a decreasing trend in the amount of variability in first-year engineering student’s GPA that can be explained by ACT scores (Maruyama, 2012). Since 2012 research at our institution has also shown a decreasing trend in the amount of variability in first year GPA of engineering students that can be explained by ACT scores from nearly 20% in 2012 to 5% in 2017. While some universities have stopped using standardized entrance exams, due to the belief that they do not accurately predict success (PBS, 2019), other universities use scores on these exams as a criteria for entrance to the university and honors programs, as well as for scholarship awards.

There are multiple methods of ACT/SAT test preparation available to students with costs ranging from free to thousands of dollars. In 2017 after analyzing scores from over 200,000 students, the College Board concluded the students who spent at least 20 hours of practice and student with the free official SAT practice on Khan Academy increased their score on average 115 points, which was twice the increase of students who did not use the free service (College Board, 2017). Research looking at other methods of preparation have also shown an increase in test scores related to test preparation (Moore et al, 2018).

The purpose of this research is start to understand why students’ ACT scores might be losing predictive ability. More specifically we seek to understand if the ACT scores of engineering students who complete more hours or different types of test preparation might explain less variability in their first year GPA, than scores earned by students who did less or no preparation. This study is relevant not only due to the high stakes impact of ACT scores at many universities, but also the potential to disadvantage students who do not have the means or are not encouraged to participate in costly and time consuming ACT/SAT preparation; student, who might have equal probability of success as someone who received a higher score due to their method of test preparation.

Data on ACT/SAT preparation methods that students used and estimates of time spent in each method of preparation were self-reported on a survey administered to first-year engineering students the first week of class in the fall semester of 2017. Students were asked for each of the following methods of preparation: self- study, courses, one on one tutoring: When preparing for your standardized College Entrance Exam (e.g., SAT, ACT), please estimate how many hours total you spent in each activity listed. Possible responses were 0, 1-5 hours, over 5 hours.

The same survey questions were administered at two large metropolitan universities. At one institution, class time was given for students to complete the survey. At the other institution students were given the survey as a part of a homework assignment. The resulting sample size was over 1,500 and the response rate was over 90%. Initial analysis of the responses to survey questions show variability in the methods and the time students spent preparing for the ACT or SAT. Demographic and performance data were extracted from university records. The data will be analyzed using regression analysis. If the subsamples are large enough to ensure statistical validity, gender, race and Pell grant eligibility will be considered in the analysis.

If the results of this study show a significant difference in the predictive ability of ACT scores based on student’s preparation methods and time, this could have a major impact on engineering education research. Specifically researchers trying to explain variability in engineering student performance using ACT scores would have to also consider the variability in preparation for the test.

AU - Teresa Lee Tinnell
AU - Nora Honken
AU - Patricia A. Ralston
CY - Virtual On line
DA - 2020/06/22
PB - ASEE Conferences
TI - ACT Preparation and the Percent of Variability in First-Year Engineering Student GPA Explained by ACT Scores
UR - https://peer.asee.org/34089
DO - 10.18260/1-2--34089
ER -