Math Assessment as an Indicator of Program Retention

Carrie Francis; Michael Jacobson

Download Paper | Permalink

Conference: 2022 ASEE Annual Conference & Exposition
Location: Minneapolis, MN
Publication Date: August 23, 2022
Start Date: June 26, 2022
End Date: June 29, 2022
Conference Session: First-Year Programs Division Technical Session 8: Academic Progress, Retention, and Mathematics
Page Count: 10
DOI: 10.18260/1-2--41495
Permanent URL: https://strategy.asee.org/41495
Download Count: 238

Request a correction

After completing his Bachelors and Masters of Science degrees in Electrical Engineering at the University of Minnesota he supported pharmacology researchers while working at Data Sciences International. At the United Arab Emirates University, he researched instrumentation of physiological signals related to diabetes mellitus and earned a Ph.D. at Edinburgh Napier University, UK. Currently, he is serving as Professor of Engineering at the University of Northwestern in St Paul, MN.

visit author page

Download Paper | Permalink

Abstract

A study evaluated the preparedness of first year students enrolled in the Bachelor of Science in Engineering program at a the University of Northwestern-St. Paul, a small private master’s university in the Midwestern United States. Retention statistics indicated that most students leave during the first year program, which is dominated by Calculus and Introduction to Engineering courses. Student preparedness for the engineering program was assessed through a fundamental skills questionnaire. The results identified some shortcomings in mathematical preparation. It was hypothesized that sufficient skills in mathematics indicates a student’s preparedness for the engineering program, and as such, improves first year program retention.

During the first session of an Introduction to Engineering course, students completed a ten topic, selected answer, questionnaire that briefly assessed skills in manipulating standard international (SI) units, solving basic word problems, relating functions to graphs, and estimating volume. The questionnaire was repeated for three subsequent, annual cohorts (n=30, 31, and 40, respectively), and analyzed per question. Shortcomings were identified as an average performance less than 70%. The results indicated weaknesses in abilities to solve by consideration of unit dimensions, interpret graphical relationships, and estimate volume with SI units.

Students in the engineering program are expected to be ready for calculus I. The calculus course perquisite is based on student performance in a pre-calculus course, the ACT math, SAT math, or ALEKS assessment, with minimum scores set at B-, 26 or 28 (depending on year), 660, and 76, respectively. This study focused on ACT assessment as the most common at this institution. ALEKS is typically used only when the ACT score is insufficient or unavailable.

Analysis suggests that ACT performance does not predict student retention and the relationship of average performance to calculus success is unclear. ACT scores were available for 89 students, of whom 65 persisted and 24 left the program. The average ACT math scores were 27.6 ± 3.8 and 26.1 ± 4.2 (p = .052), respectively. However, success in Calculus I on the first attempt was also compared for students scoring ≥ 26 (n=60) to those scoring ≤ 25 (n=29), with results of 77% and 59%, respectively (p<0.05). Overall, an ACT math score lower than 26 may identify students who will need additional support to remain in the engineering program.

Of the 35 students who completed the ALEKS assessment, 22 remained in the program and 13 left. Both groups had identical average scores 83 ± 12 with median scores of 85 for persisting students and 86 for those who left. As such, there is no apparent correlation between the ALEKS assessment and program retention.

Apart from assessment of mathematical skill, the study considered overall academic skill by comparing the student high school grade point average (GPA) and composite ACT scores as a possible indicators of program retention. High school GPA is similar for persisting (n=69) and non-persisting (n=31) students with GPAs of 3.6 ± 0.3 and 3.6 ± 0.4 (p = 0.11), respectively. Similarly, ACT composite scores were 26.6 ± 4.1 and 26.1 ± 3.8 (p = 0.3), respectively for the persisting and non-persisting students.

Given the similarity of metrics prior to matriculation, the study then considered first term GPA and whether students took a calculus course prior to Introduction to Engineering. The GPA of persisting and non-persisting, first term students was found to be: 3.2 ± 0.6 and 2.4 ± 1.1 (p =0.02), respectively. Among students taking Introduction to Engineering, 43 had previously taken Calculus I (69.8% persisted), 51 took Calculus I in the same term (68.6% persisted). Additionally, 16 of the 31 students who did not persist in engineering either did not attempt Calculus I or did not complete it successfully on the first attempt.

From the analysis, it appears that student retention is not well correlated to standardized metrics of mathematical skill and does not predict student preparedness for program entry. Readiness for calculus at the start of the program and first semester performance may be better indicators of program persistence. Further work is needed to develop improved indicators of student preparedness for engineering and skill-building approaches to improve retention at the first semester of the program.

Citation
Format

Francis, C., & Jacobson, M. (2022, August), Math Assessment as an Indicator of Program Retention Paper presented at 2022 ASEE Annual Conference & Exposition, Minneapolis, MN. 10.18260/1-2--41495

TY - CPAPER
AB - A study evaluated the preparedness of first year students enrolled in the Bachelor of Science in Engineering program at a the University of Northwestern-St. Paul, a small private master’s university in the Midwestern United States. Retention statistics indicated that most students leave during the first year program, which is dominated by Calculus and Introduction to Engineering courses. Student preparedness for the engineering program was assessed through a fundamental skills questionnaire. The results identified some shortcomings in mathematical preparation. It was hypothesized that sufficient skills in mathematics indicates a student’s preparedness for the engineering program, and as such, improves first year program retention.

During the first session of an Introduction to Engineering course, students completed a ten topic, selected answer, questionnaire that briefly assessed skills in manipulating standard international (SI) units, solving basic word problems, relating functions to graphs, and estimating volume. The questionnaire was repeated for three subsequent, annual cohorts (n=30, 31, and 40, respectively), and analyzed per question. Shortcomings were identified as an average performance less than 70%. The results indicated weaknesses in abilities to solve by consideration of unit dimensions, interpret graphical relationships, and estimate volume with SI units.

Students in the engineering program are expected to be ready for calculus I. The calculus course perquisite is based on student performance in a pre-calculus course, the ACT math, SAT math, or ALEKS assessment, with minimum scores set at B-, 26 or 28 (depending on year), 660, and 76, respectively. This study focused on ACT assessment as the most common at this institution. ALEKS is typically used only when the ACT score is insufficient or unavailable.

Analysis suggests that ACT performance does not predict student retention and the relationship of average performance to calculus success is unclear. ACT scores were available for 89 students, of whom 65 persisted and 24 left the program. The average ACT math scores were 27.6 ± 3.8 and 26.1 ± 4.2 (p = .052), respectively. However, success in Calculus I on the first attempt was also compared for students scoring ≥ 26 (n=60) to those scoring ≤ 25 (n=29), with results of 77% and 59%, respectively (p<0.05). Overall, an ACT math score lower than 26 may identify students who will need additional support to remain in the engineering program.

Of the 35 students who completed the ALEKS assessment, 22 remained in the program and 13 left. Both groups had identical average scores 83 ± 12 with median scores of 85 for persisting students and 86 for those who left. As such, there is no apparent correlation between the ALEKS assessment and program retention.

Apart from assessment of mathematical skill, the study considered overall academic skill by comparing the student high school grade point average (GPA) and composite ACT scores as a possible indicators of program retention. High school GPA is similar for persisting (n=69) and non-persisting (n=31) students with GPAs of 3.6 ± 0.3 and 3.6 ± 0.4 (p = 0.11), respectively. Similarly, ACT composite scores were 26.6 ± 4.1 and 26.1 ± 3.8 (p = 0.3), respectively for the persisting and non-persisting students.

Given the similarity of metrics prior to matriculation, the study then considered first term GPA and whether students took a calculus course prior to Introduction to Engineering. The GPA of persisting and non-persisting, first term students was found to be: 3.2 ± 0.6 and 2.4 ± 1.1 (p =0.02), respectively. Among students taking Introduction to Engineering, 43 had previously taken Calculus I (69.8% persisted), 51 took Calculus I in the same term (68.6% persisted). Additionally, 16 of the 31 students who did not persist in engineering either did not attempt Calculus I or did not complete it successfully on the first attempt.

From the analysis, it appears that student retention is not well correlated to standardized metrics of mathematical skill and does not predict student preparedness for program entry. Readiness for calculus at the start of the program and first semester performance may be better indicators of program persistence. Further work is needed to develop improved indicators of student preparedness for engineering and skill-building approaches to improve retention at the first semester of the program.
AU - Carrie Francis
AU - Michael Jacobson
CY - Minneapolis, MN
DA - 2022/08/23
PB - ASEE Conferences
TI - Math Assessment as an Indicator of Program Retention
UR - https://strategy.asee.org/41495
DO - 10.18260/1-2--41495
ER -

Math Assessment as an Indicator of Program Retention

Paper Authors

Carrie Francis

Michael Jacobson University of Northwestern

Abstract

Citation

APA

APA - LaTeX bibitem

MLA

MLA - LaTeX bibitem

Bibtex

EndNote - RIS