Prevost, L. B., & Haudek, K. C., & Henry, E. N., & Berry, M. C., & Urban-Lurain, M. (2013, June), Automated Text Analysis Facilitates Using Written Formative Assessments for Just-in-Time Teaching in Large Enrollment Courses  Paper presented at 2013 ASEE Annual Conference & Exposition, Atlanta, Georgia. 10.18260/1-2--19250

\bibitem{asee_peer_19250}
  Prevost, L. B., \& Haudek, K. C., \& Henry, E. N., \& Berry, M. C., \& Urban-Lurain, M. (2013, June), \emph{Automated Text Analysis Facilitates Using Written Formative Assessments for Just-in-Time Teaching in Large Enrollment Courses}
  Paper presented at 2013 ASEE Annual Conference \& Exposition, Atlanta, Georgia.
  10.18260/1-2--19250

Luanna B Prevost, Kevin C Haudek, Emily Norton Henry, Matthew C Berry, and Mark Urban-Lurain.  "Automated Text Analysis Facilitates Using Written Formative Assessments for Just-in-Time Teaching in Large Enrollment Courses".  2013 ASEE Annual Conference & Exposition, Atlanta, Georgia, 2013, June.  ASEE Conferences, 2013.  https://peer.asee.org/19250 Internet.  15 May, 2024

\bibitem{asee_peer_19250}
  Luanna B Prevost, Kevin C Haudek, Emily Norton Henry, Matthew C Berry, and Mark Urban-Lurain.
  "Automated Text Analysis Facilitates Using Written Formative Assessments for Just-in-Time Teaching in Large Enrollment Courses".
  \emph{2013 ASEE Annual Conference \& Exposition, Atlanta, Georgia, 2013, June}.
  ASEE Conferences, 2013.
  https://peer.asee.org/19250 Internet.  15 May, 2024

@INPROCEEDINGS{asee_peer_19250
author = "Luanna B Prevost, Kevin C Haudek, Emily Norton Henry, Matthew C Berry, and Mark Urban-Lurain"
title = "Automated Text Analysis Facilitates Using Written Formative Assessments for Just-in-Time Teaching in Large Enrollment Courses"
booktitle = "2013 ASEE Annual Conference \& Exposition"
year = "2013"
month = "June"
address = "Atlanta, Georgia"
publisher = "ASEE Conferences"
note = {https://peer.asee.org/19250}
number = {10.18260/1-2--19250}
}

TY  - CPAPER
AB  -            Automated Text Analysis Facilitates Using Written Formative        Assessments for Just-in-Time Teaching in Large Enrollment CoursesWritten formative assessments can provide instructors with rich insight into students’ thinkingabout scientific concepts. However, the time and effort involved in grading deter instructors fromhaving students write in large courses. As large-enrollment introductory STEM courses becomeincreasingly common, the need for innovations that facilitate the use of written assessmentscontinues to grow.We explored the use of automated text analysis to overcome these obstacles and facilitate the useof written formative assessment in a large-enrollment introductory biology course. Studentresponses to online homework on reaction thermodynamics were collected in two 300-personcourse sections one day before the next class meeting. We performed a two step analysis ofstudents’ written responses. First we used automated text analysis to extract and categorizethermodynamics concepts from student writing. Second, we used K-means cluster analysis toaggregate responses into distinct groups. We used IBM SPSS Modeler software to create“streams” which automate the complex series of analyses. We developed and piloted a rapidfeedback system to provide instructors feedback on students’ responses before the next classperiod ( less than one working day), so that they could use this feedback to inform theirinstruction.Rapid feedback reports presented the instructors with the number of clusters of responses, andthe percentage of students in each cluster. The reports describe the thermodynamics conceptsexpressed in student responses as captured by lexical categories, and provide the number ofresponses assigned to each category. The reports also summarize the categories which wereimportant to the clustering model and the category means for each cluster. For each cluster asample of responses closest to the cluster centroid, and therefore most representative of thecluster, was included. Additionally, instructors were provided with web diagrams - visualrepresentations of students’ ideas and the connections made among these ideas.Using this method in two course sections, we observed differences in students’ writing aboutreaction thermodynamics at the start of the semester. For example, our analysis isolated a clusterof students that identified the effect of increased temperature on reaction rate, but failed topropose a mechanism. This information was presented to the instructor in the rapid feedbackreport, so that mechanisms could be addressed during the next class meeting. Instructors alsoreceived examples of student writing that demonstrated student difficulties with thermodynamicconcepts, such as confusing reaction free energy and activation energy, or conflating bondstrength with reaction rate. The instructor incorporated students’ misconceptions into clickerquestions to tackle these misconceptions in class the following day. Our results suggest thatautomated text analysis paired with statistical analyses facilitates student writing in largeenrollment courses and provides instructors with feedback for just-in-time teaching.The full paper will include examples of the reports and more details about how the instructorsused the reports in their teaching.
AU  - Luanna B Prevost
AU  - Kevin C Haudek
AU  - Emily Norton Henry
AU  - Matthew C Berry
AU  - Mark Urban-Lurain
CY  - Atlanta, Georgia
DA  - 2013/06/23
PB  - ASEE Conferences
TI  - Automated Text Analysis Facilitates Using Written Formative Assessments for Just-in-Time Teaching in Large Enrollment Courses
UR  - https://peer.asee.org/19250
DO  - 10.18260/1-2--19250
ER  -