West, M., & Ita, M. E., & Rumreich, L., & Kajfez, R. L., & Kecskemety, K. M. (2021, July), Development of a Direct Assessment for Measuring Students’ Ability to Make Connections  Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--36956

\bibitem{asee_peer_36956}
  West, M., \& Ita, M. E., \& Rumreich, L., \& Kajfez, R. L., \& Kecskemety, K. M. (2021, July), \emph{Development of a Direct Assessment for Measuring Students’ Ability to Make Connections}
  Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference.
  10.18260/1-2--36956

Meg West, Meagan Eleanor Ita, Laine Rumreich, Rachel Louis Kajfez, and Krista M. Kecskemety.  "Development of a Direct Assessment for Measuring Students’ Ability to Make Connections".  2021 ASEE Virtual Annual Conference Content Access, Virtual Conference, 2021, July.  ASEE Conferences, 2021.  https://strategy.asee.org/36956 Internet.  08 Jun, 2024

\bibitem{asee_peer_36956}
  Meg West, Meagan Eleanor Ita, Laine Rumreich, Rachel Louis Kajfez, and Krista M. Kecskemety.
  "Development of a Direct Assessment for Measuring Students’ Ability to Make Connections".
  \emph{2021 ASEE Virtual Annual Conference Content Access, Virtual Conference, 2021, July}.
  ASEE Conferences, 2021.
  https://strategy.asee.org/36956 Internet.  08 Jun, 2024

@INPROCEEDINGS{asee_peer_36956
author = "Meg West, Meagan Eleanor Ita, Laine Rumreich, Rachel Louis Kajfez, and Krista M. Kecskemety"
title = "Development of a Direct Assessment for Measuring Students’ Ability to Make Connections"
booktitle = "2021 ASEE Virtual Annual Conference Content Access"
year = "2021"
month = "July"
address = "Virtual Conference"
publisher = "ASEE Conferences"
note = {https://strategy.asee.org/36956}
number = {10.18260/1-2--36956}
}

TY  - CPAPER
AB  - The integration of entrepreneurial minded learning (EML) into engineering courses to develop students’ entrepreneurial mindset (EM) is growing in popularity through efforts such as the Kern Entrepreneurial Engineering Network (KEEN). As the integration of EML occurs, it is important to assess students’ development of an EM to inform the efficacy of course changes. Following KEEN’s definition, we operationalize EM as the ability to create value, make connections, and be curious. While we acknowledge the importance of all aspects of EML, in this full paper, we focus on assessing students’ ability to make connections. Concept maps have been previously used to assess student development of an EM as a whole, and we believe they can also be used to specifically assess the ability of students to make connections. Specifically, we collected responses from a pre-existing individual concept map activity used in two sections of a first-year engineering course and two sections of an aerospace engineering course at The Ohio State University. A total of 238 responses were collected, 106 responses from the first-year engineering course and 132 responses from the aerospace engineering course. The concept maps were evaluated using the traditional concept map scoring method. Through our analysis, we found no strong correlation between course grades and scores on concept maps developed by students for the course. This result supports moving forward with the concept map scoring methodology without the need for a correction factor related to grades. That said, other results indicated the need for modifications to the concept map instructions and scoring method that accounted for intra-hierarchy connections. In future studies, we will explore these findings further including the possibility of creating a new concept map scoring method with a stronger focus on measuring connections.

For this work, we collected responses from a pre-existing concept map activity used in 2 sections of a first-year engineering course and 2 sections of an aerospace engineering course at a large, public Midwest university. The activity in both courses involved students individually creating a concept map of their course at the end of the semester. A total of 200 responses were collected, 135 responses from the first-year engineering course and 170 responses from the aerospace engineering course. The concept maps were evaluated using the Integrated Rubric for Scoring Concept Maps. This rubric assesses concept maps for their comprehensiveness, organization, and correctness. One concern of using the Integrated Rubric was the possibility of bias that could occur due to the potential connection between students’ success in the course and their ability to make meaningful connections in the concept map about the course. This potential bias would provide an inaccurate measure of students’ ability to make connections. Using each student’s concept map score and final grade in the course, statistical analysis was performed to determine if the correlation between the scores and grades was high enough to warrant a change in the assessment methodology.  This change would account for potential bias if this activity were to be used to measure students’ ability to make connections. 

Through our analysis, which is currently ongoing but will be completed by the conference paper deadline, we posit there will be a strong positive correlation between students’ final grade and their concept map score. This analysis will inform modifications made to the direct assessment methodology including possibly adding a correction factor based on student success in the course so the concept mapping activity can be used to measure students’ ability to make connections. The direct assessment developed as a result of this study will be used in future studies related to the integration of EML into various engineering course content.
AU  - Meg West
AU  - Meagan Eleanor Ita
AU  - Laine Rumreich
AU  - Rachel Louis Kajfez
AU  - Krista M. Kecskemety
CY  - Virtual Conference
DA  - 2021/07/26
PB  - ASEE Conferences
TI  - Development of a Direct Assessment for Measuring Students’ Ability to Make Connections
UR  - https://strategy.asee.org/36956
DO  - 10.18260/1-2--36956
ER  -