Asee peer logo

Cognitive Validation Of A Computer Based Assessment Of Problem Solving: Linking Cognitive And Online Processes

Download Paper |

Conference

2002 Annual Conference

Location

Montreal, Canada

Publication Date

June 16, 2002

Start Date

June 16, 2002

End Date

June 19, 2002

ISSN

2153-5965

Conference Session

ASEE Multimedia Session

Page Count

15

Page Numbers

7.299.1 - 7.299.15

DOI

10.18260/1-2--10720

Permanent URL

https://peer.asee.org/10720

Download Count

497

Request a correction

Paper Authors

author page

Gregory Chung

Download Paper |

Abstract
NOTE: The first page of text has been automatically extracted and included below in lieu of an abstract

Main Menu Session 2793

Cognitive Validation of a Computer-Based Assessment of Problem Solving: Linking Cognitive and Online Processes

Gregory K. W. K. Chung 1, Linda F. de Vries 1, Alicia M. Cheak1, Ronald H. Stevens2, & William L. Bewley 1 1 National Center for Research on Evaluation, Standards, and Student Testing (CRESST) / 2UCLA School of Medicine / 2 Graduate School of Education & Information Studies

Abstract

In this study we tested a novel cognitive validation strategy that yoked participants’ verbal protocols with their clickstream data using a problem solving assessment (IMMEX—Interactive MultiMedia EXercises). Participants were presented with a scenario and provided with relevant and irrelevant information to solve the task. Participants could access the information in any order and attempt to solve the problem at any time. The most frequently occurring cognitive processes were paraphrasing text, making accurate cause-effect inferences, and monitoring of problem solving behavior. Productive processes were related to success and consistent with scientific reasoning behavior on the task and unproductive processes were related to unsuccessful performance and consistent with poor reasoning. We found strong evidence of the cognitive validity of the IMMEX task. Components of an online process-based assessment testbed are identified.

Embedded Assessment of Complex Skills

Engineering education has been a leader in the novel use of computer-based instruction. The intended use of many of the applications is to increase students’ understanding of the content and to develop their problem solving skills in a particular content area. This is consistent with recent calls for engineering schools to increase students’ problem solving skills 1-10. However, assessing cognitively complex skills such as problem solving requires use of modern assessment methods; typical course evaluations, anecdotal instructor evaluations, and surveys of student attitude are inadequate1-4. One method being explored is the use of computer-based instructional applications as assessment platforms. Essentially, the idea is to provide students with instructional tools and embed assessments of complex learning and problem solving in the tool. From the students’ perspective, the task appears instructional; however, embedded within the software are assessments of student performance11-15. This is a departure from most computer-based assessments, which are usually stand-alone assessments16-24.

Proceedings of the 2002 American Society for Engineering Education Annual Conference & Exposition Copyright Ó 2002, American Society for Engineering Education

Main Menu

Chung, G. (2002, June), Cognitive Validation Of A Computer Based Assessment Of Problem Solving: Linking Cognitive And Online Processes Paper presented at 2002 Annual Conference, Montreal, Canada. 10.18260/1-2--10720

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2002 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015