Ai, Y., & Baveja, M., & Girdhar, A., & O'Dell, M., & Deorio, A. (2024, June), A Custom Generative AI Chatbot as a Course Resource  Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/46433

\bibitem{asee_peer_46433}
  Ai, Y., \& Baveja, M., \& Girdhar, A., \& O'Dell, M., \& Deorio, A. (2024, June), \emph{A Custom Generative AI Chatbot as a Course Resource}
  Paper presented at 2024 ASEE Annual Conference \& Exposition, Portland, Oregon.
  https://peer.asee.org/46433

Yutong Ai, Maya Baveja, Akanksha Girdhar, Melina O'Dell, and Andrew Deorio.  "A Custom Generative AI Chatbot as a Course Resource".  2024 ASEE Annual Conference & Exposition, Portland, Oregon, 2024, June.  ASEE Conferences, 2024.  https://peer.asee.org/46433 Internet.  30 Jul, 2024

\bibitem{asee_peer_46433}
  Yutong Ai, Maya Baveja, Akanksha Girdhar, Melina O'Dell, and Andrew Deorio.
  "A Custom Generative AI Chatbot as a Course Resource".
  \emph{2024 ASEE Annual Conference \& Exposition, Portland, Oregon, 2024, June}.
  ASEE Conferences, 2024.
  https://peer.asee.org/46433 Internet.  30 Jul, 2024

@INPROCEEDINGS{asee_peer_46433
author = "Yutong Ai, Maya Baveja, Akanksha Girdhar, Melina O'Dell, and Andrew Deorio"
title = "A Custom Generative AI Chatbot as a Course Resource"
booktitle = "2024 ASEE Annual Conference \& Exposition"
year = "2024"
month = "June"
address = "Portland, Oregon"
publisher = "ASEE Conferences"
note = {https://peer.asee.org/46433}
}

TY  - CPAPER
AB  - Generative AI (GenAI) chatbots have taken the world by storm with their ability to summarize information and produce complex natural language responses. At the same time, the demand for computer science education has grown enormously.

We developed a custom GenAI chatbot and released it to over 700 students to assist with a course programming project. The tool was trained on the project specification, lecture content, lab materials, and past course forum posts relevant to the project. The course is a high-enrollment upper-level computer science elective at a large public research university. The project is a simple distributed system using processes, threads, and sockets. The goal of the tool was to provide an additional course resource that supplies valuable information to students comparable to advice they would receive from course instructors.

We measured the effectiveness of our tool with a student survey, evaluation by course instructors, and comparison with a state-of-the-art general purpose chatbot. The overall survey data indicated high rates of correctness and helpfulness in the Bot responses. We found that hallucination was not common, and most incorrect responses were identifiable by students. The Bot also performed better than general purpose bots for project-specific help.

Our experience can provide insights for faculty using GenAI to assist students in their courses. A customized chatbot can be helpful to students and augment traditional course resources.
AU  - Yutong Ai
AU  - Maya Baveja
AU  - Akanksha Girdhar
AU  - Melina O'Dell
AU  - Andrew Deorio
CY  - Portland, Oregon
DA  - 2024/06/23
PB  - ASEE Conferences
TI  - A Custom Generative AI Chatbot as a Course Resource
UR  - https://peer.asee.org/46433
ER  -