Graduation Year
2025
Date of Submission
4-2025
Document Type
Open Access Senior Thesis
Degree Name
Bachelor of Arts
Department
Computer Science
Reader 1
Professor Ran Libeskind-Hadas
Terms of Use & License Information
Abstract
This thesis identifies strengths and weaknesses of Large Language Models
(LLMs) in a game-based, academic computer science setting. Specifically, it builds a
platform for LLMs to compete on a foundational assignment for entry-level Computer
Science students: playing the game of Picobot. To enable the analysis, it reconstructs
an earlier Picobot simulation: translating from Python2 to Python3, and introducing an
LLM mode that interacts with the Application Program Interfaces (APIs) of multiple
classes of LLMs.
We find that, while the LLM models understand the existence of the Picobot
game, the LLMs require specific instructions to generate valid moves. Therefore, this
thesis project determines a Minimal Viable Prompt necessary for the LLMs to join the
game, and it determines how LLM performance improves with additional game-based
prompts, beyond what is typically provided to entry-level students. It also compares
and correlates game success with common, public LLM benchmarks.
Secondarily, to explore further uses of Generative Artificial Intelligence (Gen
AI) in Education, this thesis project builds a prototype of brAIn: a college-level
Teaching Assistant based on GenAI. Through the first version of the brAIn prototype,
we explore the nuances of full-stack prototype development, and prepare to evaluate
impact and acceptance. Preliminary findings are in the Appendix.
Recommended Citation
Wise, Jonathan J., "Artificial Intelligence Should Get to Play Games Too: Large Language Models Playing Picobot and Generative Artificial Intelligence as a Teaching Assistant" (2025). CMC Senior Theses. 4026.
https://scholarship.claremont.edu/cmc_theses/4026
