Analysis of students' GenAI prompts during a practical SQL test

This study investigates the integration of Generative AI (GenAI) into an SQL programming assessment for a cohort of 1,304 students. The assessment consists of two components: a closed-book component and a GenAI-assisted component, which allows students to use AI tools during the test. A custom application recorded students’ interactions with GenAI in the test environment to provide insights into its impact in an authentic assessment setting. Both quantitative and qualitative methodologies have been used to explore the relationship between the types of prompts used by students and their assessment performance. The analysis of the data yields several significant findings regarding the adoption of GenAI among students. Key findings include a high GenAI adoption rate, with 93.91% of students using the AI tool during the test. Interestingly, one in ten high performers (scoring well on both test components) chose not to use GenAI. Also, 68 students who had a low score in the closed-book component but high in the GenAI-assisted component engaged with GenAI the most, averaging about 10 prompts per student. The analysis of these 676 prompts highlights three key strategies for score improvement: copy-pasting the test question, enhancing the meaning of the interactions through contextualisation, and debugging. These findings also reveal the different ways students engage with GenAI tools to improve their performance.