Research seminars

13 February: Majeed Kazemi (University of Toronto)

Can large language models that generate code help K–12 students effectively learn Python programming?

Large language models (LLMs) can generate Python code from natural language descriptions, which may help students learn to program by reducing frustration and increasing self-efficacy. Majeed Kazemi explores whether this is true by using Codex, a LLM code generator. Majeed asked two groups of students — one with access to Codex and one without — to perform a range of coding tasks that consisted of a code-authoring part and a code-modification part. He found that the Codex students completed more tasks, scored higher, and performed better in unassisted assessments a week later, with no detriment to their performance on the manual code-modification tasks.

Majeed identified four main approaches employed by the students. The ‘AI single prompt’ approach resulted in both the highest scores on code-authoring tasks, and the lowest correctness scores on code-modification tasks. This highlights both the potentials and pitfalls of LLM code generators in educational settings, and the need for balanced integration and curriculum development.

Majeed Kazemi, a Computer Science PhD student at the University of Toronto under Professor Tovi Grossman, explores the intersection of HCI and computing education. With a focus on developer and creativity support tools, his work delves into tangible programming, data-driven visual programming, and LLM-powered coding assistants' impact on computing education. His PhD thesis focuses on scaling computing education in self-paced environments.

12 March: Yash Tadimalla & Prof. Mary Lou Maher (University of North Carolina at Charlotte)

The impact of AI tools on the student experience in programming courses: A preliminary study with an intersectional analysis approach ​

AI tools like ChatGPT have significant potential benefits for students: they allow students to work constructively on their own schedule and they provide an easy way to ask for help. However, before these tools can be meaningfully introduced into a course, their impact on a student’s ability to learn must be assessed and concerns that they lead to lower competence and poorer programming skills must be addressed. 

Yash Tadimalla and Prof. Mary Lou Maher have performed a preliminary study to understand university students’ experiences of learning to program using AI tools. The students were asked to perform a programming task under one of three conditions: 

1. With no external help

2. With the help of an AI chatbot

3. With the help of a generative AI tool like GitHub Copilot

The mixed method study also included pre- and post-task assessments and interviews with the students about their experience and perceptions of the tools. The students demonstrated a range of attitudes and behaviours towards the AI tools and these preliminary findings provide insights for future research and plans for incorporating such AI tools into a course.

Yash Tadimalla is a PhD student in the College of Computing and Informatics at UNC Charlotte, where he is pursuing an interdisciplinary degree in Computer Science and Sociology. His research explores how an individual's identity influences their interaction with and learning of technology, particularly in the domains of artificial intelligence and computer science education.

Professor Mary Lou Maher is a professor in the Department of Software and Information Systems at UNC Charlotte, and also directs the Center for Education Innovation and co-directs the Human-Centered Computing Lab. Her research spans integrated cognitive and deep learning models in AI, ethical human–AI interaction, cognitive models of curiosity for education technology, AI-based co-creativity design, and inclusive computer science education.

16 April: Dr Brett A. Becker (University College Dublin)

Generative AI in programming education: Bridging the gap from school to what lies ahead

Secondary school students and teachers have particular needs and face specific constraints that result in a unique landscape in terms of programming education. The rise of generative AI complicates this scene but also provides several potential opportunities. One of the greatest challenges in this arena is that while university students studying computing have a high likelihood of continuing on a path in computing, most school students choose to go into non-computing disciplines for further/higher education and/or their careers. This talk will focus on how generative AI may be effectively utilised in secondary school programming education and how it can be leveraged so that students can be best prepared for continuing their education or beginning their careers, regardless of discipline.

Dr Brett A. Becker is an Assistant Professor at University College Dublin in the School of Computer Science. His research area is computing education broadly construed, normally focusing on the psychology of programming, programming error messages, novice programmer behaviour, metacognition, and generative AI in education. In 2020, he was awarded a National Forum Teaching & Learning Research Fellowship, Ireland’s most prestigious national award in higher education.