Ian M Church

One of the emergent challenges of student work submitted for assessment is the widespread use of large language models (LLMs) to support and even produce written work. This particularly affects subjects where long-form written work is a key part of assessment. We propose a novel approach to addressing this challenge, using LLMs themselves to support the assessment process. We have developed a proof-of-concept viva voce examination simulator, which accepts the student's written submission as input, generates an interactive series of questions from the LLM and answers from the student. The viva voce simulator is an interactive tool which asks questions which a human examiner might plausibly ask, and uses the student's answers to form a judgment about whether the submitted piece of work is likely to be the student's own work. The interaction transcript is provided to the human examiner to support their final judgment. We suggest theoretical and practical points which are critical to real-world deployment of such a tool.