Adaptive Game-Based Swedish Language Learning A Hybrid AI Approach to Content Generation

Abstract

Abstract This thesis tests the performance of LLMs on pre-generated L2 Swedish learning content across beginner to intermediate CEFR levels (A1–B2) by employing them into a self-developed language learning game (including language learning and self-validated language assessments). This study addresses a significant research gap in Swedish L2 speaking assessment and learning by providing two systematic evaluation components (with respectively designed frameworks): first, of LLM capabilities of CEFR-aligned Swedish content generation, and second, of a self-developed game-based Oral Swedish L2 learning system utilizing LLM-generated content. For LLM content generation evaluation, we assessed the learning content generated by three state-of-the-art models (GPT-4, Claude, LLaMA 3.3-70B-Instruct) across 10 topics and 4 CEFR levels, totaling 360 sets, with a multi-dimensional framework. The results revealed that while each model has distinct strengths—LLaMA excelling in learner experience and CEFR accuracy, Claude in technical integration, and ChatGPT in balanced performance—there are no considerate overall differences of performance. Machine-automated CEFR prediction shows that the exact accuracy rates are below 20% across all models with only 5% gap at the biggest, while adjacent accuracy ranges from 60% to 70%, indicating similar fundamental capabilities and limitations in CEFR-aligned content generation across all selected models. For system evaluation, the LLM-generated content was integrated into “LinGo Town”, a game-based Swedish L2 speaking learning system centering real-time speech assessment and adaptive difficulty features. Its evaluation framework incorporates pre-post assessments, in-game and post-game user questionnaires, and quantitative analysis were employed to evaluate system performance. Experimental results with 20 participants demonstrated measurable improvements in pronunciation-related skills, with small to medium effect sizes for speaking accuracy and fluency development. This research provides empirical evidence for the potential of adaptive L2 learning systems assisted with LLM-generated content, which is especially beneficial for lowresource language learning, such as L2 Swedish speaking, while identifying specific limitations and opportunities for improvement in CEFR-aligned content generation. Keywords: Swedish L2 learning, Large Language Models, CEFR-Aligned Content Generation, Game-based learning, Swedish Speech assessment, ICALL