Sequence-based analysis of learner interactions in generative AI-supported interactive video programming tasks: insights from novice and experienced learners

This study examined learner interaction trajectories within generative AI-supported interactive video programming environments across two instructional contexts involving learners with differing levels of programming experience. Participants were categorized as novice or experienced based on prior programming experience. Learners in both contexts interacted with programming tasks embedded in the video, where they were required to write and execute code. When solutions were incorrect, learners revised and re-executed their code or used an “ask AI” option to receive hint-based feedback from a generative AI system. Interaction log data were analyzed using sequence analysis and transition network analysis to identify dominant behavioral patterns and action-to-action transitions. Interaction behaviors were further examined in relation to response correctness and self-reported cognitive load, and mixed-effects models were used to examine how cognitive load, action count, and correctness varied across interaction points. The findings indicate that learners across both instructional contexts initially followed similar interaction requirements imposed by the system. Over time, however, distinct interaction trajectories emerged within the two instructional contexts. In the novice instructional context, interaction patterns were characterized by more frequent AI consultation and shorter interaction sequences, whereas the experienced instructional context showed more diversified exploratory behaviors, including more frequent consultation of external resources and longer interaction trajectories. These context-specific patterns were also evident in interaction patterns associated with response correctness and cognitive load, as well as in the task-level trajectories of these indicators. Overall, the findings suggest that behavioral tendencies within the two learner contexts were less evident in the frequency of isolated actions than in the sequencing and organization of interactions over time. These results underscore the importance of designing generative AI-supported interactive video environments that adapt support mechanisms to learner context, task-specific demands, and evolving cognitive-behavioral indicators, thereby supporting the effective use of AI as a regulatory scaffold.