Exploring immersive storytelling for avatars in Realtime gaming: human data capture for AI-driven non-playable character engagement in games

Project partners: University for the Creative Arts (UCA), SKC Games Studio

A collaboration between the University for the Creative Arts and SKC Games studio received R&D funding from XR Network+ to explore immersive content creation for video games in collaboration with SKC Games Studio. 

Virtual characters in interactive storytelling games interact within a complex environment using different modes of communication, such as speech and gestures. Whilst these virtual characters can be highly realistic, sound and image qualities are often challenged by cross effects and the limitations of real time production requirements. 

The collaboration used state-of-the-art AI to develop a system that produces real-time performance for both in-game player and AI-driven NPC (non-player character) avatars. 

The outputs from the project were used to create real-time virtual characters for a chapter of SKC’s interactive role-playing video game, Project UN – Taradene.

The collaboration brought together SKC games programmers and UCA researchers and students to build links between academia and industry and develop a knowledge-transfer model. UCA students developed performance capture and games production skills in a real-world setting and SKC programmers gained access to world-leading research on AI algorithms.

Exploring immersive storytelling for avatars in Realtime gaming is one of five projects supported by the XR Network+ Embedded R&D funding call. Grants of up to £60,000 were awarded to researchers at UK universities to explore the transfer of knowledge between academia and industry in areas aligned with Virtual Production. The projects took place between September 2023 and March 2024. 

Categories: Games, Research, Technology