The ACM CHI Conference on Human Factors in Computing Systems (CHI 2025), a flagship event in human-computer interaction (HCI) and an A*-rated conference in the CORE Conference Ranking, was held in Yokohama, Japan, from April 26 to May 1, 2025. Recognized as an A-class conference by the China Computer Federation (CCF), CHI attracts global researchers to showcase cutting-edge innovations. At this year’s conference, the research team led by Assistant Professor Li Quan from the School of Information Science and Technology (SIST) at ShanghaiTech University received the Best Paper Award for their work “ClueCart: Supporting Game Story Interpretation and Narrative Inference from Fragmented Clues.”
The research addresses critical challenges in interpreting game narratives, introducing ClueCart, an AI-driven creativity support tool designed to empower players and content creators with a novel framework for analyzing fragmented narrative clues. In popular video games like Elden Ring and Hollow Knight, developers employ indexical storytelling, weaving intricate worldviews through scattered environmental hints, item descriptions, and fragmented dialogues. This non-linear, highly fragmented narrative structure poses significant difficulties for players and creators striving to organize and interpret cohesive stories. ClueCart’s innovation lies in its clue taxonomy, capture functions, and creativity support, offering a systematic approach to narrative understanding and reconstruction.
Figure 1: Hierarchical taxonomy (illustrated with Hollow Knight). The taxonomy comprises two levels. The first level, Classification, categorizes clues into three groups: Character, Location, and Achievement. The second level, Elements, further classifies clues based on in-game attributes into six types: Cut-scene, Related Character, Environment, Artifact, Text, and Dialogue. Each Classification category can encompass any type of Element, providing flexibility for diverse interpretations.
ClueCart’s core functionalities are built around three technical pillars:
1. Hierarchical narrative taxonomy: Derived from player behavior data, this taxonomy organizes clues into a three-tier structure (Character, Location, Achievement) and six element types, enabling standardized clue management from a player-centric perspective.
2. Custom game mod and AI integration: A custom game mod, built using the BepInEx framework, captures real-time in-game events (characters, locations, achievements). Paired with the GPT-4o model, it automates clue summarization, keyword extraction, and semantic relationship analysis, significantly reducing manual organization efforts.
3. Interactive analysis interface: ClueCart offers a highly interactive interface (see Figure 2) with a left panel for clue categorization and retrieval and a right panel for story interpretation. It supports drag-and-drop clue management, graph-based visualization, and a novel external knowledge retrieval feature that links in-game symbols to real-world contexts like mythology and history, fostering deeper narrative insights.
Figure 2: ClueCart interface. (A) The left panel facilitates clue categorization and retrieval. (B) The right panel supports story interpretation through interactive tools.
To evaluate ClueCart’s effectiveness, the research team conducted a between-subjects study with 40 professional content creators, comparing it against Miro, a widely used collaborative tool. Results demonstrated ClueCart’s superior performance in task completion efficiency, narrative coherence, and cross-clue association. Assessments using the system usability scale (SUS) and creativity support index (CSI) further confirmed its advantages in reducing cognitive load and enhancing creative inspiration. As an open-source project (available at https://cluecart.github.io/ClueCart/), ClueCart has garnered significant attention from the game development community. Its modular design extends beyond games like Black Myth: Wukong, which are rich in cultural symbolism, to potential applications in cross-media narrative analysis, such as novels and films.
The research team emphasized ClueCart’s transformative potential: “ClueCart shifts players from passive narrative recipients to active participants. Looking ahead, we aim to explore collaborative creation platforms, empowering global players to co-craft untold chapters in game worlds.”
In this year’s CHI, ClueCart’s award underscores the innovative strength of Chinese research in human computer interaction and signals the growing impact of AI-driven creativity tools in reshaping content creation. As the Program Committee noted: “This work pioneers a new path at the intersection of player behavior modeling, multimodal AI, and narrative science. It enables machines to begin understanding how humans ‘tell stories’—a domain once exclusive to artistic creation.”
This study is a collaboration between ShanghaiTech University, the Hong Kong University of Science and Technology (HKUST), and HKUST (Guangzhou). Third-year master’s student Wang Xiyuan at ShanghaiTech is the first author, with Prof. Li Quan as the corresponding author.