Zhejiang University Unveils Immersive Role-Playing Framework to Make AI Interactions Natural

AI role-playing is moving beyond simple text-based conversations into rich, immersive dramas. Researchers from Zhejiang University and Tencent Youtu Lab recently introduced AdaMARP, a self-adaptive multi-agent interaction framework that tackles key weaknesses in current large language model role-playing, such as weak environmental awareness and rigid storytelling. The framework gives AI the scene management and narrative skills of a director. This work has been accepted at the international conference ACL 2026.

Core Pain Points: Missing "Environment" and "Director"

In today's AI role-playing, users can talk with historical or literary characters, but interactions are often limited to text exchanges inside static settings. For example, in a detective scenario, traditional AI systems behave like repetitive talking machines, unable to reason from environmental clues like wax stains on the carpet or handle complex narrative needs such as multiple character turns and scene transitions. This hollow mode prevents users from feeling real authenticity or narrative tension.

AdaMARP Framework: Four-Channel Messages and Dynamic Scheduling

To break this deadlock, the research team designed a new interaction logic. First, AdaMARP introduces a "four-channel message format," breaking each round into "Thought - Action - Environment - Speech." Instead of only outputting dialogue, the AI now interweaves environmental atmosphere (like flickering gas lamps), internal thoughts, and body language into a complete causal chain.

Second, the framework includes a "Scene Manager" that acts as the narrative's director. It has five core capabilities: initializing the scene, selecting the speaker, switching scenes, dynamically introducing new characters, and ending the interaction. This lets the AI autonomously decide when to move from the crime scene to a witness's home, or when to have a new suspect walk through the door.

Training and Evaluation: From Literature to Simulation

To equip AI with genuine acting and directing abilities, the team built high-quality datasets AdaRPSet and AdaSMSet. These datasets contain deep character profiles and interaction trajectories from 81 classic literary works, plus 20 different thematic synthetic plots. This ensures the model learns literary texture while mastering dynamic scheduling logic.

In addition, the team introduced a complementary evaluation framework called AdaptiveBench. Unlike traditional single-turn conversation evaluations, AdaptiveBench scores models at the trajectory level, focusing on character consistency, environmental perception, and the naturalness of narrative progression. This provides a comprehensive assessment of AI performance in complex long-text interactions.

This framework opens a new technical path for immersive interactive scenarios like detective reasoning and adventure storytelling. By deeply coupling environment and narrative logic, AI is evolving from a simple chat assistant into a digital performer with advanced creative consciousness.