A system for creating virtual reality content from make-believe games

Adela Barbulescu (1) , Maxime Garcia (1) , Antoine Begault (1) , Laurence Boissieux (2) , Marie-Paule Cani (1) , Maxime Portaz (3, 4, 5) , Alexis Viand (1) , Romain Dulery (1) , Pierre Heinish (1) , Rémi Ronfard (1) , Dominique Vaufreydaz (3, 4, 5)

1 IMAGINE - Intuitive Modeling and Animation for Interactive Graphics & Narrative Environments
2 SED [Grenoble] - Service Expérimentation et Développement
3 LIG - Laboratoire d'Informatique de Grenoble
4 UGA [2016-2019] - Université Grenoble Alpes [2016-2019]
5 PERVASIVE - Interaction située avec les objets et environnements intelligents

Pretend play is a storytelling technique, naturally used from very young ages, which relies on object substitution to represent the characters of the imagined story. We propose a system which assists the storyteller by generating a virtualized story from a recorded dialogue performed with 3D printed figurines. We capture the gestures and facial expressions of the storyteller using Kinect cameras and IMU sensors and transfer them to their virtual counterparts in the story-world. As a proof-of-concept, we demonstrate our system with an improvised story involving a prince and a witch, which was successfully recorded and transferred into 3D animation.

Mots clés: IMU sensors; 3D printed figurines; 3D animation; Speech; facial expressions; Kinect cameras; make-believe games; storyteller; virtual reality content creation; virtualized story; Cameras; Head; Magnetic heads; computer animation; computer games; virtual reality; Sensors

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]; Synthèse d'image et réalité virtuelle [cs.GR]; Traitement du signal et de l'image [eess.SP]

Adela Barbulescu, Maxime Garcia, Antoine Begault, Laurence Boissieux, Marie-Paule Cani, et al.. A system for creating virtual reality content from make-believe games. IEEE Virtual Reality 2017, Mar 2017, Los Angeles, United States. pp.207-208, ⟨10.1109/VR.2017.7892249⟩. ⟨hal-01578326⟩