January 2017

A Powerful Combination: 360 and VR Deliver an Enhanced Gaming Experience

Our industry doesn't stand still, and at dock10 we are continually pioneering techniques that enable new technologies to be used in innovative and exciting ways. Working extensively with CBeebies, they expressed an interest in exploring how they could turn a TV show into a 360 experience, so we undertook a development project to show how it might work. We set out to combine real studio filming with VR and 360 innovations, creating an immersive and interactive hide-and-seek gaming experience based on the popular CBeebies and Sesame Workshop show The Furchester Hotel.

The first step was to create the set of the Furchester Hotel as a 360-degree environment. The show is filmed at dock10 with an amazing almost 360-degree set which made it perfect for the project. We had already developed a range of special 360 filming techniques that deliver complete and constant 360-degree coverage while ensuring that cameras and crew are always out of shot. We then enhanced the 360 footage of the RAW physical set by adding in the missing floors, ceilings and walls through VFX post compositing in Nuke and CaraVR. The game format required iconic characters, such as Elmo and Cookie Monster, to pop up from hiding places within the hotel. Some of these characters were captured in the 360 shoot, with others being shot against a green-screen and added as a final layer in post.

Once the 360 video was finished we added two audio tracks. The first was a stereo mix of the music tracks and narrations, the second was a binaural mix which attached the audio of individual characters to their position in 3D space. This means that when listening through headphones, each character's audio appears to come from their position relative to where the player is looking. This really helps with finding the characters when you're up against the clock!

Finally, we worked closely with a specialist games development company that took all the elements and built the application in Unity, transforming a set of filmed sequences and audio into a fully-functioning VR game. This was playable across multiple platforms, either by moving a mobile device or using a VR headset so that the player 'looks around' the environment to find the Sesame Street characters as they pop out from their hiding places. A target on the screen is used to detect the characters and the game automatically registers each as being found, adding them to your score and recording the time taken to find all six.

The game is great fun to play and shows how we can combine a large number of technologies and techniques, both established and emerging, to create innovative new experiences for audiences. We plan to apply the learnings from this successful development project to a number of ventures on behalf of our clients.

Share article