The Rising: Escape the Dark - A Sky Live Project

Welcome to the immersive, fully accessible world of our R&D project with Sky. "The Rising: Escape the Dark" pushes the boundaries of technology, storytelling, and accessibility, even incorporating elements of virtual production and "AI" techniques.

Created through a collaborative process between Reflex Arc and Sky, with funding and academic research support from XR Stories at The University of York, the "Escape the Dark" project looks at ways of redefining how we experience entertainment through Sky live.

Encompassing innovation, accessibility and inclusion, we've looked at ways of immersing the audience in Sky’s television series, “The Rising”; a supernatural murder mystery.

What is Sky Live?

Sky Live is a small camera and self contained device that sits atop the Sky Glass television. It offers new dimensions of engagement, connectivity, and empowerment across various realms, including wellbeing, exercise, chat, watch-along experiences, and gaming.

What is "Escape the Dark"?

Escape the Dark is a demo we've designed to offer a captivating glimpse into the immersive potential of Sky Live. Picture diving into the universe of a TV episode you've recently seen, experiencing the drama in a fresh, interactive manner.

1. Accessible First. Enjoyable to All.

At the heart of "Escape the Dark" lies an ambition to create an experience that transcends traditional limitations. Initially conceived to meet the accessibility needs of blind or visually impaired people and hard of hearing, our project's scope expanded as we ventured deeper. Our goal, as ever, is to ensure an enjoyable experience for all, regardless of their abilities.

2. Meeting the Needs of Visually Impaired people.

We created 3D game/story areas built around exploration, discovery, and navigation. The experience blends 3D soundscapes and spatial sounds pinned to objects within the environment, creating beacons of navigation for people who are blind or visually impaired. This approach empowers participants to explore the space, uncover objects, and progress through the experience.

With the invaluable insights of Matt Brown, a registered blind sound designer with a gaming background, we've meticulously refined and tested the experience to cater to visually impaired participants.

3. Meeting the Needs of the Hard of Hearing

Our immersive creation hinges on participants identifying sound sources that divert the protagonist, Neve. For an all-inclusive encounter, we've introduced dialogue subtitles and a bespoke directional cues system, visualising the important spatial audio elements and taking another step to ensure the experience's accessibility and enjoyment for all.

4. Sense the Drama: Immersion Beyond Boundaries

While accessibility remains paramount, we also harnessed Sky Live's ability to capture and interpret body movements. Drawing on our many year's of experience of working with depth sensors and motion tracking devices, it’s a multisensory odyssey, blending skeletal body tracking, 3D audio, and immersive visuals to bridge the gap between viewer and narrative.

5. Keeping it (Un)real: Bridging Reality and Fantasy

"Escape the Dark" effortlessly integrates with its source series, maintaining a seamless connection to the narrative's essence. Achieved by:

• Aligning the experience's storyline with the episode's narrative.
• Leveraging virtual production assets (utilising another games engine: Unreal) with an assets pipeline including 3D photogrammetry provided by Sky's Virtual Production team.
• Incorporating the episode's original sound effects, score, and soundtrack.

This grounding approach allows us to create a standalone experience while resonating with the series' core.

It also scratched a creative itch we'd had since working within Virtual Art Departments on a couple of large productions, including Ant-Man & The Wasp Quantumania. That is, to make truly interactive use of assets that are generated through the Virtual Production process.

(Above) An example of the photogrammetry assets we utilised, direct from the virtual production process used to make the series.

(Above) The Rising was made using Virtual Production techniques

6. Ethical use of "AI" techniques.

The Sky Live camera tracks body movement, recognising skeletal joints and gestures using a machine learning process. We've also used some generative AI techniques for the voice of the narrator.

7. Immerse Yourself in The Rising

In the footage below we offer glimpses based around Episode 4 of The Rising. A supernatural thriller, it follows Neve, a murder victim returned from the beyond to solve her own demise. As she delves into her community's secrets, a riveting tale unfolds.

Our experience encapsulates pivotal scenes from Episode 4, as Neve endeavors to locate the source of haunting scratching noises, ultimately leading her to Room Number 7 in Keaton Hall. In our interactive concept, players aid Neve in pinpointing the noise's origin by eliminating distractions. The demo spotlights the removal of a distracting crow from the scene, as the audio and visuals of the crow guide you to the next play area.

8. Ready to Dive In?

With the backdrop set, it's time to plunge into the mystery and "Sense the Drama." Step into the practice room, where you'll transcend darkness and embark on a voyage of discovery within Keaton Hall.

Special thanks go out to XR Stories, especially Melodie Ash for supporting us throughout this project, and the Sky team for the support with the technology, giving access to their library of virtual production assets, scripts, audio and other elements used in the making of the series.

Credits

• Richard England: Creative Technology Director
• Richard Hilson: Producer
• Joe P: Unity Developer
• Matt Brown: Audio Engineer
• George Garton: 3D and Lighting Artist
• Ilona Wheldale England: Project Management / Accessibility Coordinator
• Lauren Ward: Accessibility Research Fellow (XR Stories)
• Melodie Ash: Creative Producer (XR Stories)