The Rising: Escape the Dark

Accessibility-first interactive drama for Sky Live

Client: Sky

Programme: XR Stories (University of York)

The Rising: Escape the Dark is an R&D project exploring how immersive interaction could sit naturally in a living-room, TV-first context. Designed to place viewers inside the narrative world of The Rising, the experience allows participants to support the story through movement, exploration, and sound. Accessibility was treated as a creative driver rather than a constraint — resulting in clearer navigation, stronger narrative cues, and deeper immersion for all audiences.

What is Sky Live?

Sky Live is a compact camera and sensor device that sits on top of the Sky Glass television. It enables new forms of interaction and participation through body tracking and gesture recognition, opening up possibilities across entertainment, wellbeing, fitness, social viewing experiences, and gaming.

The concept

Set around Episode 4 of the series, the demo follows Neve — a young woman returned from the dead to solve her own murder. Participants help her locate the source of unsettling sounds within Keaton Hall. By identifying and eliminating distractions, such as a crow whose audio and visual cues guide navigation, players directly support the unfolding story, building tension, atmosphere, and character.

Accessible first. Designed for everyone

From the outset, Escape the Dark was designed as an accessible-first experience. While initially focused on the needs of blind and visually impaired audiences and deaf and hard-of-hearing participants, we quickly found that inclusive design principles strengthened the experience for everyone — resulting in clearer navigation, stronger narrative cues, and deeper immersion.

For blind and visually impaired audiences

We created 3D environments built around exploration, discovery, and independent navigation, using sound as the primary interface. Spatialised 3D audio anchored to objects and points of interest, audio "beacons," and layered soundscapes communicate space, distance, and progression — enabling blind and visually impaired participants to confidently explore the environment, locate interactive elements, and advance the narrative.

The experience was developed and refined with expert input from Matt Brown, a registered blind sound designer with a background in games, ensuring the design was both practical and enjoyable.

For deaf and hard-of-hearing audiences

To ensure full accessibility for deaf and hard-of-hearing audiences, we introduced complementary visual systems — including clear dialogue subtitles and a bespoke directional cue system that visualises key spatial audio sources. These visual cues mirror the audio-driven gameplay mechanics, ensuring all participants receive the same narrative information, agency, and sense of immersion.

Sense the drama: embodied interaction

Beyond accessibility, Escape the Dark showcased our expertise in embodied interaction and motion-based design. Using Sky Live's camera and sensors, the experience captures and interprets real-time body movement. Drawing on years of experience working with depth sensors and motion-tracking systems, we combined skeletal body tracking and gesture-based interaction with spatial audio and real-time visuals to create a multisensory experience that physically connects participants to the unfolding drama.

Bridging broadcast and virtual production

Maintaining authenticity to The Rising was central to the project's success. To ensure a seamless connection between the interactive experience and the broadcast series, we aligned the interactive directly with the episode's storyline, integrating virtual production assets created in Unreal Engine and building an asset pipeline incorporating 3D photogrammetry supplied by Sky's Virtual Production team. We also used the original sound effects, score, and soundtrack from the series.

This approach allowed us to make interactive use of assets created for virtual production — an area informed by prior work within Virtual Art Departments on major productions, including Ant-Man & The Wasp: Quantumania.

Example photogrammetry assets utilised, direct from the virtual production process used to make the series.

The Rising was made using virtual production techniques (ARRI Stage, London).

Ethical use of AI techniques

The Sky Live camera uses machine-learning techniques to recognise skeletal joints and gestures. We also explored generative AI for the narrator's voice — with Matt Brown, our sound designer, choosing to use AI to replicate his own voice for the prototype. This approach kept creative control with the performer while allowing rapid iteration during development.

Outcomes and impact

The prototype was extensively tested internally at Sky and with visually impaired participants, receiving consistently strong and positive feedback. The project demonstrates the creative and commercial potential of Sky Live as a platform for interactive drama, while showing the value of accessible-first design in enhancing immersion for all audiences.

It also demonstrates our approach to applying research insights to practical, broadcast-quality outcomes — helping to connect experimentation with production-ready work. The project reveals new opportunities for repurposing virtual production assets as interactive experiences, offering a useful starting point for exploring future interactive television formats and more accessible immersive content.

See it in action

This video documents the development process and demonstrates the prototype in use.