Makaton signing game: real-time gesture recognition

This game improves the language skills of Makaton users (Makaton is a form of sign language). Using depth sensors, skeletal tracking and a machine-learning recogniser trained on Makaton signs, you play it purely by performing sign language gestures – recognised in real time, back in 2014–15.

With a focus on important life skills such as self care and relating to the world of work, the game can improve the quality of life of its users.

Using Microsoft Kinect (for Xbox One), participants interact with a cartoon superhero, who challenges them to perform signs properly to help him with daily tasks such as brushing his teeth.

The game won a TIGA award (Best Educational Game), a BAFTA Children’s nomination (Learning Secondary), the runner up in its category in the 2015 NHK Japan Prize and was also a finalist in the 2016 BETT awards.

This was developed in partnership with GamelabUK and Hassell Inclusion.

It was originally released as The Boris Games, a name chosen by the children we tested it with in 2014/2015.