The SOL Surdo-Online service, a platform which interprets Russian sign language into speech in video format, allows the deaf and hard of hearing to communicate with the non-deaf in airports, pharmacies or anywhere else where an exchange of information is required.
Now, the company is working to introduce artificial intelligence (AI) into the equation in order to improve the system further.
"Using a platform where translators work online, it's possible to improve the efficiency at least fivefold. This is how the idea of project SOL Surdo-Online was born, and we launched it in September 2016. Now, we have decided to improve the system through the introduction of artificial intelligence," Melnik said.
At the moment, the service operates according to the following scheme: companies are given access to an online cloud service which allows users to contact sign language interpreters. These interpreters then relay the information over to cashiers, pharmacists, or whomever else the deaf person may be speaking to, and interpret speech back to the user. The service uses an ordinary Google Chrome browser, enabling it to be connected using an ordinary phone, tablet or computer.
The system is being trained to recognize gestures with the help of a neural network. At the moment, developers are assembling a database of gestures using video conferences between sign language users. It is estimated that the AI requires at least 1,000 demonstrations of gestures by different people to accurately figure out the patterns involved and their variations.
The next step will be to synthesize these gestures to account for Russian sign language's 10,000+ gestures. The main difficulty, according to developers, will be the digitization of this '3D language' to enable it to become fully automated.
There, the project was evaluated by foreign experts. Peter Childs, director of the Innovation Design Engineering program in the UK, said that SOL Surdo-Online was a realistic project which already has a functional prototype of its platform. The project now requires investment and advice, but has great potential even at the international level, Childs said.