Google runs a lip-sync challenge, an AI experiment to teach reading lips

Author at TechGenyz Google
Google AI Lip Sync
Credit: techxplore

Google has launched a new lip-sync challenge, run by the AI Experiment group of Google, which is the same group working on Google’s Pixel device. The aim of this particular lip-sync challenge is to teach Google’s AI system the art of reading lips.

Google plans to use real singers to make its AI learn the skill of synchronization. This initiative is seen as a part of an effort by Google to come up with applications for people who have disabilities, particularly speaking (ALS, for example).

Regarding the process of taking the challenge, it is designed in a very simplistic way. People interested in taking the challenge will have to simply click on the “Launch Experiment” button. This will redirect the users to the configuration screen which will configure Google’s system to work with the equipment of the users.

The process does not state the use of a microphone as mandatory. A video camera is all that is needed. The user will be shown in a way very similar to that of video chats in smartphones. After clicking on the “I’m Ready” button which will launch the lip-sync tune, “Dance Monkey” by Tones and I. As of now, there is no other option for the lip-sync tune available.

Then the user will have to either sing the song out loudly or silently. Once the user is done singing, another screen will open where the user will be provided with a score on a scale of one to five, in the form of stars. Users using the site can also get the web application code along with the procedure for setting up the application and using it with other web apps.

Google declares on the initial challenge page that the lip-sync challenge is built on Google’s TensorFlow.js AI technology. The company further adds that this challenge tracks facial landmarks as the user lip syncs. It is also said that the application is designed in a way to ensure that it works best with Google Chrome.

Google, as of now, has decided not to disclose any extra detail about the expansion or up-gradation of the application. The company has also refrained from stating anything about the availability of the application.

The lip-sync challenge has been launched by Google for people who care to take part. Users willing to testify their individual lip synchronizing ability are advised to visit the site set up by Google.

Career

Subscribe