At the Google I/O 2023 event held in the early hours of May 11th, Google showcased a variety of projects and devices they are currently working on. Along with the much-leaked Google Pixel 7a and Pixel Tablet, they also introduced their first foldable smartphone, the Pixel Fold.
As in previous years, Google not only introduced the hardware they will release but also various software and projects they are working on. One of the most interesting ones was how they will integrate AI into various aspects, such as the Universal Translator project.
Google Tests AI Dubbing Video Universal Translator
According to Google, the Universal Translator is an AI that will translate a video through real-time dubbing. This capability is made possible by their shift to PaLM 2, a language model that Google will use in their future products, including Google Bard.
Senior Vice President of Technology and Society at Google, James Manyika, stated that they are concerned about the potential for AI to produce and spread false information, particularly the notorious deepfake technology.
The Universal Translator will introduce features to prevent the spread of false information by embedding various “guardrails.” These may include adding watermarks or metadata to the video.
The Universal Translator allows users to hear a video in a language they do not understand and have it dubbed directly into their preferred language. The prototype was demonstrated on stage, with tests conducted by Arizona State University.
Universal Translator May be Coming to YouTube in the Future
This AI works in four stages. In the first stage, the model follows the speaker’s lip movements. In the second stage, the model activates an algorithm that provides instant speech output. In the third stage, the model pays attention to intonation to make translations more natural. Finally, in the fourth stage, the model combines all of the previous work into a dubbed translation.
Google itself has stated that this development looks promising in the early stages, although it has not yet been publicly released and is only being used by a small test group. However, after Google implements various guardrails, the Universal Translator may be introduced in other Google products, such as YouTube or Meet.
If successful, the application of this AI will improve opportunities for international collaboration by overcoming language barriers. However, we still have to wait for some time to see the results of Google’s work with this AI.
