Video localization differs substantially to any other form of localization, due to its multiple constraints and technical peculiarities. Originally reserved for media content, video localization workflows are quickly becoming part of all market verticals, as video has taken over the digital world as the prime form of communication. Automating such workflows requires strict adherence to the aforementioned constraints and technical elements, while the video localization market becomes a playing field for interesting applications and combinations of all language technologies plus more.

In this presentation we delve into the challenges of automatically localizing video, for live broadcasts or offline content across diverse genres. We explore the transformation of speech recognition output into semantically and syntactically formatted subtitles that adhere to reading speed and other parameters. We show how metadata integration can enhance the quality of machine translation output and solve issues such as length control and formality register, fundamental aspects of subtitling workflows. We create unique artificial voices that simulate natural ones using the technique of zero-shot speaker adaptation. We combine all technologies into an automatic speech-to-speech pipeline, to translate from audio in one language to audio in another. And we underline the pivotal role of a human-in-the-loop to refine the output at different stages of this cascaded pipeline.

Video localization has many facets. It can present itself in the form of the widely used, conventional subtitles in any given language or in accessible options, such as captions for the deaf and hard-of-hearing, with the addition of a sign language interpreter, or as audio subtitles and audio description for the blind and visually impaired. It can also be a new voice in the target language on top of the source; or a whole new production by a cast of target language actors. In each of these scenarios, automation can be built in by harnessing natural language technologies tailored for specific aspects of the video localization process. In this presentation, we will explore the multifaceted world of video localization through innovative uses of language automation for all or part of the workflow.

Host organization: Globalization and Localization Association