Current MT Cannot Meet Quality Needs
By: Stephen Doherty (Centre for Next Generation Localisation)
06 March 2013
The Goal of Quality Translation: QTLaunchPad is an EC-funded collaborative research initiative dedicated to overcoming quality barriers in machine and human translation and language technologies - this article presents its aims and initial findings.
Translations of publishable quality are in greater demand today than ever before. As Paula Shannon from Lionbridge puts it, we have reached the point where translation is “beyond human scale”, and only a miniscule amount of content that should be translated actually is. The lack of fast and affordable high-quality solutions hurts business and communications everywhere, and the volume of content requiring translation only continues to grow.
Despite progress with MT, the need for high-quality translation has not yet been satisfied. Josep Bonet from the DGT recently noted that, in the past, “people were happy to accept an inferior-quality translation rather than no translation at all.” The bar for keeping consumers happy is getting higher: gisting isn’t good enough anymore.
New approaches incorporating MT, especially in post-editing scenarios, have made quantifiable savings for buyers and providers. However, especially in post-editing scenarios, much of translators’ time and efforts are still taken up by repetitive tasks, an expense that leads to a mismanagement of resources at a high cost to all industry stakeholders.
While the quality of gisting is, by and large, getting better, its use for publishable content or even for post-editing leaves a lot to be desired. While most MT research focuses on gradual improvements in statistical and hybrid systems, a long-range perspective demands a more systematic approach to take on quality issues, yet little funding resources are directed toward high-quality translation technology. Our approach goes beyond improving gist translation to raise the standard for the content requiring human post-editing, to push content currently being post-edited up to a publishable level, and to recognize true high quality by implementing quality estimation.
Figure 1: Barriers in Translation Quality
For most small and medium-sized businesses, adoption of MT represents a huge leap and long-term investment in resources and expertise before ROI is apparent. Larger buyers and providers with more resources still struggle with quality issues and delivering quantifiable results to their customers in a meaningful way.
For smaller LSPs and freelance translators, the availability and requirements of an increasingly technology-centric landscape can also provide lucrative gains in productivity. While adoption of technologies is increasing, the use of MT for effectively dealing with repetitive content and errors remains suboptimal. Improving the quality of translation technologies and their ability to detect where human intervention is needed will enable human translators to focus on the interesting, high-value aspects of their jobs, thus making their contributions more valuable and rewarding.
The QTLaunchPad Project
QTLaunchPad focuses on the barriers to high-quality translation in a range of applied scenarios. Of particular relevance to this focus are evaluation metrics, quality estimation, and optimization of language resources. In this way, we’re preparing the groundwork for a large-scale translation quality initiative for Europe (and the world).
The consortium consists of four world-leading research centers: the German Research Center for Artificial Intelligence, Dublin City University, University of Sheffield, and Athena Institute for Language and Speech Processing. Strong links with many key players in the translation and localization industries and communities around the world provide an excellent foundation for a focused and intelligent push to overcome the hurdles along the path towards high-quality translation and language technologies.
Just after the half-year mark, the QTLaunchPad project is about to begin a series of workshops and share its initial findings.
Multidimensional Quality Metrics
QTLaunchpad has conducted a thorough examination of existing evaluation and QA methods for both human and machine translation to identify deficits and best practices. The findings of this have informed the construction of a unified and easy-to-use platform called Mulltidimensional Quality Metrics (MQM).
Its first drafts were launched publicly and at no charge in January 2013; it provides a platform for building project type-specific metrics that evaluate translation quality using “dimensions” based on ISO/TS-11669 specifications. Users can specify important information for each job and its expectations and customize the importance of issues based on their own values. The metrics view quality as threefold: the fluency for both the source and target, the accuracy of the translation (how well is the meaning preserved?), and end-user adequacy (how well do the source and target texts fulfil their intended purpose?). This approach allows users to identify the cause of issues and solve them where appropriate rather than perpetuating the tendency for translators to be blamed for all problems. This approach would, for instance, allow translators to be recognized for fixing problems in the source rather than being penalized for them.
The metrics are also complementary to models such as the TAUS Dynamic Quality Framework and can precisely replicate the functionality of existing models such as the LISA QA Model or SAE J2450. This caters for legacy users and proprietary methods to maintain compatibility while still benefiting from these new features.
In close contact with translation and localization buyers and suppliers, the Multidimensional Quality Metrics are being further developed throughout 2013 so that they truly meet user needs. In consultation with industry bodies and leading companies, this approach will also provide a number of profiles that represent best practice for specific translation types (e.g. medical and technical) to enable users to ensure full compatibility across company and project boundaries.
As a means to showcase our progress and to share the project’s future directions, QTLaunchPad is running a series of workshops on the MQM framework and its implementation into applied industry scenarios. The first of these workshops will take place in Rome, 14 March, co-located with the MultilingualWeb workshop, and on 19-20 March at the GALA 2013 conference in Miami.
We invite all interested parties to learn more, evaluate MQM and other QTLaunchPad activities, and provide constructive feedback through the following channels:
- As QTLaunchPad develops, results and information are being made available on the project homepage.
- Follow the discussions and highlights of our workshops on our LinkedIn group.
- Connect with us on Twitter @qtlaunchpad and Facebook.
- To be added to our mailing list and for inquiries, please contact: [email protected]
Stephen Doherty, BA, HDip, PhD, MBPsS, is a post-doctoral researcher in the Centre for Next Generation Localisation in Dublin City University. He conducts research and lectures on topics of language and cognition, human-computer interaction, translation and language technologies. He is currently working on QTLaunchPad, a collaborative European research initiative dedicated to overcoming barriers in translation and language technologies, and teaches translation technology with a focus on machine translation.