Don't Despair: It's Only QA!
By: Katell Jentreau (Box and Women in Localization)
05 December 2013
If you ask anyone in a tech company if Quality Assurance (QA) is important, you'll get a lot of nods: “Yep, sure, of course, absolutely, super important.” Of course we need to QA what we produce, whether it's localization or anything else. Developers submit their code for review, then to QA for release. Translators submit their first draft to review (well, you hope), and then check again before they deliver their piece.
But what does QA mean, exactly? Who performs this QA, humans or machines? If you look up QA in Wikipedia, you'll learn that it's all about procedures put in place to make sure products and services meet their goals. It also states that the two principles of QA are 1) 'Fit for purpose' (is the target audience happy with the product?) and 2) 'Right first time' (i.e., we don't want customers to return the product or complain too much about it).
So what does QA mean in practice? And when push comes to shove - when deadlines are looming, and competitors threaten to release before you do, when that precious new market needs to be reached as soon as possible and everyone is so agile you don't even take time to think - then does QA really happen? And if so, in what form and according to what rules?
Localization is to Product Development what Quality Assurance is to Localization
Marina Malkevich, Director of Globalization Quality Engineering at Yahoo! defines QA as a "second rate citizen" in the globalization world. Everybody agrees it's important, but do we give QA all the tools and assets it needs to succeed?
"It is a semi-automated process. Tons of manual work for current releases, automation for stable UI. Automation with tools like Selenium, UIAutomation, Robotium works," explains Malkevich, "but only if the strings don't get modified at the last minute before release… which is not the reality in our fast-paced, ever-changing world".
There are also more and more players on the market. Malkevich explains there are "loads of start-ups with tons of new apps and no QA happens there at all. And of course it's even harder with agile, as everything needs to be done quickly and there is no time for proper QA. You have to rely on developers."
Soham Mehta, Director of Engineering Services at Box, confirms: "Core testing used to be automated, but the state of tooling wasn't there, it was too flaky, so we had to go back to manual testing. We may revisit this." User interface testing at Box is currently done manually either by developers or offshore by vendors. "We would like to have more automation," adds Mehta, "but we have a strong developer accountability. They own the code and they are responsible for it." As for international QA, it still needs to be fine-tuned. "Once we grow, things will have to change to a model where you have one tester per team - and this would also include international testing."
Proportionality - The Evolving Standards of Localization and QA
If you work in globalization you may have felt that you won a great battle when your developers and senior managers finally recognized that the majority of the world doesn't speak English. But when QA is introduced, you hear: ”What do you mean we have to spend time, resources, and money on QA-ing all this foreign stuff? We've already spent time, resources and money on core testing!”
Even if you have a solid internationalization process in place, QA is important – especially if your internationalization process is not rock-solid. You would love to recruit a large team to constantly test your localized product, improve the product and provide your customers with a flawless simship (simultaneous shipping in all languages at once) experience, but with your limited resources all you have is two days at the end of the localization process with at best a promise that Engineering will fix some of the issues with the next release.
The good news is that if you have adopted agile processes, you can probably get your bug fixes through with the next release pretty quickly, but you will still need a process in place. And the other good news is that we all need to relax about it.
Let me explain...
While attending numerous conference sessions and talks recently, I met dozens of very interesting and knowledgeable people, and I noticed a common thread: Globalization is adapting, adapting to an ever-changing world.
I've heard about a lot of very interesting and creative initiatives to optimize Linguistic Review and QA with the best resources available. In a lot of instances, this involves using tools and technology to support the best model for the budget available: for VISA, this means designing and implementing an ingenious PDF-powered tool that provides vendors and internal reviewers with a reliable and trust-worthy exchange platform; for Wikipedia it's calling upon hordes of avid contributors to check and edit published articles seconds after they come online; for Google, it's asking users for feedback on the quality of the localized product by means of a survey; and for Microsoft, it's using its large user community to post-edit machine translation-produced help articles.
Fit for Purpose
Why are Fortune 500 companies using the crowd? Let's remember that we all have budgets and deadlines and we all need to do the best we can with the resources and the time available. Does that mean that we shouldn't try and be as thorough as possible? Of course not. But the operative word here is: "possible." Remember the Wiki-definition: it has to be 'fit for purpose.' Who is your target audience and who is your end-user? How much do they care and who should you prioritize for?
How do you test all Android devices on the planet when you release a mobile app in 30, 50, or more languages? "Use data," says Malkevich. "Look at your top two most used devices in a particular locale and go for those. Yahoo! uses a specialized testing vendor who will go and buy the devices you need." Will a lot of devices be left untested? You bet. But you've prioritized and catered to the bulk of your customer base.
Keep Calm and Carry on QA-ing
Just as translation is faced with a growing need for global content – and not all of this content can be translated by PhDs in literature or even “translation professionals – localization QA must work with the tools at its disposal to reach that elusive target: the Q in the QA. Who defines quality? It used to be professors only, then internal reviewers. But now the users have their say, and thanks to email, blogs and social media, they say it loudly. Listen to them and design your process around them.
QA is hard. And it's not shiny. Just like localization and globalization overall, you only talk about it when it fails. So don't try to do too much. Look at your resources, define a process that will work for you, work closely with your Engineering team and your project managers; make a plan, fight for time to do the best you can, and use all the resources at your disposal: professional testers, user communities (watch out for the number of bugs!), internal staff, and real end-users. They're the ones who ultimately care, after all.
Katell Jentreau started her career as an English to French translator, and has been working in localization for over fifteen years, both on the vendor and client sides. She was was part of the centralized localization team at Yahoo! a couple of years ago, and is currently responsible for all localization work at Box, one the leaders in enterprise cloud-based collaboration platforms, based in Los Altos, CA. In 2013, Katell also joined the board of the Women in Localization, an organization dedicated to promote professional development, networking and continuous education among its rapidly-growing membership.