GPT-3: Hype or Hyper-useful to the Globalization and Localization Industry?

 

Black and white robot toy sitting on a red bench reading a book

 

In June of 2020, the US research lab OpenAI, which is backed up by Elon Musk, among others, launched its breakthrough language model: GPT-3—Generative Pre-trained Transformer 3.

Why was it such a big event? Because GPT-3 may just change the way we work: we’re talking about a tool that can summarize mails and news articles, generate tweets and create chatbots, and yes…translate language.

GPT-3 versus Other Language-prediction Programs

As a language-prediction program created with neural networks, GPT-3 is by far the most powerful language model ever created. It is an artificial-intelligence system trained on a gigantic corpus of text: thousands of digital books, the entire Wikipedia, and roughly a trillion words from blogs, social media, and the internet at large. Given enough text—and enough processing—the machine learns probabilistic connections between words. In a nutshell: GPT-3 can read and write. 

In fact, GPT-3 can process text far better than its predecessor, GPT-2 (2019). GPT-3 is also distinguished by its sheer size: it boasts 100+ times more parameters than its previous version. GPT-3 uses 175 billion parameters, compared with the 17 billion of its closest competitor, Microsoft’s Turing NLG—making it the largest neural-network ever built. 

GPT-3: The Power and the Glory

Get this: GPT-3 can correct English. It can write poetry. It can write like Faulkner, Proust, or Goethe. Any author you “feed” it—really. Even more relevant to our industry, GPT-3 can also:

  • translate legal text into plain language—and generate legal text from simple “instructions”;
  • write the code to create a website, based on your directions;
  • generate well-formatted charts and infographics—from your summary; and
  • fill in the missing parts of a spreadsheet, by computing formula on its own.    
What We Love about GPT-3

First, using easily understood natural language, GPT-3 can answer questions on any topic, while retaining the context of previously-asked questions. GPT-3 can also translate from spoken languages and programming languages—Python, JavaScript, and CSS (very useful for websites!), for example. GPT-3 can also generate questions and answers for UX, due diligence document search, and report generation.

GPT-3: Dealbreakers

At first, GPT-3’s generated text can be impressive. But longer texts tend to dissolve into meaninglessness.

Another problem: like many chatbots, for example, the program amplifies biases inherent in the data—including racism and sexism.  

Then there’s the ginormous amount of processing power that GPT-3 sucks up: the program is miles more “compute-hungry” than even bitcoin.

Finally, there’s cost: conservative estimates place a GPT-3 training run at a staggering $4.6 million—well beyond the reach of most companies.

GPT-3 and You

Right now, GPT-3 isn’t ready to replace the leading conversational interfaces on the market—Amazon’s Echo and Apple’s Siri, for example—but it can be used as a powerful tool to improve them. For now, GPT-3 is a crucial steppingstone to building language models of the future. 

So, How Can GPT-3 Be Used Effectively?

It’s unclear. After all, if GPT-3 is eventually shown to generate the right text only half of the time, will it satisfy professionals? Be of use as more than a steppingstone? Let’s see!