OpenAl, a non-profit research company (San Francisco), with well-known Elon Musk among its co-founders, has released GPT-3 neural network (Generative Pre-trained Transformer 3). According to the developers, their algorithm is able to work in Excel or in a graphic editor, prepare summaries, and even write poems. The company representatives say that the development of the neural network has been just started.
GPT-3 is a step forward from its predecessors GPT and GPT-2, released in 2018 and 2019 respectively. GPT-3 has 175 billion machine learning parameters (compared with GPT-2’s 1.5 billion); the algorithm was tested on a huge body of data, namely, 570 GB of the text, including free Common Crawl base, Wikipedia, and full texts of books and newspapers. This is the evidence that the new language model is able, based on a small number of examples from the relevant text, to write a report, to compose a poem, to discuss professional topics with a user of the landing page, and even to write training codes for already existing algorithms.
To date, a closed beta testing version of GPT-3 has been released. This was the decision made by the developers in order to avoid any software abuse by unscrupulous users.
“We have limited access for those wishing to test the algorithm, in order to prevent any unlawful actions, including cyberharassment, cyberbullying, dissemination of spam, fake news, and other things,” a representative of OpenAl reported.
Certainly, the news about the release of the world’s most powerful language model prompted an outburst of comments from Internet users; however, reactions have not been just mixed, but polarized. On the one end are the commentators predicting that many professions will soon be edged out by GPT-3; on the other, underwhelmed experimenters pointing out all that the algorithm cannot do — or cannot do well.
According to experts, GPT-3 neural network is not able to analyze the data provided; it is only able to properly generate a text by offering the most probable words. It is, therefore, too early to speak about a significant impact that GPT-3 might make on the translation industry. Having many neural network layers is not a solution to the core machine-translation challenges the language industry is currently facing, although it will be interesting to see GPT-3 at work.
Source: Slator – language industry intelligence