Are Programmers in Danger of Being Automated?

When we think about jobs that could potentially be automated, what usually comes to mind are manual jobs being done in warehouses, factories, supermarkets and other places. However, thanks to new developments in AI, even technical jobs such as programming and QA could be done by computers. One of the biggest innovations that is making this possible is GPT-3, an enormous artificial neural network trained on huge amounts of text scraped from the web. Let’s take a closer look at the capabilities of GPT-3 to write code and the data annotation work that was required to create it.

What is GPT-3?

GPT-3 stands for Generative Pre-trained Transformer 3 and it’s the third version of the tool to be released. It was created by OpenAI, a research business co-founded by Elon Musk. It is able to generate text using algorithms that were trained on 570 GB of text. This training data was gathered by crawling the internet (a publicly available dataset known as CommonCrawl) along with other texts selected by OpenAI, including the text of Wikipedia. All of this training data would need to be annotated using sentiment annotation, semantic annotation and labeling methods including many other techniques

After all of the data has been annotated, the system is now able to understand human speech and put together sentences on its own. In fact it can do anything that has a language structure. This includes thighs like answering questions, writing essays, translating text and coding. Let’s explore GPT-3’s ability to write code to see its capabilities and current flaws. 

How is GPT-3 Able to Write Code? 

Modern-day smartphones and other gadgets offer autocomplete functionality, so when you are writing a text message or an email, it will offer to complete parts of the sentence for you. Well, GPT-3 is offering computer programmers the same convenience. Right now the scope is fairly limited, and can write code only in Microsoft Power Fx, a simple programming language derived from Microsoft Excel formulas that’s used mainly for database queries. However, this new technology is already showing a lot of promise in helping new programers by functioning as an autocomplete for code. 

Today we can already take advantage of this functionality to create low-code or even no-code tools allowing products to be used by a wider audience. Also, as GPT-3 becomes more advanced, it can start handling even more sophisticated tasks such as writing code from scratch in a wide variety of languages. 

What are the Limitations of GPT 3? 

As impressive as GPT-3 is, it still has a long way to go in terms of replacing human programmers. For starters, it has a very short context window of about a few hundred words. This means that while it may be able to write long texts and code, it will have trouble remembering what it wrote at the very beginning. This means that the beginning of the code may not be compatible with the end making the entire code unusable. Also, we need to make a distinction between natural language and formal language. Natural language is a language spoken by humans, such as English. It evolved over time and is full of ambiguity. Most of the time, meaning is only completed with contextual information.

Formal language is a programming language, such as Python. There are many ways to arrive at the same solution, but each of them has to strictly follow the syntax rules of the language. There’s no space for uncertainty. So right now, GPT-3 might be able to autocomplete a short portion of the code, but it still cannot learn the finite rules of the programming language. However, as time goes on, you can expect these problems to be fixed. 

Consider Working With Mindy Support to Annotate Your Training Data

While you may not be working on the next GPT-3, even small machine learning projects require a large volume of data to be annotated. Mindy Support understands that this is a very time consuming process, which is why we are taking such work off the shoulders of our clients. We are one of the largest BPO providers in Eastern Europe with more than 2,000 employees in six locations all over Ukraine. Our size and location allow us to promptly source and recruit the needed number of candidates. Contact us today to learn about how we can scale up your project without sacrificing the quality of the annotation work Contact us today to learn about how we can help you.