The major way AI can help programmers the most—and its limitations

Close up of a computer programmer’s hand programming with computer code floating around screen.

TippaPatt // Shutterstock

How AI can—and can’t—boost efficiency for programmers

Artificial intelligence took the world by storm when OpenAI launched ChatGPT in November 2022.

Although its creators didn’t tailor the product for any specific purpose or profession, the simple chatbot quickly proved a nimble and capable tool for countless tasks. People could use it to conduct research, write school essays, and even generate love letters in the voice of celebrities. The tool enamored people around the globe.

But, for many professionals—particularly computer programmers—OpenAI presents both a career advantage and a professional threat.

AI naturally lends itself to some computer programming tasks, like code documentation. Although many leading AI models were not explicitly trained in software development, their users quickly discovered these tools could be as adept at programming languages as human ones.

Trends in programmers’ use of AI are still emerging. Verbit analyzed data from various surveys to see how AI is changing software developers’ work.

Despite varied survey results, one thing is clear: Programmers are adopting AI tools at a dizzying pace. A March 2023 GitHub survey of 500 U.S. developers found that the vast majority, 92%, use AI coding tools both in and outside of work.

Taking a broader view, just over 3 in 4 programmers used ChatGPT, according to a JetBrains survey of some 26,000 developers from around the world. Nearly half of those surveyed used GitHub Copilot, a programming assistant similar to autocomplete that launched in 2021. Survey data from StackOverflow in 2023, which polled programmers from 10 countries, including the U.S. and Canada, found that 44% of professional developers used artificial intelligence for work.

A chart titled, Artificial Intelligence Makes Programmers Faster, showing how much time it takes programmers to typically complete tasks, including code documentation, code generation, code refactoring, and high-complexity tasks, compared to how much time it takes with AI. AI is most helpful when it comes to simpler tasks.

Verbit

Who benefits the most?

McKinsey conducted an experiment with over 40 of its programmers, asking them to complete tasks of varying difficulty levels in a lab. They found that developers who used AI for a simple task, code documentation, finished their work in around half the time it took their peers who did not use AI. Artificial intelligence was far less effective at helping complete more complicated tasks, only saving developers about 10% of their time to finish their work compared with their colleagues who did not.

Another study, published by researchers at Microsoft, GitHub, and MIT, analyzed the impact of a particular AI tool (GitHub Copilot) on the productivity of freelance programmers. On average, programmers who had access to Copilot completed their work about 55% faster than those who did not.

However, not all of the workers benefited from AI assistance equally. The researchers found that less-experienced workers, older programmers (between 25 and 44), and those with heavier workloads were much more likely to benefit from using AI than their peers.

Programmer working on code on a computer.

Gorodenkoff // Shutterstock

How far will AI go?

Despite AI’s added value in expediting repetitive or mundane tasks, many programmers remain skeptical about its broader implications. JetBrains data indicates that 59% of developers have security concerns about generative AI, while 42% agree that its use presents ethical challenges.

Artificial intelligence tools can also be unreliable, suffering from the same confabulation problem as chatbots. AI’s tendency to generate complications of facts that can be misleading or downright false might be why AI has yet to earn the trust of developers. Less than 3% of developers say they “highly trust” AI, while about 39% say they only “somewhat trust” AI, according to the StackOverflow survey.

Scores of new startups are trying to push the technology forward. Devin by Cognition Labs is one product piquing the interest of investors and engineers. Advertising itself as “the first AI software engineer,” the company boasts that its software can beat other leading AI tools at meeting computer programming benchmarks and completing software engineering tasks with minimal supervision. However, the product hasn’t launched yet, and because it relies on the same technology that powers ChatGPT, it could share many of the same weaknesses regarding complex problem-solving.

Despite suggestions that AI will trounce human capacity, software developers are generally optimistic about AI’s impact on their work. According to the JetBrains survey, 60% of programmers believe AI coding tools will dramatically alter the job market for software development, while just over half think AI will boost demand for human programmers. Only a fraction, 13%, believe AI will completely replace human developers.

The technology behind many artificial intelligence tools is advancing at lightning speed. Though speculation on the future of AI abounds, there are still more questions than answers about what’s next. For now, though, computers are augmenting human intelligence—not replacing it.

Story editing by Alizah Salario. Copy editing by Kristen Wegrzyn.

This story originally appeared on Verbit and was produced and distributed in partnership with Stacker Studio.