Why AI code generators are producing more legacy code and what AI-driven software development needs now

AI code generation has taken the world by storm, but now, 2 years after the release of Copilot and ChatGPT, research indicates that code quality is actually decreasing worldwide, taking into question the actual usefulness of AI-powered software development. Let’s take a look at why this may be the case.

The current state of AI software development

Although the theory behind the large-language models (LLMs) that power modern AI code generation have already been around for a while, it wasn’t until GitHub released its Copilot platform in 2022 when AI code generation really took off. Still, in less than 1 year after its release, Copilot alone had been adopted by millions of developers worldwide and is currently responsible for generating billions of lines of code annually. 

That number is probably much higher considering the release of OpenAI’s ChatGPT a few months after the release of Copilot. Whereas Copilot targets software developers, ChatGPT enables virtually anyone to create software.

Based on one of OpenAI’s earlier models, the “codex-davinci-002”, researchers at Stanford University investigated the impact of AI code generation on code quality and found several bleak trends:

• Participants with access to an AI wrote significantly less secure code than those without access.
• Participants with access to an AI were more likely to believe the code they wrote was secure than those without access.
• Participants who trusted the AI more, and engaged less with the language and format of their prompts, provided code with more security vulnerabilities.

The study’s findings are consistent with the more recent white paper published by GitClear, analyzing more than 150 million lines of code produced between 2020 and 2023:

• The percentage of added code is increasing compared to the percentage of changed code (i.e. updated, moved or deleted).
• The percentage of code lines that are reverted or changed within two weeks of their production, is increasing compared to the pre-AI baseline.

AI code generators are making it easier than ever before to produce software, but we are now clearly beginning to prioritize quantity over quality.

Why AI code generators are prone to produce legacy code

The crux of the issue with AI code generators like Copilot and ChatGPT lies in their foundational mechanics. These tools are trained on vast datasets composed of existing code, much of which includes outdated or suboptimal practices that inadvertently get replicated in the new code they generate. 

This perpetuation of old patterns, combined with the inexperience of an increasing number of citizen developers, contributes to the rapid production of what can essentially be considered "instant legacy code"—code that is born with maintenance challenges and technical debt.

Moreover, the ease of generating code with AI tools can lead to an over-reliance on their capabilities, reducing the incentive for developers to deeply engage with the intricacies of their codebases. This detachment can result in a lack of ownership and a deeper understanding of the code, qualities that are crucial for maintaining software quality over time.

Another factor is the AI’s focus on functionality over security and maintainability. While AI tools can efficiently produce functional code, they often do not prioritize optimizing this code for security practices or future scalability. As developers push for quicker releases and updates, this oversight can accumulate significant technical debt rapidly.

How to increase the usefulness of AI code generators

To truly harness the power of AI in software development while mitigating its pitfalls, a multi-faceted approach is necessary:

Education and awareness

Developers must be educated about the potential pitfalls of using AI in coding. Understanding when and how to use AI assistance effectively can lead to better outcomes.

Quality over quantity

Integrate tools and metrics to assess the quality of code generated by AI and flag potential issues for human review.

Integration with development workflows

AI tools should be integrated as partners in the coding process, where they suggest rather than dictate solutions, and developers are encouraged to review and refine AI-generated code before finalizing it.

Prompt engineering

You can structure your prompts in such a way that the AI model will consistently produce more precise responses. See our previous article for more information on this topic.

Supercharge AI code production with Uniscale

Another way to make AI code generators more useful, is to feed the AI with structured context and domain information. 

Uniscale offers a sophisticated approach to integrating AI in code generation, designed to enhance the quality and relevance of the output. At the heart of Uniscale's innovation is the ability to feed AI with structured context and domain-specific information, ensuring that the code generated is not only functional but also meticulously aligned with project requirements.

With Uniscale Describe, developers can effortlessly create structured, high-quality functional product specifications. These specifications are automatically linked to our dynamic SDKs and comprehensive technical documentation, laying a solid foundation for accurate and effective code generation. This process ensures that the AI understands the context deeply, allowing it to generate more precise and applicable code snippets.

Uniscale Develop takes this a step further by serving as a plugin for your preferred code editor. It utilizes the SDKs generated by Uniscale Describe as context for your AI code generator, providing tailored prompts directly in your code editor. This integration ensures that every piece of code generated is a reflection of your domain description, maintaining high accuracy and utilization consistently.

This setup not only streamlines the development process but also significantly reduces the likelihood of generating legacy code. By ensuring that AI-generated code adheres to current standards and practices, Uniscale helps maintain and even improve the security and maintainability of the software over time.

By leveraging Uniscale’s comprehensive tools, organizations can harness the power of AI to accelerate development cycles without sacrificing code quality. This ensures that AI code generation aids, not hinders, developers aiming to innovate quickly and effectively.