Generative AI: Silver Bullet or Lead Balloon?

Abstract

Does GenAI improve developer productivity? Does it affect the cognitive skills of developers? In other words, if developers no longer need to develop much code manually, but rely on GenAI tools to generate the “boring” bits, will they lose these skills? The goal of this project is to investigate the impact of GenAI on software development.
public 4 months ago 3 months ago BAPC

Description https://project.cs.ucc.ie/project/1603

Frederick Brooks in his famous essay “No Silver Bullet—Essence and Accident in Software Engineering” that there are no silver bullets that can address the essential complexity in the development of software.

Brooks differentiated between “accidental” and “essential” complexity. Essential complexity refers to the creation and design of complex conceptual structure that compose the software. Accidental complexity refers to the representation of those entities in programming languages and the mapping of those onto machine languages within space and speed constraints.

In other words: there are “accidental” complexities that are rather arbitrary, for example the complicated syntax of some languages, or the constraints imposed upon developers by tools. Essential complexities are those that inherent to software. This refers to the design, including data structures, relationships among the various data items, algorithms and functions.

Of course, a lot has changed in the decades since Brooks’s famous essay. There have been many breakthroughs that have addressed complexities, but they all appear to be the accidental complexities. There doesn’t seem to be a “silver bullet” that addresses the essential complexity.

Many silver bullets have been proposed: modern languages (at the time Ada); object-oriented programming (OOP), and, “artificial intelligence” (see page 190).

Enter Generative AI. Is GenAI the silver bullet that can address the essence of SE?

Project idea Brooks focused primarily on developer productivity, and at the time that was expressed in lines of code, whether that’s assembly or higher level programming languages. Other measures of productivity have since been proposed, which may or may not be considered in this project.

Does GenAI improve developer productivity? Does it affect the cognitive skills of developers? As the mantra goes, Use it or Lose it. In other words, if developers no longer need to developm much code manually, but rely on GenAI tools to generate the “boring” bits, will they lose these skills? In other words, what will be the effect of GenAI on developers?

Is GenAI the Silver bullet that software engineering have waited for? Or is it a lead balloon that will sink future maintenance efforts. This is the question I’d like to investigate. All generated code will have to be maintained at some point – maintenance taking up as much as 70% of the total cost of a system in its lifetime. Maintenance is related to program comprehension, which in turn links to theories of understanding. There may be useful theories in psychology that could be drawn upon, e.g. as a theoretical framework.

Project goal

The goal of this project is to investigate the impact of GenAI on software development. Several angles can be taken, and several “variables of interest” can be focused on; productivity is just one, but others can be considered, such as the cognitive skills of developers as mentioned above.

I envisage a few different activities. A literature review will be necessary. Given the very rapid pace at which new studies are published, this will be a major part of the work, but at some point, with a sufficiently sharp focus, a cut-off and scope can be found. The idea is to conduct empirical studies, e.g. survey studies among either professionals or students or both, as long as they are programmers. It could also involve a “case study”, though that would require company access – which is a possibility, but would have to be discussed and explored with the PI (KS).

FP Brooks. The Mythical Man-Month. 20th anniversary edition, 1995. Addison Wesley.