Over the last couple of weeks, my social media feed and also regular media I consume converged into a soup full of AI stuff. Articles, videos, artworks and even whole apps generated by AI popped up all over the place, all to prove how this technology might change our lives and may change our perspective of software development, arts and probably media in general.

Don’t get me wrong, I am not trying to fight that nor do I want to oppose the opportunities that deep learning or actual artificial intelligence will bring us in the future. However it is still very much in the future. While AI generated content is fun to watch and a good showcase of what might be possible, it’s really not yet fit for realistic use cases. Code generated by AI cannot currently be trusted the same way you would trust your IDE’s autocomplete suggestions while both look similarly trustworthy. As researchers at the New York University’s Center for Cybersecurity (NYU CCS) stated in their paper Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions, roughly 40% of the code generated by GitHub Copilot contains security vulnerabilities that are listed on MITRE Top 25.

And this, arguably, is a problem. See, I am that kind of ‚I want it all. I want it now‘ person. When I am looking out for ideas in the wild on how to improve my skills or facilitate my efficiency on a daily basis I don’t want to chase any rabbits. I don’t really get the point when e.g. my code assistant is ‘kind of working’ and generates ‘half bad’ stuff ‘most of the time’. For me it wouldn’t feel like a tool but rather a project on its own as I would have to tweak it and fiddle around with it all the time. But there is an even bigger issue we need to address – it is maybe the issue that makes us ready for AI and AI ready to be used.

The misconception about improvement

As a Technology Consultant and System Architect in my day job, I think it is just natural for me to focus on why the interest in AI in combination with software development is such a big deal. Just don’t expect me to be the right person to discuss AI art with.

First of all, I guess we should distinguish between the two greatest benefits/promises of using something like ChatGPT or Copilot as a developer:

• We need to write less boring or boilerplate code and thus improve our efficiency.
• We just need to have a basic understanding of software development and a good understanding of what we want to achieve. AI is filling in the gap for us.

 

While this might sound intriguing, it is still in the future 😉 and not the direction we want to steer to with software development or digitalization in general. Why not? Shouldn’t it be very much desirable to spend more time on important features than on ‘dealing with the framework’? Absolutely! But just having more time to do it does not mean you will pick up the pace automatically. The individual performance of a development team might suffer when there is not enough time to deliver, the catch is the question why there is not enough time in the first place? Is there a prioritization problem? Does the chosen development process induce impediments? Is it because the requirements aren’t clear? A dependency to another team? …However, I wouldn’t say those issues are likely to be solved by an AI soon.

The factor for pace and efficiency I want to focus on is the team’s experience. As a group of people working together but also in the individual skill of each and everyone. Skills that are built up over time and through spending that time e.g. tackling a challenging puzzle in code. When we leave things like that to AI, we will save time but this time would not have been a waste in the first place. I guess that is just the investment you have to spend to get better at something the more often you do it. Or in other words: the more muscle memory you build up over time, the longer you work with a specific technology or programming language the deeper you follow its rabbit hole until you finally achieve an inherent understanding.
I don’t think there is a shortcut to that and I am even more convinced that excessive use of AI code generation will eventually lead to a degradation of knowledge about ‚how the old timers did it‘.

This isn’t a bad thing to happen in general – we’ve actually seen new technology arising in other industries over the last decades many times. Take Computer Aided Design (CAD) for example. Technical designers that use CAD are still very capable of doing it the ‚pen and paper‘ way – however, they might nowadays favor using software instead.

This example just doesn’t work out when transferred to software development. Our industry is way too volatile and constantly changing for everybody to still be capable of, for example, writing something in assembler, or C, et cetera… you get the point. We all witnessed technologies come and go over and over but we tend to stack up upon the legacy. Taking the experienced advantages for granted next time. Where in reality we are utterly bad at preserving the cumulative knowledge we had just a couple of years ago. Or do you think that every developer can just explain what the little endian format means in terms of CPU instructions and why we still use it today? Without using stackoverflow. Oh and don’t ask me, I am no exception to that!

Furthermore, we are even worse in looking back at things that people before us achieved and learned to compare if we did actually improve e.g. in performance or efficiency. I mean, I just have mixed feelings when a modern frontend application takes as much time to build as the entire linux kernel from scratch. On the same hardware. I don’t want to blame frontend development and if you disagree with me, that’s fine. My statement is completely arbitrary and there are a lot more examples that could take its place. My point is that we tend to not optimize more recent, or more abstract, technology as we did in the past, because we don’t have to as computing resources, in many scenarios, aren’t that rare anymore.

So, what’s the twist then?

If we are too busy to structure and architect our code in a proper way but let an AI generate it, we stop hunting for deep understanding – while already leaving viable performance on the table by not taking fundamentals as ‚How does my code get executed on the CPU?‘ or ‚What does my routines and structure mean to my program’s memory?‘ into account.

Combine this and you get the stuff we train the current AI models on. But the task we are asking the AI to do is not about recognizing pictures of cats or dogs. Software can have a massive impact on people’s lives, economies, our environment and another million important things. If it is not clear already let me put it this way:

We, as a community or industry, need to comprehend how we can sustain improvement before training an AI to do it for us.

Training a neuronal network with code that contains a bunch of boilerplate, utility methods, security flaws, ordinary bugs or inefficient runtime behavior does not make it generate code without such in the end. We would end up with the urge to still deal with all of that, debug it, fix it, maintain it, improve it.

And if we stop being the real experts in the technology, we’re going to have an even harder time dealing with the code AI generates as time goes by. It will become more difficult to debug code that we are more and more having our issues to understand and follow along. Let alone that we might not be able to judge its security, quality and best practices anymore because we don’t have to meet our own measurements and metrics like we used to. But more importantly we might skip the entire discussion whether a certain metric or quality/design rule is viable or not. As AI won’t argue with you about it but just generate the code to meet the specification you asked for or it has been trained on.

So finally, let’s imagine we actually did it and noticeably improved our efficiency with the use of AI. What do you think we’re gonna do with that extra time then? I guess we all know that. It’s meetings – Just. More. Meetings… I am looking at you SAFe 😀

Cheers and thanks for reading this far. I know that I made some controversial claims and I would be happy to see some discussions starting from it.