The (AI) Canary in the Coal Mine

Here is an interesting question: If AI is truly so good at creating code, why don’t we see many more code contributions in Open Source repositories made with the help of AI?

Here is Satya Nadella, CEO of Microsoft:

“Maybe 20 to 30 percent of the code that is inside our repos today in some of our projects is probably all written by software.”

That’s a lot of “maybe” and “probably”… And we just can’t know, as Microsoft’s code repository is (of course) private. But Open Source code lives in public places like GitHub and thus is inspectable. And when you look closely, you will find very little evidence that code in those repositories is written by AI.

Admittedly, a lot of Open Source projects aren’t particularly excited about AI-generated pull requests:

“It’s true that a lot of open source projects really hate AI code. … the biggest one is that users who don’t understand their own lack of competence spam the projects with time-wasting AI garbage.”

But that aside, when you look at the data, it’s just not there:

“TL/DR: a lot of noise, a lot of bad advice, and not enough signal, so we switched it off again.”

In many ways, AI keeps furthering the skill gap:

“The general comments … were that experienced developers can use AI for coding with positive results because they know what they’re doing. But AI coding gives awful results when it’s used by an inexperienced developer.”

Overall, a good reminder to look past the marketing and hype…

Here is the full article.

Pascal Finette @radical