We recently had someone come to us with a software project they wanted implemented. They had used replit (an AI tool) to create some screens. They were savvy enough to understand that this was not code that would work even for a Minimum Viable Product. However, the experience of generating slightly working mockups with AI prompts skewed them to presume it would be trivial to build. (By "slightly working" I mean that it may allow some editing of data on some screens, while other screens do not function, and few edge cases are handled.)

After talking with them and looking at their screens and features, we made a recommended list of which features we thought they should include in their MVP, where the goal was to stop using a cobbled-together solution consisting of documents transferred on Google Drive and a SaaS product that didn't really fit what they were doing. They were interested in getting something done in a month, and we sized something that could in fact be delivered by one developer in just over a month.

They responded with a much larger list of features they considered minimum. To give you a flavor of this, one of the features was a "comprehensive calendar management system" that included two-way synchronization with both Outlook and Google calendars as a minor sub-element. Their suggested MVP scope was in the neighborhood of 1-3 person-years of development, depending on how you might interpret the details. At the bottom of the document was an estimation of "about two weeks" for planning and design, followed by "about two weeks" for development.

Now, it's common for people without software experience to assume that software is easy to build. It's also common for people to have a hard time paring back their MVP feature set to something that is truly minimum. However, the hubris of a non-developer assuming a large set of features (the feature list document was 17 pages long) could reasonably take only 2 weeks to develop is new.

I think what's going on here is that the emergence of tools like replit and lovable which deliver slightly working mockups via AI based on vaguely described English requirements is driving a new level of expectation that AI cannot yet deliver. Historically, "no code" solutions actually end up requiring some level of logic (regardless of whether you consider it "code") and a moderate amount of training in how to use the tool. Those tools can produce more finished and useful products than the current generation of AI code generation.

AI is a wonderful tool that all our staff use on a daily basis. And it is more useful than it was a year ago. But it's not yet a magic wand that radically increases developer speed. What we have experienced, and what many others have experienced, is that AI is particularly useful in areas where the developer has little experience, or in writing small snippets of code or small functions.

For example, over the last 6 months, I've only had AI generate a large chunk of code (30+ lines) I could use substantially unchanged in three instances. One was a datetime parsing function in C. One was a datetime parsing function in PHP. And one was generating a javascript routing function using Express. These are all very common activities that the AI has a large body to draw from. I'm still amazed that AI can generate working code and that it now almost never makes a syntax error, but AI is not replacing developers yet.

Also, while I regularly use AI for search now because it picks up and interprets multiple web pages quickly and can be a time saver, it has sometimes led me down the wrong path. For example, in September I was trying to get Azure set up to send email. Both Claude and ChatGPT insisted that I needed to create a three-part identifier for the SMTP username, even though in the user interface I was creating an "SMTP username" with a textbox to enter that value. ChatGPT claimed it was just misleading UI. I asked ChatGPT to triplecheck its sources and show me exactly which URLs it was using for documentation of this three-part identifier idea. When I read the URLs provided, their interpretation was NOT justified, and in the end it turned out they were wrong. Now, the documentation was not wonderful, which was why I was asking AI in the first place. But nowhere did it describe the three-part identifier that AI was claiming I needed to use. My best guess is that it was from some alternate or older method that it picked up somewhere else.

AI is moving quickly, huge amounts of money are being spent trying to replace programmers with AIs, and various startups are claiming great success. But at this point, you should still be skeptical of great claims and assume small to moderate gains (eg 10%) development savings if you decide to have developers use AI to help develop your project.