Have you ever wondered how far you can go with AI-assisted software development before things start to break? I did. And as with every good quest, there was a challenge, a path, a few obstacles, and some painful lessons along the way.

In this post, I will share what I learned from experimenting with AI-assisted software development and Codex. The challenge was simple: Can AI create a working system for scheduling meetings, similar to Calendly? Simple idea. Not-so-simple execution.

Experiment 1: “Execute and everything will be okay”

The assumption for the first experiment was optimistic: AI can do almost everything with little oversight and limited context. I started the journey by trying to speed up the requirements definition. I asked ChatGPT to help me design a system similar to Calendly. I gave it basic requirements and, to move faster, I asked it to analyze the Calendly website.

ChatGPT analyzed what needed to happen. Then the first roadblock appeared. What should I do next? I decided to take on the role of the AI development agent and asked ChatGPT to provide me with prompts that I could use to build the system step by step.

ChatGPT came back with 20 prompts. I thought to myself: “So far, so good.” Now I only had to execute them sequentially, and my dream application would appear. Easy, right? Not exactly.

I started feeding the prompts to Codex one by one. First prompt number one, then number two, and so on. Very quickly, I noticed two things. First, ChatGPT was also suggesting that I test the outcome to make sure everything worked. Second, after a few prompts, the context overflowed and had to be compacted by Codex.

I still remember what I thought when I saw the context reaching the limit: “That can’t be good.” But I continued anyway. Prompt after prompt. After about an hour, I reached the end. All prompts were posted, and all commands were implemented by Codex.

Then came the moment of truth. Would it accomplish what I imagined? Would a few system requirements and generated prompts from ChatGPT be enough to produce a fully functional system? If your answer is “No,” you are correct.

There was supposed to be a database. The database was not configured. And that was only one of the issues. There were many more.

Outcome: Using generated prompts with vague requirements gave me a 0/10 outcome. I do not recommend this approach to anyone. The lesson was clear: if the requirements are vague, the output will be vague. If the oversight is missing, the problems will grow quietly until they become very expensive to fix.

Experiment 2: “He who fails to prepare, prepares to fail”

After the first failure, I decided to improve my approach. Not everyone is aware of the planning functionality in Codex. And even people who know about it do not always use it, which is a pity.

For the second experiment, I decided to start with planning mode. I reused some artifacts from the first iteration, but this time I gave Codex more context, more commands, and more constraints. I also adjusted the plan before Codex executed it.

This already worked better. Adding more constraints and guidelines helped a lot. Adding architecture diagrams inside the project helped. Having a solid AGENTS.md file helped.

For this experiment, I tried to do what a true AI Engineering Leader would do: prepare the context, feed it to the AI, guide the implementation, and review the result. There was no more manual copy-pasting of 20 prompts. This time I used architecture diagrams, database expectations, and even architecture decisions. That was enough for AI to generate a useful MVP.

Was it perfect? No. There were still many things I did not like. For example, it tried to copy the original site too closely, which did not make sense for my use case. The Google API integration still did not work. There were other serious issues as well.

But this experiment proved something important to me: with better preparation, you can achieve something useful with less oversight. Not zero oversight. Less oversight. That difference matters.

Outcome: The site worked, even if it was not what I was hoping for. At least I could manipulate the site and interact with it. I would rate this as a 3/10 outcome. Again, not the best approach.

Funny fact: I added a few meetings, and when I restarted the system, I realized there was still no real database. Everything was in memory. My immediate thought was: “What is wrong with AI and databases?”

Experiment 3: From Padawan to Master

For the third experiment, I decided to improve again. I created a new directory and started from scratch. This time, I used a more advanced AGENTS.md file, better architecture, better constraints, and my own UI image template. I wanted to leave much less room for interpretation.

Again, I asked the AI to implement the entire system based on the available information. The results were much better. Way better. Do not get me wrong, this was still not production-ready code. But the skeleton of the application was much more aligned with what I wanted.

The structure was better. The UI direction was better. The outcome was more predictable. Most importantly, I could see the difference that better preparation made.

Outcome: This experiment reached a 5/10 outcome. Still not great, but now we were moving in the right direction.

What I Learned

AI is not a magic bullet. If you give it bad input, you will get bad output. AI is not a mind reader, so do not rely on it to understand everything you forgot to explain.

If you are an expert builder and you understand how to guide AI, you can become very productive. But if you are lost, AI can make you even more lost. And there is another risk: you may not only waste your own time, but also the time of other engineers who need to review, understand, and fix the bad code generated from poor instructions.

AI does not remove the need for engineering discipline. It increases the importance of it.

You still need requirements, architecture, constraints, testing, review, and judgment. The better your preparation, the better your AI-assisted development workflow becomes.

Final Thoughts

My quest to improve the AI SDLC continues. So far, the journey has shown me that AI can be a powerful partner, but only when treated properly. Not as a magical senior engineer. Not as a replacement for thinking. But as a very capable assistant that needs clear direction, good context, and regular checkpoints.

The more I experiment, the more I believe that the future of AI-assisted development will not belong to people who simply “ask AI to build things.” It will belong to people who know how to structure the work, prepare the context, and evaluate the output.

Disclosure: no engineers were hurt during this experiment. Also, I did not ask anyone to review the code. That was probably better for everyone.

What other AI SDLC experiments would you like me to perform?