The Digital Ninja

Escaping the AI Feedback Loop: Tips for Devs

2024-07-175 min read
aicodingtroubleshootingbest-practicesprogramming

I wanted to write a post about a common theme when coding with AI tools such as Claude, Gemini, or ChatGPT. That is the AI sometimes gets stuck in a repetitive loop where the AI gives you wrong code and when you correct it, it gives you the same code again that you originally gave it. Often this kinda snowballs and poisons your context window.

Preventing the Loop

Ideally the best way to stop this is to keep it from happening in the first place. Now I know you might be thinking how can you stop it from doing it in the first place and I'm not talking about prompt engineering. Rather you can go back in time using most AI tools and edit your previous "user message" to better guide the AI to avoiding the mistake. This might be reminding it that the component needs to be a server side react page or vice versa.

Editing a message in Claude AI

The Impact of Context

While many AI's models these days have decent context windows, having broken or incorrect code in there just clutters it up, pushes some of your old context out, and from my experience increases the chances of it getting it wrong again as its context is filled with more and more mistakes.

Claude AI's response after editing a message

Caution with Editing

Do be careful of editing too far back as this will most often delete all the responses since then and if you are using the API's directly to modify the message but not future responses it can cause the models to start hallucinating in wild ways.

When to Turn to Google

All that being said I still find the fastest and simplest solution is just to go back to google the moment the ai starts going sideways at all. It can often feel like it would be slower to tab away from the AI tool and go hunting though google, but I find it to be quite the opposite. You may feel like you are too far into a solution but often when your eyes land on that stackoverflow answer you immediately can see the problem you AI missed.

Conclusion

In conclusion try to steer the AI by editing your messages and adding additional instructions but don't forget your roots, all of this was built by google and stack overflow in the first place.

Stay Connected

Follow me on social media for more tech insights and updates!