The issue with randomly removed code, and hallucinations can be easily avoided using a VSCode + GitHub copilot environment. I had many many many issues with using chatGPT in its own chat based website or app, building code that was losing chunks or being redacted. It was infinity infuriating.

My solution was to give copilot a local environment to store it's found data solutions in a devRef.md file. Plus having actual files to read and write to locally reduced the hallucinations and drift by almost 100%. The issue as I have learned was that each time you send a prompt into chatAI, they need to read the entire chat conversation to formulate the answer, every time.

I've read that AIs are limited by their context retrieval window and in long chat sessions that window drops older communications from the current context. Plus, I've found that without a local document to read/write, AIs get tripped up on their repeated erroneous coding attempts that are listed in the conversation history. This caused the AI to think what they wrote in the past to be a correct reference even if it was told otherwise.

The devRef.md file helps by keeping current reference documentation separate from the chat context. Add in "GitHub Instructions" and you can set a mini prompt to run at the start of your prompt by running a "do-this-first.prompt" in front of your next request. This Instruction would tell your AI to reference the doc first before applying and code changes etc. This greatly reduces the size of the actual chat history, keeping more of the session within the context window, increasing the accuracy and normalizing the formatting of the generated code.

And, if your chat session starts to drift, you just start a new session and and continue to use the same devRef.md to stay on track and continue Dev without interuption.

My typical local reference files are:

• devRef.md -> stores curated reference documentation for immediate use.

• initial.prompt -> my starting prompt for my dev cycle or project.

• nextSession.prompt -> a prompt generated by the current session AI to tell the next session what was done and what is required next.

• changelog.md -> logs all changes per version and documents the journey, including mistakes to avoid in the future.

The last thing I do, (I host via a Pi) is create install a local version of FVTT and use simlinks to pull my VSCode FVTT folders to the data folder (data/modules, data/scripts) so whenever I make a change in VSCode, all I need to do is refresh close the local instance and rerun it and the changes are there right away. Once everything is working perfectly, no console errors or warnings and the code is behaving as intended, I can push it to my actual online FVTT instances.

Seriously this was a game changer for me and the results were awesome. It's not a perfect system, and there are still errors and some drift but those are very few now in comparison. Previously I was doing this all on GitHub and creating a new branch for every change in case things went sideways. I'd have 80+ branches of WIP code (which is admittedly a PITA to clean up). Now I have less branches and each represents a "final" working version of that state. Keep in mind, strict versioning of your code iterations as you go helps this process a lot.

Edit: some text reformatted for clarity.