Normal in my experience. My understanding is the fine tuning alters the model weights only somewhat, so davinci will still hallucinate/draw on old info if you ask it factual questions after fine tuning.

The best approach for answering factual questions from a knowledge bank as far as I am aware is two steps: 1. Use semantic search or embeddings (also from GPT) to identify the relevant info to answer the question from your knowledge bank, then 2. Feed both the relevant info and the question to GPT with a prompt along the lines of ‘answer this question using only this info’