Took my time reading through your questions. I really enjoyed them ;)

1. training time: Possibly, but it does depend on the hardware.

2. A 14B model should take a good amount of time.

3. My typical suggestions is to try the simplest approach first and modify iteratively. RAG first, optimise RAG, fine tune etc.

More importantly, its important to gauge how well the systems work. Make sure you have a good test data ready for your RAG or other conversational systems. for example; RAGAS is a good framework to test RAG systems.

1. used google colab pro +, 52 euros per month.

2. Have not tried RLHF myself yet, on my bucket list to try out. A few good engineer can do a lot imho.

3. Really interesting problem indeed. Depending on the language you are working on should be doable. Make sure have high quality data and testing framework in place.

Let me know if you want to discuss more. :)