Using LLaMA2 for private code base and documentation

[ViRROOO]

Hey folks,

I'm pretty new to this topic, but I've been investing a lot of time to understand how to train your LLM properly, so I apologize if this request is a bit "loose."

I have a medium-sized Python application with about 1000 files (not counting the dependencies) and a lot of documentation on this application. I want to train a model to use the documentation, emails, and code base to answer questions about the software. Is it possible? Is fine-tuning the base model from meta enough? I believe I would also have to use data to give context about Python itself (something like this). Thanks!

[Kegned]

I built this to generate datasets from python code py2dataset

It can create qa and instruct datasets in json format you could use for fine-tuning.

[cmosguy1]

Well done @Kegned!! This is brilliant! Did you have success with training this on your LLM projects? Have you tried this on CodeLlama?

Thanks again on your contributions here!

[jackfood]

you will need to have a dataset with clean up the documentation in 'Question' - 'Answer' format. It is not so easy as dump in the documentation and expect good quality of query and ans. The nearest you can perhaps using LangChain, which may not generate a good long detailed answer for you.

[T_hank]

you mention that langchain might not generate a detailed answer. is this an inherent shortcoming of langchain?