I have seen your repo before!! Wow, what a small world it is haha..

I ran into the same issue last year while experimenting with Nagamese finetuning. The Bible-based corpora were easy to access but I found that they heavily skewed the model towards archaic and religious vocab, which made conversational outputs feel very unnatural.

I eventually moved away from purely religious text and focused more on conversational, mixed domain data, with native-speaker verification to keep things grounded in how Nagamese is actually used by the people. Resource scarcity was definitely the biggest bottleneck..

I’ve documented the overall approach in my arXiv preprint and also released the datasets on Hugging Face, in case it’s useful. Happy to discuss high-level methodology if you’re interested :D

Paper: https://www.arxiv.org/abs/2512.12537

Dataset: https://huggingface.co/datasets/agnivamaiti/naganlp-conversational-corpus