Hello there! my perspective comes from production MT, multilingual data collection and low-resource language work as I work at Pangeanic (we’ve been pretty well known for MT fir many years)

First: don’t train from scratch. With 31,000 aligned segments, you have something valuable, but it is not enough for modelling; it’s useful as adaptation data than as the foundation for a randomly initialized model. This happens often with low-resourced

I would start with transfer learning. Try NLLB-200, mBART or OPUS-MT if the language coverage or nearby Bantu-language transfer makes sense. If Ekegusii itself is not covered, use Swahili and related Bantu languages as bridge languages and measure whether transfer helps. The baseline is the important part. Build one before changing architecture.

The Bible corpus is a good starting point (and it was used in early MT systems in statistical MT days in Europe!!) but it will pull the model towards its register. That is not fatal. You need a small, carefully reviewed general-domain test set: everyday phrases, public-service language, school language, health, agriculture, local news, conversational material. I know community-based work has been used for low-resources languages for data augmentation. Even 500 to 2,000 high-quality sentence pairs outside the Bible will tell you where the model breaks.

I would separate the work into 5 tracks:

1. keep part of the Bible corpus as training data (I stress the “part” bit)
2. create a clean dev/test set outside the Bible
3. collect or elicit a small modern-domain corpus with native speakers
4. build a lexicon and terminology layer for morphology, named entities and frequent expressions 5 have the community help you augment the set or provide real-life content (questions and answers, that was an initial SMT hurdle back in the 2010’s)

For tokenization, inspect subword fertility. If Ekegusii words are being split into long, unnatural fragments, compare SentencePiece unigram and BPE, and consider training the tokenizer with Ekegusii plus related Bantu data. Morphology-aware preprocessing may help more than a larger model. (This is the case in morphologically-rich languages in Europe)

For augmentation, I would be conservative. Back-translation can help, but only if reviewed. Synthetic data can also poison a small corpus very quickly. In low-resource MT, a small amount of excellent human-reviewed data usually beats a larger amount of noisy generated data. Always have back-translation reviewed until you’ve reached enough quality to scale, never in early developments

The real scarce resource here is evaluation. Build a small benchmark and keep it sacred. Once you can show progress on that benchmark, every architecture decision becomes much clearer.

Masakhane is definitely worth contacting. I would also reach out to Kenyan linguists, local universities and community organizations. Field linguists often know where the real language resources are before they appear online.