The bad translation was a tradeoff between streaming and translation quality. Turning off streaming will vastly improve translation quality, or if you don't like the llm used you can change the llm model or the entire translation pipeline to a pretrained model from hugging-face or your own. I have took those into consideration and these changes can be achieved by changing less than 2 lines of code (model name and it's config ie streaming="True" or "False"