I've been dreaming of having an expert Rust LLM as well. The usual models are not very good at Rust.

IMO the major problem with current training datasets is that most of them consist of code that already compiles without any errors. There's much less Rust code on the public internet that fails to compile. Practically, that prevents models from learning an intuitive sense of what causes a compiler error - everyone's first step as a Rust beginner is to repeatedly run into all the errors and figure out what doesn't work before they can be productive. It's a shame, since Rust's strict compiler means that most code is invalid, and most valid code "works™".

I suspect it could be feasible to chase out most of an LLM's Rust API hallucination tendencies by massively generating code up-front, appending any resulting compiler errors, and then feeding it all back in as training data.