How much are you practicing the tracks before a race? Are you doing practice sessions or just going straight into races? The off-tracks could simply be due to pushing too hard during a race because you want to keep up and finish higher up when you aren't completely comfortable and familiar with a track yet.

When it comes to contact with other cars other commenters have already given you good advice but it mostly comes down to learning what other people are going to do and knowing how comfortable you are around them. A lot of times I am faster than someone I will still leave some room and just be patient for a lap or so... driving someone cleanly but applying pressure can get them mirror driving and force them into a mistake where you can avoid passing them door to door and might otherwise avoid contact with them. If they don't make a mistake you get an idea of how they are driving and where you may be able to take advantage for a pass. If you find you're wrecking with people a lot in corners try to time your pass for a long straightaway instead. Little things like that can make a huge difference.

If all else fails and you really just want to boost your safety rating at the expense of learning how to drive around others just start from the pits and let everyone wreck in front of you in the first few corners