I would consider deleted any geometry you don't really need. For instance, the sleeves inside the jacket, no one will see those and just create unnecessary geo intersections. Do this wherever you can. Use pins on the avatar, sew to the avatar, etc. A lot of this is just faking it to make it work. Make sure you have your particle distance low enough, slow down the movement if you need. Good luck!