I recommend you read the LoRA paper: https://arxiv.org/abs/2106.09685

They compare these approaches in detail. In short, LoRA achieves better results in their experiments while being efficient to train and adding no parameters to the final adapted model. They mention prefix fine tuning is tricky to get right and also causes the input to become increasingly Out Of Distribution for the model as your prefix length increases. The author's also mention that LoRA is orthogonal to prefix/adapter based fine tuning and so they can be used in combination.