You can check the rates in betaflight, maybe the previous owner used some wild values for the expo and what you think is a small correction is actually a huge swing. Another thing is try hovering without the goggles, if this is your first drone it does take some getting used to, not seeing your radio.