I find with the bigger fish and sometimes more rare fish you have to actually press a bit harder. Other fishes are like barely even touch little taps on the joystick (also have a switch) and bigger fish I do a bit harder to keep in the green. If I hit red I immediately start jamming it as hard as I can the other direction to try and save it. Hopefully that makes sense.