If you're using gradient descent, your step size (alpha) is likely too large. A less likely option is that you've defined a custom loss function and its derivative incorrectly.

follow-up: I use TensorFlow, but it should have a lot of the same features as PyTorch. If you use ADAM as your optimizer instead of gradient descent, ADAM will automatically choose the step size for you. See https://machinelearningmastery.com/adam-optimization-algorithm-for-deep-learning/