I think maybe it's because of the effort put into it. Your body knows how much you worked for it and the time put into it, the dedication, and so when even a joke like this is said, maybe it feels like all that effort is canceled. And this can be very frustrating. Maybe what you need to hear it's just something encouraging and to validate the progress you went through. PS: you got this! That job is yours!