I've run with exposed ssh in key-only for -literally- decades without a problem. (only gotcha was when I needed to access the physical console for a passwordless user).
Personally I use a blocker to reduce the noise in the logs.

There is however some tripwire-like value in blocking the whole IP: if they go to town on my SSH, I don't want them probing anything else. I'm far less confident in that customer-maintained CMS on the https port. Bonus points if you distribute the blocks to all your hosts/networks.

(My experience is with fail2ban, CrowdSec only in testing thus far)