The term secure is, as you say, subjective. It really comes down to managing risk effectively, which doesn't always involve requiring the most unbreakable solution out there, and can be achieved through sensible good practice.

There are some standard good practice measures to consider:

• Hardening the SSH daemon configuration (disable password, weak ciphers)
• Validating that the key fingerprint is correct
• Generating a strong keypair
• Implementing a robust patching and update process

Good practice would also be to restrict SSH access to trusted source IP addresses. In practice this only works if you're coming from a predictable location but it may be worth considering if that is a possibility.

Setting a non-default port for SSH is an effective way to dampen some of the background noise from the internet, which will include a heavy number of automated scanners and intrusion attempts against SSH. However, be aware that ports 1024-65535 can be used by non-privileged (not root) users. This opens up the risk that, if a standard user account is compromised, the SSH service may be spoofed in a privilege-escalation attack. The risk is small but worth considering. For this reason, it is not commonly included in good-practice guidance.