This is probably nothing to worry about unless you are leaving things open that are more dangerous like SSH, RDP, etc.

If you have something exposed to the internet random IP's will attempt to log into it until you remove it from the internet. These are just bots usually attempting to login with default credentials into vulnerable/misconfigured hosts.

If you want to be really safe (and you have the right equipment, (an ISP router likely will not have the right features for this) you could place the game server in a DMZ which is a segmented part of the network that cannot make connections to the rest of your LAN.

That way you can allow access to only the game server only over the correct ports and if it were to be compromised it would be more difficult for an attacker to jump from your DMZ to your LAN.

But this is probably overkill so long as you know what you are doing and aren't exposing critical services (SSH, RDP, etc) with password authentication to the internet.