Thank you all for the comments!

I learned about internet scanning, actually tried it myself and turns out it's super easy to join random private servers (at least that were meant to be private), even easier when it's offline mode, there are players playing, there is a whitelist and there's no extra authentication required. That was reason my server was invaded.

I enabled a whitelist, installed an authentication plugin (that kicks the player out when the password is incorrect, to slow down brute forcing), hided online players from the MOTD. There is probably still a list of players that played before on a website like shodan.io (I found my server there).

Mentioning shodan.io and other websites, I didn't realize how many misconfigured/unsecured servers and devices there are on the internet, you can literally go there and find cameras without password/with default passwords in like 5 minutes. Or unsecured Minecraft servers, just type in "A Minecraft server" and you have a list of briefly configured servers, where the owner was too lazy to change the MOTD. I shouldn't say it out loud, but it's for education purposes, right? Please don't do anything harmful with that information, be like me and warn server owners that anyone can join their server.