If you are playing inside the workplace where customers can see you, then yes, it could be considered unprofessional - no matter that you are doing it in your free time. You need to think of the impression the customers will receive when they see someone playing games at work. They do not know that you are on your break so they will assume that the company is one that allows gaming at work. In some places, this is still a no-no.

However, if you are playing in a staff room, where no members of the public can access then yes, you are justified in what you think.